Dr. Memory
|
This section is divided into the following subsections:
Distribution Contents
The Dr. Memory distribution contains the following:
- The front-end executable
drmemory
for launching Dr. Memory. - The back-end of Dr. Memory, contained in the
bin
anddynamorio
subdirectories. Dr. Memory runs on top of the DynamoRIO tool engine (see http://dynamorio.org for more information on DynamoRIO). - Documentation for Dr. Memory (you're looking at it).
- System Call Tracer for Windows
- Symbol Query Tool
Changes Since Prior Releases
The current version is 2.3.20005. The changes between 2.3.20005 and version 2.5.0 include:
- Added glibc 2.34+ support on Linux.
The changes between 2.5.0 and version 2.4.0 include:
- Added better callstack walking on Linux via -callstack_use_unwind which is on by default.
The changes between version 2.4.0 and version 2.3.0 include:
- Renamed the following options (the old option names will still work for compatibility):
- -lib_whitelist is now -lib_allowlist
- -lib_whitelist_frames is now -lib_allowlist_frames
- -src_whitelist is now -src_allowlist
- -src_whitelist_frames is now -src_allowlist_frames
- -lib_blacklist is now -lib_blocklist
- -lib_blacklist_frames is now -lib_blocklist_frames
- -check_uninit_blacklist is now -check_uninit_blocklist
The changes between version 2.3.0 and version 2.2.0 include:
- Added preliminary 64-bit Mac OSX support for small single-threaded applications.
- Added recent Windows 10 support to the drstrace and drltrace tools.
The changes between version 2.2.0 and version 2.1.0 include:
- Fixed a problem with Windows 10 1903.
The changes between version 2.1.0 and version 1.11.0 include:
- Added preliminary 64-bit full mode (i.e., with uninitialized read checking) support.
- Added preliminary support for the Windows Subsystem for Linux environment with the regular Dr. Memory Linux package.
- Added automated generation of system call information on Windows, enabling running on new releases of Windows 10.
- Added support for Windows 10 1703.
- Added support for Windows 10 1709.
- Added support for Windows 10 1803.
The changes between version 1.11.0 and version 1.10.1 include:
- Added support for Windows 10 1607.
- Added support for loading system call numbers from a file, to make it easier to update for future major Windows 10 updates.
- Added an option -ignore_kernel to attempt to continue on even when running on a kernel with unknown system call numbers.
- Added support for Mac OSX El Capitan.
- Better international string support.
The changes between version 1.10.1 and version 1.10.0 include:
- Fixed problems with configuration directories on Android.
- Fixed several bugs, including Windows false positives, ARM instrumentation issues, and VS2015 crashes.
The changes between version 1.10.0 and version 1.9.0 include:
- Added preliminary 64-bit Dr. Memory support for Windows and Linux (but not MacOS yet). This does not yet include uninitialized read checking, so we still recommend compiling your application as 32-bit.
- Added preliminary Linux/ARM and Android/ARM 32-bit Dr. Memory support. This does not yet include uninitialized read checking.
- Added support for Windows 10 1511.
- Fixed incremental error output to the console for graphical applications, which was missing in prior versions when using certain consoles such as cmd.
- Removed the -fuzz_mutator option and replaced it with separate options for each component of the mutator.
- Added custom mutator support to Dr. Fuzz.
- Changed the mutator option passing scheme for Dr. Fuzz to take in an argv-style array of parameters, to better support custom mutators.
- Added checking that the program counter points to valid memory, controlled by the on-by-default -check_pc option.
- Added a new option -malloc_callstacks for recording malloc callstacks to include in alloc/free error reports when leaks are disabled. The option is off by default due to the additional overhead that it incurs.
- Added code coverage measurement via a new option -coverage.
- By default, without the
-show_reachable
option, reachable leak callstacks are no longer compared to suppressions, in order to reduce overhead.
The changes between version 1.9.0 and version 1.8.0 include:
- Added support for Mac OSX Yosemite.
- Added Dr. Fuzz, a fuzz testing framework that facilitates repeated execution of any program function while mutating the arguments.
- Added a fuzz testing mode to Dr. Memory. Use new options -fuzz_target and -fuzz_mutator to fuzz test any C or C++ target function having a buffer argument and a corresponding buffer size argument.
- Switched the Windows installer from an NSIS-based executable to a WIX-based .msi file. The new installer automatically adds Dr. Memory to the PATH for the current user.
- Added an unaddressable warning report on a write to read-only memory.
- Added support for the VALGRIND_DO_LEAK_CHECK annotation.
The changes between version 1.8.0 and version 1.7.0 include:
- Dropped official support for Windows 2000.
- Enabled -delay_frees_stack by default.
- The Windows installer now automatically adds Dr. Memory as an External Tool for every installation of Visual Studio on the machine.
- Changed details of the callstack maximum frame options. Added two new options: -malloc_max_frames, used for leak reports (and mismatch aux info), and -free_max_frames, used for -delay_frees_stack. Changed the default -callstack_max_frames to 20. -malloc_max_frames defaults to the original 12, and -free_max_frames is at 6.
- Added a new option -check_uninit_blacklist to reduce overhead on modules where no uninitialized read error reporting is desired. Dr. Memory automatically adds to this list when it sees a whole-module suppression.
- Disabled checks for mixing C library heap routines with Windows API heap routines (-check_heap_mismatch) for modules that contain static libc.
- Changed the drsymcache_lookup() interface.
- Added extra information to unaddressable error titles to make several common types of such errors easier to identify: "UNADDRESSABLE ACCESS of freed memory", "UNADDRESSABLE ACCESS beyond heap bounds", and "UNADDRESSABLE ACCESS beyond top of stack".
- Renamed option -stderr to -use_stderr.
The changes between version 1.7.0 and version 1.6.1 include:
- Added a Beta version of Dr. Memory for Mac. Be aware that this is a work in progress, but is far enough along to use on small applications.
- Changed Windows PDB symbols to collapse template parameters to "<>" to shrink and simplify callstacks and suppressions. The old behavior can be requested by passing 0x1000 to the -callstack_style option.
- Changed Linux and MinGW symbols to omit the "()" in C++ symbol names.
- Added a new Dr. Memory Framework Extension, Dr. SymCache or drsymcache, which provides persistent caching of symbol lookup data to reduce startup time overhead on large applications.
- Clarified the final parameter to umbra_get_shadow_memory_type() and umbra_shadow_memory_is_shared() to be umbra_shadow_memory_type_t rather than uint.
- Clarified the 2nd parameter to umbra_create_shadow_memory() to be umbra_shadow_memory_flags_t rather than uint.
- Clarified the umbra_shadow_memory_info_t shadow_type field to be umbra_shadow_memory_type_t rather than uint.
- Clarified the umbra_map_options_t flags field to be umbra_map_flags_t rather than uint.
- Added static library versions of drsyscall and umbra.
- On Windows, handle leak detection (-check_handle_leaks) is now on by default in full mode. However, it is still experimental and conservative, placing leaks it's not sure about in potential_errors.txt.
- Added preliminary shadowing of xmm registers for better Visual Studio 2013 support.
- Added preliminary shadowing of mmx registers.
- Augmented the Dr. Syscall API for MacOS support by adding drsys_cur_syscall_result(), drsys_arg_t.value64, and drsys_pre_syscall_arg64().
- The pause in cmd on a frontend error is now bypassable by -batch.
- Added an option -exit_code_if_errors to return a non-zero exit code when errors are detected.
- Replaced the perl front-end
drmemory.pl
on Linux with an executable. - Various bug fixes.
The changes between version 1.6.1 and version 1.6.0 include:
- Added Windows 8.1 support
- Added -handle_leaks_only feature for low-overhead handle leak checking
- Added the -crash_at_unaddressable and -crash_at_error options
- Added the -no_callstack_use_fp and -callstack_conservative options
- Various bug fixes
The changes between version 1.6.0 and version 1.5.1 include:
- Added a separate category of error report, "potential errors", stored in potential_errors.txt. These are errors that are suspected of being false positives or limited to system libraries. It is possible that they are in fact true application errors, however. This separation of errors can be disabled with the option -lib_blacklist_frames 0, and tuned via the options -lib_blacklist and -lib_blacklist_frames.
- Added options -lib_whitelist and -lib_whitelist_frames to support only displaying errors whose top N frames reference a whitelisted module. Any error that does not match the whitelist is separated as a "potential error" (see the prior blacklist entry).
- Removed -srcfilter option and replaced with new options -src_whitelist and -src_whitelist_frames to support only displaying errors whose top N frames reference a whitelisted source file. Any error that does not match the whitelist is separated as a "potential error" (see the earlier blacklist entry).
- Added full support for applications linked with the debug Visual Studio C library.
- Added checks for mixing of C library heap routines with Windows API heap routines on the same allocation, under a new on-by-default new option -check_heap_mismatch.
- Switched to replacing the malloc implementation with Dr. Memory's own allocator, rather than wrapping the system allocator, for improved performance and stability and improved interoperability with the Visual Studio debug C library.
- Improved still-reachable allocation reporting, adding suppression support, de-duplication, and hiding of system allocations.
- Added option -check_prefetch to make it easier to turn off the warnings on prefetching unaddressable memory.
- The GDI error on using a DC by more than one thread is now off by default. It can be enabled with the -check_gdi_multithread option.
- Added drsys_syscall_gateway() to the Dr. Syscall library.
- Extended -soft_kills to handle the most common job termination sequences.
- Further reduced false positives in Windows system libraries by continuing to make progress on Windows system call handling.
- Added a source file package to the release process.
- Improved Windows 8 support.
- Improved Visual Studio 2012 support.
- Added non-ASCII symbol support.
- Added the total execution time to the logfile.
- Added default suppressions for real bugs in the Visual Studio C and C++ libraries detected by Dr. Memory.
- Various bug fixes.
The changes between version 1.5.1 and version 1.5.0 include:
- Re-organized and improved the documentation.
- Added Visual Studio External Tool support (-visual_studio option)
- Added experimental support for Windows 8. However, false positives may occur on graphical or networked applications on Windows 8. Future releases will add full support.
- Added Dr. Syscall library and Dr. Memory Framework for building other tools from Dr. Memory's feature base. Future releases will add other libraries and add further tool-building support.
- Added
drstrace
, an initial version of a system call tracer. Future releases will expand its capabilities. - Added heuristics to distinguish substrings that look like pointers and thus reduce false negatives in leak reporting. See the -strings_vs_pointers option.
- Added a new, experimental lightweight mode:
-unaddr_only
. - Added non-ASCII application support.
- Added option
-show_duplicates
to list details on duplicate errors. - Eliminated reliance on the WDK/DDK when building from sources.
- Fixed PATH limit issue with the NSIS installer.
- Reduced false positives from bitfields.
- Fixed problems with debug C libraries on Windows.
- Fixed some problems in callstack walking.
- Various bug fixes.
The changes between version 1.5.0 and 1.4.6 include:
- Compatibility change: GDI usage errors are no longer of type WARNING but have their own type GDI USAGE ERROR. This affects any existing suppressions.
- Added online symbols for Linux, which enables -results_to_stderr. The -aggregate and -skip_results post-processing features are temporarily disabled until a future release adds them for both Windows and Linux.
- Added support for the Microsoft Visual Studio dynamic debug C library
- Added automated retrieval of certain system library debug files at the end of a run for better error reports in future runs.
- Added -log_suppressed_errors option
- Added -show_duplicates option
- Support wildcards in instruction lines in suppressions
- Improved callstack walking
- Added -callstack_exe_hide
- Eliminated most false positives with bitwise operations
- Various bug fixes
The changes between version 1.4.6 and 1.4.5 include:
- Added MinGW support
- Added GDI API usage error reports
- Added -light mode that does not detect uninitialized reads or leaks but is more performant
- Added -pause_at_error option
- Added mod!... suppression wildcard
- Reduced false positives from graphical operations
- Various performance improvements
- Various bug fixes
The changes between version 1.4.5 and 1.4.4 include:
- The tool now detects mismatched use of malloc/new/new[] vs free/delete/delete[]
- Added a script to convert Memcheck suppression files to the Dr. Memory format.
- Performance improvements on large applications via a symbol cache
- Renamed the -check_cmps option to -check_uninit_cmps
- Renamed the -check_non_moves option to -check_uninit_non_moves
- Various bug fixes
The changes between version 1.4.4 and 1.4.3 include:
- Fix for missing output in cmd on Win7 SP1
- Check for /MDd and abort
The changes between version 1.4.3 and 1.4.2 include:
- Changed the default callstack printing style and added control over the style via the -callstack_style runtime option
- On Windows, printing errors to stderr is now on by default
- On Windows, several options now control simplifying callstacks: -callstack_truncate_below, -callstack_srcfile_prefix, -callstack_modname_hide, -callstack_srcfile_hide. The -brief option sets these to common defaults and hides absolute addresses for simpler reports.
- Callstack walking in presence of FPO is now supported.
- Added a new mode that is several times faster but does not check for uninitialized reads: -no_check_uninitialized.
- Thread creation stack traces are now available in the global.pid.log file for identifying threads.
- Suppressions can take names, and the list of used suppressions is printed out in the results file.
- Increased default -redzone_size to 16
- Suppression can take instruction specifiers for narrower scope.
- The Valgrind Memcheck suppression format is supported for legacy suppression files: however, C++ symbols must be de-mangled at this time.
- Various bug fixes
The changes between version 1.4.2 and 1.4.1 include:
- Initial graphical system call support to reduce false positives
- Various bug fixes
The changes between version 1.4.1 and 1.4.0 include:
- Full Windows 7 support
- Various bug fixes
The changes between version 1.4.0 and 1.3.1 include:
- Initial Windows 7 support
- Performance improvements
- Various bug fixes
- Release build is used by default
The changes between version 1.3.1 and 1.3.0 include:
- Performance improvements
- Various bug fixes
The changes between version 1.3.0 and 1.2.1 include:
- Enabled -check_leaks and -possible_leaks by default (Issue 8)
- Improved suppression support: support for vertical wildcards "..."
- Locate target app on path on Linux (Issue 33)
- Various bug fixes
The changes between version 1.2.1 and 1.2.0 include:
- Fix bug in symbol lookup causing debug build asserts
The changes between version 1.2.0 and 1.1.0 include:
- Avoid false positives and false negatives when using debug versions of msvcrt, whose own debugging facilities conflicted with Dr. Memory's
- Added -perturb feature that uses random delays to attempt to trigger data races such as use-after-free accesses, which Dr. Memory will immediately detect
- Separate indirect leaks from direct leaks
- Added -version option to front end
- Generalized "invalid free" to "invalid heap argument"
- Improved performance of many instruction types including string instructions
- Improved performance of heap routines accessing heap headers
- Improved performance of stack adjustments
- Improved performance of shadow table accesses
- Improved performance of hashtables via dynamic resizing
- Display messages in cmd window, where DynamoRIO has trouble printing
- Switch to online symbol processing on Windows, greatly simplifying the Windows design and making it more robust by eliminating perl entirely
- Eliminate need for admin privileges to run on Windows
- Added wildcard support to suppressions
- Various bug fixes
The changes between version 1.1.0 and 1.0.1 include:
- Eliminated common C++ cases of false positives in possible leaks
- Report additional information on unaddressable errors: list the nearest malloc chunks above and below, and whether the access was to freed memory
- Report timestamp and thread id of reported errors
- Added -leaks_only feature for low-overhead leak checking
- Improved support for heaps used as stacks when such stacks are small and are adjacent to non-stack data
- Improved Linux system call parameter handling to reduce false positives
- Added default suppression file
- Added -aggregate option to combine error listings and eliminate duplicates among a set of processes
- Report the total number of leaked bytes
- Perform error duplication checks in the client
- Reduce memory usage of callstacks stored on each malloc
- Eliminate false positives from libc string routines by replacing them
- Switch to true reachability-based leak detection, trigger-able from a nudge at any time during a run
- Split possible leaks from certain leaks
- Improved documentation
- Improve performance of consecutive similar memory references
- Improve performance of 64K-boundary-crossing stack adjustments
- Many other performance improvements
- Many bug fixes
Limitations
Dr. Memory is still under development. It has some missing features and undoubtedly some bugs. The missing features include:
- Uninitialized read checking is not yet supported for ARM platforms.
- Windows system call parameters are not all known, which can result in false positives and false negatives, especially on graphical applications on more recent versions of Windows. Often, these false positives are solely in system library code. Dr. Memory attempts to separate such errors into potential_errors.txt.
- Definedness is tracked at the byte level, not at the bit level, which when bitfields are in use can lead to false positives.
- Mac OSX supports only 32-bit applications and does not yet have a full isolation barrier between Dr. Memory and the application, which can cause failures on large applications.
- Race corner cases:
- Pathological races between mallocs and frees can result in Dr. Memory's shadow memory structures becoming mis-aligned with subsequent false positives. However, such a scenario will always be preceded by an invalid free error.
- General races between memory accesses and Dr. Memory's shadow memory can occur but errors will only occur with the presence of erroneous race conditions in the application.
- This release can produce false positives if multiple threads write to adjacent bytes simultaneously. Future releases will provide options to trade off performance, memory usage, and accuracy.
Interoperability
Dr. Memory may have conflicts with security software or with other software that injects into the same process as Dr. Memory. There are known interoperability problems with some anti-virus applications, including Avast 8.0.