This is the heap profiler we use at Google, to explore how C++ programs manage memory. This facility can be useful for
The profiling system instruments all allocations and frees. It
keeps track of various pieces of information per allocation site. An
allocation site is defined as the active stack trace at the call to
There are three parts to using it: linking the library into an application, running the code, and analyzing the output.
To install the heap profiler into your executable, add
-ltcmalloc to the link-time step for your executable.
Also, while we don't necessarily recommend this form of usage, it's
possible to add in the profiler at run-time using
% env LD_PRELOAD="/usr/lib/libtcmalloc.so" <binary>
This does not turn on heap profiling; it just inserts the
code. For that reason, it's practical to just always link
-ltcmalloc into a binary while developing; that's what we
do at Google. (However, since any user can turn on the profiler by
setting an environment variable, it's not necessarily recommended to
install profiler-linked binaries into a production, running
system.) Note that if you wish to use the heap profiler, you must
also use the tcmalloc memory-allocation library. There is no way
currently to use the heap profiler separate from tcmalloc.
There are several alternatives to actually turn on heap profiling for a given run of an executable:
Define the environment variable HEAPPROFILE to the filename
to dump the profile to. For instance, to profile
% env HEAPPROFILE=/tmp/mybin.hprof /usr/local/bin/my_binary_compiled_with_tcmalloc
In your code, bracket the code you want profiled in calls to
(These functions are declared in
HeapProfilerStart() will take the
profile-filename-prefix as an argument. Then, as often as
you'd like before calling
GetHeapProfile() to examine the profile. In case
IsHeapProfilerRunning() will tell you
whether you've already called HeapProfilerStart() or not.
For security reasons, heap profiling will not write to a file -- and is thus not usable -- for setuid programs.
You can more finely control the behavior of the heap profiler via environment variables.
||default: 1073741824 (1 Gb)||Dump heap profiling information once every specified number of bytes has been allocated by the program.|
||default: 104857600 (100 Mb)||Dump heap profiling information whenever the high-water memory usage mark increases by the specified number of bytes.|
You can use the heap profiler to manually check for leaks, for instance by reading the profiler output and looking for large allocations. However, for that task, it's easier to use the automatic heap-checking facility built into tcmalloc.
If heap-profiling is turned on in a program, the program will periodically write profiles to the filesystem. The sequence of profiles will be named:
<prefix>.0000.heap <prefix>.0001.heap <prefix>.0002.heap ...
<prefix> is the filename-prefix supplied
when running the code (e.g. via the
environment variable). Note that if the supplied prefix
does not start with a
/, the profile files will be
written to the program's working directory.
The profile output can be viewed by passing it to the
pprof tool -- the same tool that's used to analyze CPU profiles.
Here are some examples. These examples assume the binary is named
gfs_master, and a sequence of heap profile files can be
found in files named:
/tmp/profile.0001.heap /tmp/profile.0002.heap ... /tmp/profile.0100.heap
% pprof --gv gfs_master /tmp/profile.0100.heap
This command will pop-up a
gv window that displays
the profile information as a directed graph. Here is a portion
of the resulting output:
GFS_MasterChunk::AddServeraccounts for 255.6 MB of the live memory, which is 25% of the total live memory.
GFS_MasterChunkTable::UpdateStateis directly accountable for 176.2 MB of the live memory (i.e., it directly allocated 176.2 MB that has not been freed yet). Furthermore, it and its callees are responsible for 729.9 MB. The labels on the outgoing edges give a good indication of the amount allocated by each callee.
You often want to skip allocations during the initialization phase
of a program so you can find gradual memory leaks. One simple way to
do this is to compare two profiles -- both collected after the program
has been running for a while. Specify the name of the first profile
--base option. For example:
% pprof --base=/tmp/profile.0004.heap gfs_master /tmp/profile.0100.heap
The memory-usage in
/tmp/profile.0004.heap will be
subtracted from the memory-usage in
/tmp/profile.0100.heap and the result will be
% pprof --text gfs_master /tmp/profile.0100.heap 255.6 24.7% 24.7% 255.6 24.7% GFS_MasterChunk::AddServer 184.6 17.8% 42.5% 298.8 28.8% GFS_MasterChunkTable::Create 176.2 17.0% 59.5% 729.9 70.5% GFS_MasterChunkTable::UpdateState 169.8 16.4% 75.9% 169.8 16.4% PendingClone::PendingClone 76.3 7.4% 83.3% 76.3 7.4% __default_alloc_template::_S_chunk_alloc 49.5 4.8% 88.0% 49.5 4.8% hashtable::resize ...
kth entry in the third column is the sum of the first
kentries in the second column.)
The following command will give a graphical display of a subset of
the call-graph. Only paths in the call-graph that match the regular
DataBuffer are included:
% pprof --gv --focus=DataBuffer gfs_master /tmp/profile.0100.heap
Similarly, the following command will omit all paths subset of the
call-graph. All paths in the call-graph that match the regular
DataBuffer are discarded:
% pprof --gv --ignore=DataBuffer gfs_master /tmp/profile.0100.heap
All of the previous examples have displayed the amount of in-use
space. I.e., the number of bytes that have been allocated but not
freed. You can also get other types of information by supplying a
||Display the number of in-use megabytes (i.e. space that has been allocated but not freed). This is the default.|
||Display the number of in-use objects (i.e. number of objects that have been allocated but not freed).|
||Display the number of allocated megabytes. This includes the space that has since been de-allocated. Use this if you want to find the main allocation sites in the program.|
||Display the number of allocated objects. This includes the objects that have since been de-allocated. Use this if you want to find the main allocation sites in the program.|
By default -- if you don't specify any flags to the contrary --
pprof runs in interactive mode. At the
you can run many of the commands described above. You can type
help for a list of what commands are available in
./to your filename: