Memtools

The memtools module primarily contains tools for debugging memory usage. The tools can deal with many aspects of memory usage such as heap allocations, heap fragmentation, out of memory conditions and stack usage.

In addition there are API:s in this module to lookup stack traces and address information (the function, source file and line number an address belongs to). They are used by the tools, but are sometimes useful for debugging other parts of Opera as well.

Happy malloc (aka LTH malloc)

Happy malloc is a powerful tool for debugging memory usage. It tracks memory usage by replacing new, malloc etc. In addition to tracking memory usage it is also possible to limit the heap size to test code under low memory conditions.

A summary about the memory usage is displayed when the program exits. Call summarize_allocation_sites() to get a summary about the current memory usage.


The summary contains information about the allocation sites of the following form:

 A   298.9K   300.5K 0x08348500 GdkOpBitmap::InitPixeldata () at platforms/unix/base/gdk/GdkOpBitmap.cpp:90

The letter at the start of the row indicates the kind of allocation that was performed:

The first number is the number of bytes currently in use (either in K:s or plain bytes if less than one K).

The second number is the peak usage, the max number of bytes that has been allocated at the same time, at this allocation site.

The third number is the allocation site, the address of the code that caused the allocation.

The rest of the line is gdb style information about the function and source line of the allocation site. It is only available if ADDR2LINE_SUPPORT is enabled.

USAGE: happy malloc is enabled by setting MALLOC_VARIANT=LTH in the make files (you should do this in your platforms/unix/build/user_settings/*.mk or platforms/unix/share/build/user/*.mk)

ENVIRONMENT VARIABLES:

aoff-malloc

I haven't used this myself. This is from the .cpp file:

* Address-ordered-first-fit malloc implementation.
* Lars T Hansen
*
* A malloc engineered to mimic that of Symbian, to measure fragmentation
* without having to run on the emulator or target.  NOTE, however, that
* even the description we received was sketchy on how realloc behaves:
* realloc can choose to operate in several ways.  I have marked the choice
* points that I know about with POLICY in the code.
*
* Platforms currently supported:
*   QNX
*   Unix/Linux
*   WIn32

Stacktrace API

Stacktrace is an API for printing or saving the callstack.

Stacktrace is enabled by setting STACK_TRACE_SUPPORT=YES in the Makefiles.

Addr2Line API

Addr2Line is an API used to look up function names, source files and line numbers from a program address. Addr2Line is derived from the linux addr2line tool (part of binutils) and currently only works on Linux.

Addr2Line is enabled by setting ADDR2LINE_SUPPORT=YES in the Makefiles.

Stackometer

Very simple tool used to measure stack usage. It measures the number of bytes between the first frame and the current frame. It relies on the gcc feature '-finstrument-functions' to keep track of the current stack frame.

Constrained Valgrind

Constrained Valgrind is a version of Valgrind that can be used for OOM-testing.

Valgrind is a very powerful tool for memory testing since it is very detecting when uninitialized or deleted memory is being accessed. It makes it ideal for OOM-testing as the two most common reasons for OOM-crashes seems to be that either object initialization is not completed, so an object is left half-way initialized, or objects are deleted prematurely as part of the OOM-cleanup process, so they end up being used after they have been deleted. Both of these things have a tendency to cause random, unpredictable crashes. Without a tool like Valgrind it can be very hard to find out what causes the crashes. Unfortunately standard Valgrind doesn't let you do OOM-testing - Constrained Valgrind will!

Constrained Valgrind is used the same way as the standard Valgrind. The difference is that Constrained Valgrind supports the additional allocation functions used by constrained malloc (in the lea_malloc module).

How to use Constrained Valgrind:

  1. Build constrained Valgrind:
    cd constrained-valgrind; ./configure; make 
    Optionally run make install, but beware, it might overwrite previously installed Valgrinds.
  2. Build Opera with CONSTRAIN_MEMORY=YES and VALGRIND defined.
  3. Limit how much memory Opera is allowed to use:
    export OPERA_ALLOC_LIMIT=5000000
  4. Run Constrained Valgrind:
    ./valgrind --log-file=opera-valgrind.log --error-limit=no --tool=memcheck opera

Constrained Valgrind understands OPERA_ALLOC_LIMIT. Any Opera allocation that would cause the total amount of Opera allocated memory to go beyond the limit is OOM:ed. Every time an OOM occurs a stack trace of the allocation site is printed to the log file (or stdout) along with the other errors detected by Valgrind. When (if) Opera crashes because of an OOM-error, both the allocation site that OOM:ed and the memory access that caused the crash is in the log, so it is usually quite easy find the faulting code and fix the problem.

Addrtool

Addrtool looks up information about addresses in Linux executables and the shared libraries they are linked against. It can get information from many different sources:
  1. link-maps to map addresses in stripped binaries to functions.
  2. obscurifier-maps to translate obscurified symbols to clear text.
  3. memory-maps to find locations of shared libraries and lookup addresses in shared libraries.
  4. running processes.
  5. debug information in executables (source-file/line-number information).

Depending on which information is available to Addrtool it can lookup different things about an address including the function the address is in, the source file and line number corresponding to the address or the object file the address is in.

One of the main motivations behind this tool is to make it easy to translate stack traces from stripped and/or obscurified binaries into a readable form. For instance, if a customer sends you a core file from a stripped binary you can run gdb on it and get a stack trace. Unfortunately that stack trace is usually not very helpful. Because the binary does not contain any symbol information gdb will often give you incorrect information about the addresses in the stack trace, and even if symbol information is available the symbols might be obscurified.

Addrtool addresses these problems. If you run addrtool with the -g flag you can cut'n'paste "bad" gdb stack traces into it. Addrtool will recognize gdb stack traces and rewrite the address information of known addresses to reflect what Addrtool knows about them. As long as you provide Addrtool with enough information (link-maps and the like) you usually end up with a more correct and readable stack trace...

Another scenario is if you use the stack trace API for debugging purposes. It can give you stack traces with symbol information, but only when the Addr2Line API is enabled. Unfortunately, there are many reasons why you might want to keep the Addr2Line API disabled. It has a high impact on memory consumption, it allocates memory when used, it makes printing the stack traces slower and the libraries needed by it might not be available on devices. Basically it might not be feasible to enable Addr2Line.

Without Addr2Line, you will end up with stack traces containing only addresses. Addrtool helps you decipher them. It is quite easy too. Just take your log file and pipe it through Addrtool. It will interpret any line starting with an hexadecimal number as an address and rewrite it with symbol information. All other lines will just be echoed back. The end result is the same as the original log file except that all the stack traces now contain symbol information.

Note that you can run Addrtool on your desktop machine and look up information from binaries for other architectures, so you don't have to worry about getting Addrtool to run on devices.

How to use Addrtool:

  1. Build Addrtool
    cd addrtool; make
  2. Run addrtool
    ./addrtool -h

Improvements

Module Based Happy Malloc

By running 'nm' on the compiled code figure out what spaces of memory the code of that module will reside in. Then activating Happy Malloc for that module only, making it easy for module owners to oom test only their code. Heap restrictions etc. only apply to the chosen module.


Analyzing Memory Fragmentation

  1. Run a set of sites in opera (url player)
  2. Close the window (manually)
  3. Empty caches
  4. Report currently allocated memory
  5. Report memory fragmentation
  6. Repeat above a couple of times

Happy Malloc is to be used to report allocated memory Lea Malloc is to be used to report memory fragmentation (since it is a good representation of ordinary allocators)

Feature: Use two heaps at the same time, happy malloc allocates as ordinary but also saves a pointer for an identical allocation with lea malloc in heap two. This would enable single run for full test and identical memory allocation.

Realloc: make sure the pointer to lea-memory in happy malloc is updated

OOM: Make the happy heap twice as big as the lea heap and make happy malloc report oom iff lea malloc reports oom, bail out if happy malloc gets oom and not lea. Might be possible to have unlimited happy heap.

Requirement: Must be able to close window (and document manager etc) without closing opera in GogiOpera.

Improved UI

Add specific UI components for running memtools

Hacks

Hacks exist to persuade happy malloc to do all of the following:

Hacks live on side-branches !

Note that making any 'lib*.a' into a 'lib*.so' (by using 'ar x lib*.a' to extract its object files, then 'g++ -shared -o libthing.so *.o') makes it possible to discover (with a little help from 'nm' and the '/proc/$pid/maps' file) more about allocation sites than 'gdb' will report, if the original code was compiled without debug symbols.