The minpng module

API Documentation

For details about this module see the Doxygen generated API documentation

Functionality

The minpng module decodes and encodes PNG-images. The decoder supports all image formats (color-formats/bpp) specified in the PNG specification as input and outputs image data in RGBA or indexed-color format. It also supports animated pngs (APNG). The encoder, on the other hand, is sort of minimalistic and does only support input in the form of RGBA data and outputs PNG data in color formats 2 (truecolor) and 6 (truecolor with alpha).

minpng depends on the zlib module.

Where is minpng used in Opera?

The minpng decoder is used by the img module to decode animations and images on web-pages or in the UI.

The minpng encoder is used by the canvas module to implement the toDataUrl method for HTMLCanvasElement according to the specification of the canvas element.

The encoder is also used by CoreGogi to take screenshots in the form of PNG images for the purpose of regression testing.

Design goals and implementation strategies

The design of the external api for minpng as well as the internals has been focused on providing a simple api based around one function (minpng_decode for the decoder and minpng_encode for the encoder) that supports partitial encoding/decoding, that is - the user don't need to supply all input data at one, but can instead "chunk it up" into smaller portions for feeding the decoder/encoder chunk by chunk. The purpose of this is to permit loading of images to begin before the full image has been (down)loaded, and also for not stalling Opera while e.g. decoding large images (or animations).

The strategy for implementing this goal has been to implement the decoder and encoders as "state machines", with an internal state that keeps track of the decoding/encoding process so that minpng knows where to continue when being called multiple times for decoding/encoding of the same image.

The state is recorded internally in minpng as the structs minpng_state (for the decoder) and minpng_encoder_state (for the encoder). These struct keep track of the current decoding/encoding progress, e.g. "has the png header been processed yet" or "how many scanlines have been decoded so far", etc.

The other important structs used are the PngFeeder and PngRes structs (for the decoder), and the PngEncFeeder and PngEncRes structs (for the encoder). These are used to "feed" the decoder/encoder with png data/uncompressed scanlines which will generate the result in the "result" structs as uncompressed scanlines/png data

The decoder are in general only keeping the part of the image which should be returned to the caller for the current call to minpng_decode in memory as decoded image data. The exception is while decoding interlaced PNG images, then does minpng need to keep data from previous interlace-passes in memory in order to compose the final deinterlaced output image data.

Memory and testing

Used OOM policies

minpng is organized as a library, and as such returns a special status code from minpng_decode (PngRes::OOM_ERROR) or minpng_encode (PngEncRes::OOM_ERROR) when an OOM error occurs.

Who is handling OOM?

When minpng_decode or minpng_encode returns OOM_ERROR, then is the caller responsible of freeing up memory by calling minpng_clear_result and minpng_clear_feeder, or in the encoding case minpng_clear_encoder_result and minpng_clear_encoder_feeder (as well as the 'result' and 'feeder' structs themselves of course...).

Description of flow

When an OOM error occurs in the module, the error will be immediately propagated up to the caller of minpng_decode or minpng_encode.

Heap memory usage

When decoding interlaced images, a temporary buffer sized width * height * 4 bytes is used to store the deinterlace progress.

Normally the memory usage depends on the amount of data that is fed to the module at a time, it allocates aproximately (width * number_of_lines_in_current_data * 4) bytes of memory for the decoded image, and on average (uncompressed_size * 1.2) bytes of memory for the decoding buffer.

The interlace image buffer is still there, though. It can be removed by adding a box-drawing interface to the img objects. This would probably only be useful when the footprint requirements are very strict.

The decoding state is kept in heap memory, and occupies around 32 bytes of memory.

Stack memory usage

There are no deep callstacks or large stack buffers in the minpng module. The worst case is around 1Kb used stack.

Static memory usage

There is no static memory usage at all.

Caching and freeing memory

All cached data will be freed by minpng_clear_result and minpng_clear_feeder, or in the encoding case minpng_clear_encoder_result and minpng_clear_encoder_feeder. These functions can be called at any time between calls to minpng_decode and minpng_encode.

Freeing memory on exit

Because the encoder and decoder are not implemented as C++ classes, it's the user of minpng who is responsible of clearing the structs as mentioned above.

Temp buffers

There are no temp buffers.

Memory tuning

No tuning possible, sorry...

Tests

The testsuite includes the complete png-testsuite from the official png site as well as a couple of smoketests. PNG images are decoded and compared to identical raw-imagedata reference images during selftest.

Coverage

The testsuite covers everything about decoding except the OOM handling.

Design choices

The design choice which matters the most for memory usage is the kind of image data to output from the decoder, here we stick to the ones used by OpBitmap (32-bit RGBA and 8-bit indexed).

Suggestions of improvements

See this!

Using minpng

Sample minpng decoding usage code (including the 'AGAIN' handling, and a hypotetical 'Image' class):
// The Image class is something that only exists in this example (not the Image class in the img module... )
bool decode_image( Image img, void *png_data, int png_data_len )
{
    bool finished=false, inited=false;
    Image img;
    PngFeeder feeder;
    PngRes result;

    feeder.data = png_data;
    feeder.len = png_data_len;

    // Clear the result struct
    minpng_clear_result(&result);
    while( !finished && feeder.len > 0 )
    { 
        bool again = true;
        while( again )
        {
            switch( minpng_decode( &feeder, &result ) )
            {
               case PngRes::OK:
                 finished=true;
               case PngRes::NEED_MORE: // Assumes all data we have were present in png_data
                 again=false; 
               case PngRes::AGAIN: // Note all the intentional fallthroughs
                for( int i = 0; i<(int)result.lines; i++ )
                {
                   // Intentionally here. Avoid calling init if we do not have at least one line
                    if( !inited ) {
                         inited = true;
                         // If minpng has outputted indexed image data will result.image_frame_data_pal
                         // specify a palette as a byte array in the format r0,g0,b0,r1,g1,b,1...etc.
                         // result.has_transparent_col and result.transparent_index will give information
                         // about a possibly transparent color (index).
                         img.Init( result.mainframe_x, result.mainframe_y, result);
                    }
// Indexed output is only sent if this macro is defined
#if defined(SUPPORT_INDEXED_OPBITMAP)
                    // indexed image data outputted from minpng
                    if(result.image_frame_data_pal != NULL) 
                        img.AddLine( i+result.first_line, result.image_data.indexed+i*result.mainframe_x );
                    // RGBA or BGRA image data
                    else 
#endif // SUPPORT_INDEXED_OPBITMAP
                        img.AddLine( i+result.first_line, result.image_data.rgba+i*result.mainframe_x );
                }
                break;
              case PngRes::OOM_ERROR:
              case PngRes::ILLEGAL_DATA:
                return false;
            }
            minpng_clear_result(&result);
        }
    }
    return true;
}

Future plans