Module: Updaters

About this module

The updaters module will retrieve documents from a location specified by a URL provided by the application. After download, the file can then be processed and actions taken based on the data. Multiple downloads can be organized using AutoFetch_Manager, which can also be customized.

Currently the only format directly supported is on based on XML. When received, the document's digital signature will be (optionally) verified, and the file parsed as an XMLFragement. The parsing is then handed over to the implementation.

Interface overview and API documentation

API documentation generated by Doxygen contains all necessary information for the external APIs.

The module currently have 3 classes:

The basic APIs

AutoFetch_Element

Organizes the fundamental API and the application specific finished message posting. It does not handle any URLs directly

Usage:

Construction (Construct())

Specify the message to be posted (Will be posted with Id() of the element as par1 when PostFinished is called, par 2 is a application specific result code).

StartLoading()

Application specific function to start the operation. Return OpStatus::OK if operations has started, but not finished; OpRecStatus::FINISHED if the operation completely finished, and an error status if there was an error.

PostFinished()

Called by implementation when finished with the result code to be posted back to the caller. After this call IsFinished() will return TRUE

IsFinished()

Return TRUE if the operation is finished. Can be used to poll elements

AutoFetch_Manager

Can organize multiple fetch elements and send an application specific message when all are finished. Which message(s) to listen for from the elements have to be configured by the application; the class assumes any message with an ID in the list is a finished message

Usage:

Construction/InitL

The constructor is initialized with the Message code posted by the manager when all currently loading updaters are finished (the message have no parameters).The owner/application MUST register the callbacks for the messages to listen for.

AddUpdater
Adds a new updater to the list. Takes ownership and calls StartLoading(). When finished, the object will be automatically deleted.

The midlevel API

URL_Updater

Downloads a URL and then passes it on to the application specific processing.

Usage:

Construct()

The object is initialized with the URL of the resource to be loaded and the message to post when finished

StartLoading()
Starts loading the resource. Must not be called if added to a manager.
SetLoadPolicy()
This changes the load caching policy from always do a full load (the default) to the specified policy.
ResourceLoaded()

Application specific function that processes the file called if the loading is successful. The loaded url is passed as an argument

XML_Updater

Downloads a URL and processes the file as an XML document, and can verify digital signatures that follows a specific prefix. Once the XML is parsed the processing is handed over to the implementations.

Usage:

Construct()

The object is initialized with the URL of the XML document to be loaded and the message to post when finished

StartLoading()
Starts loading the document. Must not be called if added to a manager.
SetLoadPolicy()
This changes the load caching policy from always do a full load (the default) to the specified policy.
ProcessFile()

Application specific function that processes the file. The "parser" member is initialized when this functionis called and can be used to process the XML document.

Helper functions

GetFlag
Return TRUE if the specified element is in the current parser level.
GetTextData
Gets the text content of the current element
GetBase64Data
Processes the text content in the current element as Base64 encoded content and returns the decoded binary data. Have both DataStream and SSL_varvector variants.
GetHexData
Processes the text content in the current element as hexadecimal encoded content and returns the decoded binary data. Have both DataStream and SSL_varvector variants.
VerifyFile()

Application specific function that verifies the signature on the file.

The default implementation is configured using this API (applications overrding the default should not use this):

SetVerifyFile()

This function specifies whether or not the loading document should be verifies, and if so, which public key and algorithm to use. The default is to use the rootstore module's public key and SHA-256.

By default the verification function uses (cstring-style)"<?xml version=\"1.0\" encoding=\"utf-8\" ?>\n<!--" as a prefix before the signature followed by a CRLF and the signed content (which must include the comment end tag).

Implementation description

API documentation generated by Doxygen contains information about the internal organization of the module.

Footprint

The module is fairly small, about 5KB. It implements basic functionality imported by other modules as a foundation for their functionality, while most of the actual functionality is implemented in, and imported, from other modules, primarily url, xml, and libcrypto.

Dynamic memory use and OOM handling

OOM policies

In OOM situations the current operation is aborted and a failure notice is given to the action's owner, either by message or OpStatus return value. Very few functions LEAVE.

An OOM condition is raised when detected in the module itself

Who handles OOM?

Currently OOM is handled locally by aborting the operation. In some operations, when it is possible, the caller is informed of the status

Flow

Much of the module is message callback based, and these functions are not able to report OOM situations directly to the documents or UI. In these cases the current operation will be terminated, and errormessages sent.

Heap memory usage

There is no static heap memory use. A number of operations, like processing the XML file (read and parse) will keep the entire document in memory for the duration of the operation. For large XML documents that MAY lead to OOM situation, and applications that may encounter such large files should use handlers that are designed for such sizes.

Stack memory usage

Minimal. Most operations are done in heap allocated memory.

Static memory usage

None

Caching and freeing memory

Caching is managed by URLs. Memory is only used during actual operations and released immediately after use.

Freeing memory on exit

Controlled by applications. Classes clean up their own memory use.

Temp buffers

None.

Memory tuning

Not possible, except as already available thorugh URL and possibly XML

Tests

Selftests using libssl's automatic rootstore update and EV ability, and the pubsuffix module's tests.

Coverage

Selftests

Design choices

Classes are tailored for specific operations, AutoFetch_Element to provide the basic API, AutoFetch_Manager to manage multiple loads, XML_Updater to handle digitally signed XML documents. Specific implementations in subclasses are needed to actually do something.

Improvements

Should the URL handling in XML Updater be moved down into the FetchElement? Or does the current arrangement make better sense? Alternatively, should XML updater be broken into two separate classes