Locale module

Copyright 1995-2012 Opera Software ASA. All rights reserved. This file is part of the Opera web browser. It may not be distributed under any circumstances.

Introduction

The locale module

The Locale module provides the OpLanguageManager interface used to retrieve text strings used for display in the user interface in Opera. Nowhere in Opera may hard-coded strings be displayed in the user interface, except for debugging information and similar.

The OpLanguageManager defines the interface available to the Opera core. No platform-independent code must ever use any other methods than the ones defined in the OpLanguageManager interface.

The build system

The code in the locale module requires that certain scripts are set up to generate the relevant locale include files. These include files are generated by the operasetup script in hardcore. You will also need to generate run-time or compile-time language files (depending on your platform), so that your program has access to the language strings at run time, please refer to the makelang documentation in the translations module for information on how to do that.

If you are interested in the internals of the build system for locale data, please refer to the strings module documentation on the build scripts. The rest of the documentation here is for the language string access code.

Current information about the Locale module.

Interface overview and API documentation

The API documentation is extracted automatically by Doxygen.

Use-cases

Retrieving a string for use inside the core or in the UI

The normal use-case of this module is that the core or UI code requests a string, using the enumeration value defined in the Str class. This is performed by calling the GetStringL() or GetString() methods on the g_languageManager global object.

Adding new language strings

If you need to add new language strings to the string database, please refer to the documentation in the strings module for details.

Supporting multiple languages at once

In some cases, the Opera engine needs to be able to have simultaneous support for several languages.

When having to support different localizations in different concurrent windows, for example in Opera Mini, the locale module exports the API API_LOC_CONTEXTS which enables the OpLanguageManager::SetContext() method. You will need to implement the context switching code in your platform implementation of the OpLanguageManager interface, however.

If all you need is to be able to get strings from a “fall-back language” for strings that are missing in your translation, the standard APIs in the OpPrefsFileLanguageManager implementation should be enough, it will load both the currently configured user language and what is designated the default in preferences (English).

When you need to also have the strings from this fall-back available alongside the translated strings, for instance to generate menu shortcut letters for East Asian localizations, you should probably just instantiate your own copy of the OpPrefsFileLanguageManager object and load the default translation into it. This way you will have fast access to the default strings, while the core code will use the selected localization for all the strings it needs.

Supported standards

None. The LNG file format is however based on the standard Windows INI file format, with some extensions.

Implementation and design

The Str::LocaleString class

The Str::LocaleString class encapsulates a platform-independent way of identifying language strings, allowing the build system to define the necessary strings. Due to a limitation in Visual C++ where the sum of the debug information for enumerations is limited to 64kb, Str::LocaleString is defined as a class, and not an enumeration. The class wraps several actual enumerations.

Generalisation and re-implementation

To avoid limiting all Opera platforms to using the same storage model for language strings, the abstract interface OpLanguageManager is all the core code in Opera sees of the string handling. This interface can easily be re-implemented on different platforms, for example providing a version that read language strings from static resources instead of the normal, file-based, lookup.

The OpLanguageManager uses the enumeration of available strings provided by the english.db file as described above. The numerical equivalents of the enumeration values are not used by the Opera core, and can easily be replaced for platforms requiring so, for example where the resource compiler itself maps numbers to resources.

While the core code only is allowed to use the interface defined by OpLanguageManager, platform-specific code may of course use any special features available in their platform implementation.

When re-implementing the interface, take care to make sure that your implementation passes all the selftests for this module. Core may behave in unexpected ways if it does not. Especially take care to ensure that the UniString object returns are properly reference-counted and do not go out of scope. This can be done by either using UniString as an internal storage model, or by always having the GetString() interface copy the string into the passed UniString object.

Text file based language manager

The OpPrefsFileLanguageManager class defined in modules/locale/src/opprefsfilelanguagemanager.h is intended as a cross-platform implementation to be used by platforms. It implements the language support by loading a LNG file from disk. It is enabled by FEATURE_LANGUAGE_FILE.

Binary file based language manager

The OpBinaryFileLanguageManager class defined in modules/locale/src/opbinaryfilelanguagemanager.h is intended for small systems where there is no requirement that the language files are human-readable. The files are stored in a pre-sorted binary format that can be read directly into memory without the need for costly postprocessing/conversion. It is enabled by FEATURE_BINARY_LANGUAGE_FILE.

Dummy language manager

For builds not requiring language files, or for use while building prototypes, there is a OpDummyLanguageManager class that will return empty strings for all requests. This version is enabled by defining USE_DUMMY_LANGUAGEMANAGER.

Re-implementing OpLanguageManager

Platforms that require different string handling can re-implement OpLanguageManager Since the core code will never use the numerical ids, but only the LocaleString enumeration, platforms that need to have the enumeration generated elsewhere should be easy to accommodate.

If platforms require the language strings available in a special format, these can easily be generated from the language database file, since it has a well-defined format.

Initialisation

The locale module knows how to bootstrap itself for the provided implementations of the OpLanguageManager interface. If your platform is using its own implementation, you will need to initialise it manually before calling Opera::InitL() by calling LocaleModule::SetLanguageManager().

Memory management

This discussion concerns the OpPrefsFileLanguageManager and OpBinaryLanguageManager implementations. The OpDummyLanguageManager implementation does not allocate any strings or use any memory beyond the global pointer.

Heap usage

The OpPrefsFileLanguageManager and OpBinaryLanguageManager implementations will read translation files from disk and store the strings in memory. By its very nature it needs to retain these allocated strings during the entire life-time of Opera, something which requires a careful design, so that the memory overhead is not too great, especially when considering smaller devices. The memory overhead must, however, be offset against the time consumed for string lookups, which are quite frequent.

During the initial loading operation, the OpPrefsFileLanguageManager will use more than double the amount of memory it retains during normal execution. Here the language file is parsed and loaded into memory using the PrefsFile interface provided by the prefsfile module. After the file has been loaded, OpPrefsFileLanguageManager will transfer it into its internal storage, which consists of a UniString object large enough to carry all the strings in the file, plus an array of indices with pointers to the strings inside the first array. When the loading is finished, OpPrefsFileLanguageManager will release the PrefsFile object and just keep its optimised storage area.

OpBinaryLanguageManager does not require the extra overhead of loading, converting and copying the language strings, it will instead allocate an array of strings into which it will read the language strings directly from the language file, without using any intermediate storage. This means that the initialisation will have the overhead of the OpFile object needed for accessing the language file.

The storage, which is proportional to the number and lengths of the strings, is kept in memory until the OpPrefsFileLanguageManager or OpBinaryLanguageManager objects are destroyed, which happens in Opera::Destroy().

Stack usage

There are no recursive calls, and no arrays or large objects are allocated on the stack.

Static memory usage

There is one global pointer, which is handled by the LocaleModule object.

OOM policies

All of the interfaces to functionality that can require memory to be allocated is using the TRAP/LEAVE convention. There is, however, a convenience wrapper function for retrieving strings which instead returns a status code; it is implemented as a wrapper around the leaving interface. Inherited interfaces only implement the version using leave.

Performance

The string lookup is realised by a binary search using an integer lookup key in the file-based language manager implementations, and should be quite fast. Due to the many possible implementations of the OpLanguageManager interface, all string lookups require data copying, and possible allocation (embedded in the use of the OpString class in the interface).

References