The DataStream classes are used to define an abstract interface for writing, reading and processing binary data (in this context textbased representations can also be considered binary). An earlier version of these classes exists in the util module.
Subclasses of DataStream includes:
Presently this module is used to implement the DataFile and DataFile_Record classes in the url_4 module. This module is also used in unfinished code that implements both the PGP and SSL/TLS protocols.
API documentation generated by Doxygen contains all necessary information.
NOTE: The API functions may LEAVE due to normal errors that does not permit further processing, in addition to OOM
Additionally, classes for variable length arrays, integers, referencing another DataStream object,as well as other special tasks exists.
// Allocate and open file for reading
OpFile *myfile = new OpFile;
myfile->Construct(UNI_L("myfile.dat");
myfile->Open(OPFILE_READ)
// Init filereader
DataStrem_GenericFile data_file(myfile);
data_file.InitL();
// Sample 5 bytes
byte buffer[5];
unsigned long len;
len = data_file.SampleDataL(buffer, 5);
if(len == 5)
{
// Do something, then commit the read bytes
data_file.CommitSampledData(5);
}
else
{
// do something else
}
// Read 5 bytes
len = data_file.ReadDataL(buffer, 5);
// Read a 32 bit unsigned integer (Bigendian, no MSB detection,
unsigned int read_value;
if(data_file.ReadIntegerL(read_value, 4, TRUE, FALSE) == OpRecStatus::FINISHED)
{
// Use Integer
}
// Write a record to the file
DataStream_ByteArray output_record;
// Specify that the record is 32 bytes long (but don't allocate)
output_record.ResizeL(32);
// Then read it from the file
if(output_record.ReadRecordFromStreamL(&data_file)== OpRecStatus::FINISHED)
{
// use record
}
// Allocate and open file for reading
OpFile *myfile = new OpFile;
myfile->Construct(UNI_L("myfile.dat");
myfile->Open(OPFILE_WRITE)
// Init filereader
DataStrem_GenericFile data_file(myfile, TRUE);
data_file.InitL();
// Sample 5 bytes
byte buffer[5];
unsigned long len;
data_file.WriteDataL(buffer, 5);
// Write a 32 bit unsigned integer (Bigendian, no MSB detection,
unsigned int write_value=2046;
data_file.WriteIntegerL(write_value, 4, TRUE, FALSE;
// Write a record to the file
DataStream_ByteArray output_record;
// Fill outout record
// Then write it to the file
output_record.WriteRecordL(&data_file);
class Record : public DataStream_SequenceBase
{
public:
DataStream_Octet first;
DataStream_UInt16 second;
DataStream_Octet third;
DataStream_UIntVarLength fourth;
DataStream_ByteArray fifth;
DataStream_UInt32 sixth;
public:
Record(): fourth(3) /* 3 byte length */{
// Build record structure;
first.Into(this);
second.Into(this);
third.Into(this);
fourth.Into(this);
fifth.Into(this);
sixth.Into(this);
// Set field IDs
first.SetItemID(1);
second.SetItemID(2);
third.SetItemID(3);
fourth.SetItemID(4);
fifth.SetItemID(5);
sixth.SetItemID(6);
}
protected:
// Special actions while reading a record
virtual void ReadActionL(uint32 step, int record_item_id)
{
// When the fourth integer is read use it to resize the fifth element
if(record_item_id == 4)
fifth.ResizeL(fourth);
}
}
Record write_record;
// Fill record, then write it to a file
DataStrem_GenericFile data_file_out(myfile_out, TRUE);
write_record.WriteRecordL(&datafile_out);
Record read_record;
DataStrem_GenericFile data_file_in(myfile_in, TRUE);
// Read record from
if(read_record.ReadRecordFromStreamL(&data_file_in) == == OpRecStatus::FINISHED)
{
// Use record
}
// Class that reads a string of Records
class File_Of_Records : public DataStream_GenericFile
{
public:
File_Of_Records(OpFile *file_p, BOOL _write = FALSE) : DataStrem_GenericFile(file_p, _write){};
Record *GetNextRecordL(){return (Record *) DataStream_GenericFile::GetNextRecordL();}
protected:
virtual DataStream *CreateRecordL(){return new(ELeave) Record;}
};
File_Of_Records record_file(myrecord_file);
record_file.InitL();
// Read records
Record *rec;
while((rec = record_file.GetNextRecordL()) != NULL)
{
if(rec->Finished())
{
// Use record
delete rec;
}
// read next record (or get the finsihed record (not really a problem with a files)
}
record_file.Close();
The module is fairly small, and functionality only used by PGP and/or the new SSL/TLS code (diffie_3) is automatically disabled for builds that does not activate those modules
All functions that allocates or may allocate memory in the module will LEAVE in case of an OOM situation.
The calling functions must either TRAP the LEAVE errors, and either handle them, or pass it on to its own caller for handling by returning a status or by performing a LEAVE.
All dataretrieval API functions works through virtual functions, implemented by the subclasses. The actual control flow will depend on the implementation of the classes.
Some of the classes access internal databuffers, other will access files or other DataStream objects
Actual allocation depends on the subclass implementation:
Memory is freed (and nulled) upon destruction of the object, or Resize.
Actual size of allocated memory depend on the implementation
For the most part the DataStream functions use little stackbased, usually less than 10 32-bit words. In some cases, like the DataStream_FlexibleSequence class, there are functions that uses at most two stack allocated DataStream_ByteArray as temporary values, each in the range of 150-200 bytes.
The module does not define any global variables.
No cacheing is performed by the module.
The owners of DataStream objects must delete them, deallocation of allocated memroy will occur automatically.
DataStream::WriteIntegerL(), DataStream::ReadIntegerL() uses g_memory_manager->GetTempBuf2()
DataStream::AddContentL(), DataStream_GenericFile::SampleDataL() uses g_memory_manager->GetTempBuf2k()
There is no check for external use of these buffers, and the different buffers should prevent internal collisions, unless implementations also use them in calls to/from these functions.
The Integer fucntions use these buffers instead of a stackallocated buffer, and it would be trivial to replace the calls with heap allocation, but it might cause a minor performance reduction
At present there is no opportunities to tune memory use.
Selftests, but no checks on memory usage
Loading a cache index or cookie file will exercise most of the non-PGP related code. The PGP selftests will exercise most of the other code.
The three-tier memory system in DataStream_ByteArray was chosen to avoid extra allocations for small payloads (which there are a lot of when loading cache index and cookie files), and to prevent reallocate and copy on successively larger payloads. This creates some overhead (processing and footprintwise) in order to handle the different buffers and upgrade to new buffers.
At present, no improvements are planned.