You can also specify that only some of the tests in a group should be run by appending ':GLOB' to the argument.
As an example, to run the manual tests in the Testsuite tests
(which are all named something including manual), you can use
Then start opera using the arguments described above by adding them to the 'Debug' section of the MSVC settings panel for the Opera project.
The output will end up in the same place where compilation errors normally go.
Compile opera normally, but verify that you have SELFTEST = YES in your user settings.
Then start opera using the arguments described above. The output will be sent to stderr.
First you need to set 'Build Self Test Sources' as the target and build that. This will create the self test sources. Afterwards compile opera normally, but verify that you have the feature #define FEATURE_SELFTEST = YES. Remeber to check both features.h and quick-features.h.
Then start opera using the arguments described above. The output will be sent to stderr.
Includes the file X in the .ot-file.
Example:
inc.oth:
test ("This is an included test")
{
verify (TRUE);
}
anyfile.ot:
...
test("This is a test") { }
#include "inc.oth"
test("This is another test") { }
...
uni_char *x = UNI_L(#string "foo");
Level: any
Syntax:
//! String data here
Example:
//! This is a comment converted to a string. //! Quote characters, '\' and '"' are quoted.is equivalent to
"This is a comment converted to a string.\nQuote characters, '\\' and '\"' are quoted.\n"
Works just like printf in C/C++, but outputs to the same place as the normal selftest output.
Mainly intended to be used for debugging.
Simple util macro for managing temporary heap allocated objects in asynchronous tests. As async tests don't support finally clause proper deletion can become tricky. This macro simplifies it by scheduling all objects passed to it for deletion after completing test group. Example:
SomeClass* object = OP_NEW(SomeClass, ()); // Will be dealocated after selftest group finishes ST_delete_after_selftest(object);
If this macro fails(OOM) then the object will be deleted immidietly and the macro will fail the test. This macro also fails if object is NULL, so NULL check can be ommited. As deallocation uses OP_DELETE object must be allocted by OP_NEW(not malloc, or array new allocator).
Works just like printf in C/C++, but outputs to the same place as the normal selftest output.
Mainly intended to be used for debugging.
Compares the byte contents of two files.
Takes two string arguments, both file paths. Returns true if they are equal, otherwise false.
Paints the current document to an offscreen bitmap, which is then discarded. Forces paint events, even if the document is hidden.
Takes no arguments. Throws an exception on error.
Sends a mouse event to the current document, regardless of visibility.
Takes four arguments, a document x coordinate, a document y coordinate, a type string, and a mouse button index. The type string is "mousedown", "mouseup" or "mousemove", while the button index is 1 for the left mouse button, 2 for the right mouse button, and 3 for the middle mouse button.
Sends a keyboard event to the current document, regardless of visibility.
Takes three arguments, an HTML element to target, a type string, and an Opera key code. The type string is one of "keydown" and "keyup". The function throws an exception on error.
Description: Exceptions in ecmascript threads in asynchronous selftests are detected, and the test executing will fail regardless. However there are cases where the exceptions might be expected. Use allowexceptions to prevent the selftest system from treating exceptions as failures.
Level: test
Syntax:
test("test name") async;
Example async:
group "Group 1";
test("Test 1.1") async;// this test will fail
{
setTimeout(function(){
throw new Error('Hello');
},10);
}
test("Test 1.2") async; fails;// this test will fail also
{
setTimeout(function(){
throw new Error('Hello');
},10);
}
test("Test 1.3") async;
allowexceptions;// this test will pass
{
setTimeout(function(){
throw new Error('world!');
},10);
setTimeout(function(){
ST_passed();
},20);
}
Description: Makes a specific test asynchronous. An asynchronous test will not run if "now" is given to selftest on the command line. A test is synchronous by default. An asynchronous test will end and the next test start after the test calls one of the functions ST_passed() or ST_failed("error message"). These two functions work for both c++ and ecmascript tests.
'verify', 'verify_success', 'verify_trap', 'verify_status' and 'verify_string' cannot be used in asynchronous C++ tests.
Level: test
Syntax:
test("test name") async;
Example async:
group "Group 1";
test("Test 1.1") async; // this test will run asynchronously
{
// Do something
}
test("Test 1.2") // this test will run synchronously
{
// Do something
}
Description: As html, but you must specify the type and specifying a URL is not supported.
Level: group
Syntax:
data "TYPE" { DATA }
For a detailed description, see html
Example data:
data "text/plain" {
//! Plaintext document follows
}
Description: Adds a delay between tests. By default, 'post' will be used, to add a delay after the test has been run. 'pre' means that the delay will be before the test starts. The delay is given in seconds as a floating point number.
Level: test
Syntax:
test("test name") delay [pre | post]? FLOAT;
Example delay:
group "Group 1";
test("Test 1.1") delay 3; // after this test is finished, selftest will rest for 3 seconds.
{
// Do something
}
test("Test 1.2") delay post 3.5; // after this test is finished, selftest will rest for 3.5 seconds.
{
// Do something
}
test("Test 1.3") delay pre 1; // before this test starts, selftest will rest for 1 second.
// Since "Test 1.2" had a post delay of 3.5 seconds, selftest will
// rest for a total time of 4.5 seconds between "Test 1.2" and "Test 1.3".
{
// Do something
}
Description: Disables tests. Will disable all tests in a file or a specific test, depending on where it's put.
Toplevel 'disabled' can have a condition, the rest of the file will only be parsed if the condition is true. This is useful when you use features that are not present in older selftest versions.
Level: top or common_test
Syntax:
disabled; // On top level
disabled if(!TS_CAP_SANE_TESTEQ); // On top level, with condition
test("test name") disabled; {} // On test level
Example disabled on top level:
disabled; // None of the tests in this file will be run.
syntax error ignored
group "Test group 1";
test("Test 1.1")
{
verify(2 == 3);
}
Example disabled on common_test level:
group "Test group 1";
test("Test 1.1") disabled; // "Test 1.1" will not run.
{
verify(2 == 3);
}
test("Test 1.2") // This one will, however
{
verify(2 == 2);
}
Description: A function that will run after all tests in a group. Should do all the cleanup necessary. Normally used to cleanup the things initialized in setup.
Level: group
Syntax: exit { c++ statements }
Example exit:
group "Test group 1";
global
{
char* str;
}
setup
{
str = new char[10];
strcpy(str, "Hello")
}
exit
{
delete [] str;
}
test("Test 1.1")
{
verify(strcmp(str, "Hello") == 0); // Will most likely crash if setup section was removed.
}
Description: This test should fail to succeed, that is, at least one verify() in it should fail.
Level: common_test
Syntax: test("test name") fails;
Example fails:
group "Test group 1";
test("Test 1.1") fails; // This test will succeed, since one of its verify's fail.
{
verify(2 == 2);
verify(2 == 3);
verify(3 == 4);
}
test("Test 1.1") fails; // This test will fail, since all of its verify's succeed.
{
verify(6 == 6);
verify(3 == 3);
verify(5 == 5);
}
Description: File is used to declare any external files used by the test. This allowes the selftest parser to extract a list of external files needed by the tests. It also creates a full path for you that can be used to access the file. Note that files in tables created with filelist automatically gets recognized and does not need to be declared further.
Level: test
Syntax: file [uni] identitifier "filename";
Example file:
group "Test group 1";
test("Test 1.1")
file uni testfile_uni "file.txt";
file testfile "file.txt";
{
OpString filename;
// testfile is expanded to a char* version of a full path to 'file.txt'
filename.Set(testfile);
// testfile_uni is a uni_char* string.
verify(uni_strcmp(testfile_uni, filename.CStr()) == 0);
OpFile file;
verify(OpStatus::IsSuccess(file.Construct(testfile_uni)));
verify(OpStatus::IsSuccess(file.Open(OPFILE_READ)));
verify(OpStatus::IsSuccess(file.Close()));
}
Description: Finally adds some code to a test, that will always run, even if a verify fails. You cannot use verify inside the finally block.
Level: test_body (special)
Syntax: test("test name") { c++ statements } finally { c++ statements }
Example finally:
group "Test group 1";
test("Test 1.1") // A test that fail will print "Finally", but will not print "Success".
{
verify(2 == 3);
output("Success");
}
finally
{
output("Finally");
}
test("Test 1.2") // A test that succeed will print first "Success" and then print "Finally".
{
verify(2 == 2);
output("Success");
}
finally
{
output("Finally");
}
Description: Will expand everything inside its body to variants with data from the supplied table. A way to create a multiple of tests. Good if you want to test with different data, without risking to stop testing just because one iteration failed (as in iterate). Right now you can also use foreach inside a test, but its usage is discuraged.
Level: group
Syntax: foreach (IDLIST) from [ID | { TABLE_CELL_LIST }] { GROUP_CONTENT }
Example foreach:
group "Test group 1";
table Table1(int, int, char*)
{
{ 3, 3, "Hello" },
{ 4, 54, "Hi" },
{ 5, 65, "Hej" },
{ 1, 100, "Hejsan" }
}
foreach (x, y, tooltip) from Table1
{
test("Test 1.1 " tooltip)
{
verify(x == y); // This will create 4 tests, with the first test succeeding, and the rest is failing.
}
}
foreach (x, y, tooltip) from Table1 // this will create 4 versions of Test 2.1, and 4 versions
// of Test 2.2. The first version of Test 2.1 will succeed,
// the rest of the Test 1.1 versions will fail. Test 2.2
// will only fail in the first version.
{
test("Test 2.1 " tooltip)
{
verify(x == y);
}
test("Test 2.2 " tooltip)
{
verify(x != y);
}
}
foreach (x, y, tooltip) from Table1 // this will create 16 versions of Test 3.1.
// Warning, it is easy to create so many tests
// that the system will fail due to limitations in VC++.
{
foreach (x1, y1, tooltip1) from Table1
{
test("Test 3.1 " tooltip ", " tooltip1)
{
verify(x == x1 && y == y1);
}
}
}
foreach (x, y, tooltip) from { { 5, 5, "Hello" }, { 4, 4, "HelloHi" } }
// foreach can also have a table included directly.
{
test("Test 4.1 " tooltip)
{
verify(x == y);
}
}
foreach (x, y, test_name) from { { 5, 5, Hello }, { 4, 4, HelloHi } }
{
test("$(test_name)") // The first test will be called "Hello", the second on "HelloHi".
{
verify(x == y);
}
test("Inverted test $(test_name)")// The first test will be called "Inverted test Hello",
// the second on "Inverted test HelloHi".
{
verify(y == x);
}
}
Description: Includes a number of functions, classes, variables, etc. that is normal C++ code, and can be used by all tests in a group. It supports an optional string, before the code block, which is the name of a class the test group will inherit, allowing multiple selftest groups in different files sharing the same set of utilities.
Level: group
Syntax: global { c++ statements } | global BASE { c++ statements }
Example global:
group "Test group 1";
global
{
class Value
{
public:
void SetValue(int val);
int GetValue() { return value; }
private:
int value;
};
void Value::SetValue(int val)
{
val.value = val;
}
}
test("Test 1.1") // Using the Value class defined in the global section
{
int nr = 3;
Value value;
value.SetValue(nr);
verify(nr == value.GetValue());
}
group "Test group 2";
test("Test 2.1") // Cannot use Value class, since this test is in a
// different group that has no global section
{
}
group "Test group 3"
include "utilities.h"
global "BaseUtilities"
{
// More things can be declared here.
BOOL UseUtility2(){ return UseUtility(g_variable); }
}
setup
{
g_variable = SOME_VALUE;
}
test("Test 3.1")
{
verify(UseUtility());
verify(UseUtility2());
}
// in utilities.h
class BaseUtilities
{
BOOL UseUtility(int input){ ... }
int g_variable;
};
Description: Starts a new group of tests. The group is ended by the end of the file or by another group statement. The description of a test will include the group name and the test name.
Level: top
Syntax: group "text string";
Example: group "OpString tests";
Description: You can use a html { STRING } or html URL { STRING } block to set the contents of the current document. You can use comment-to-string parsing for the document contents. The contents of the block are parsed as text/html. The URL is optional and will be used as the document's effective url, affecting too the scripting origin. Note, if an URL is specified, then there must be some content in the block, else the empty document would result in a network request. This rule is enforced by the selftest parser.
Level: top
Syntax: html { STRING } | html URL { STRING }
Example html:
group "Test group 1";
html {
//! <body bgcolor="white" text="black">
//! <h1>Hello world!</h1>
//! </body>
}
// equivalent to
html {
" <body bgcolor=\"white\" text=\"black\">\n"
" <h1>Hello world!</h1>\n"
" </body>\n"
}
// or
html " <body bgcolor=\"white\" text=\"black\">\n <h1>Hello world!</h1>\n </body>\n"
// example: spoofing an url
html "http://www.google.com/" { "<h1>We're evil</h1>" }
test("Evilness?")
language ecmascript;
{
verify(location.href == 'http://www.google.com/'); // Passes
verify(document.getElementsByTagName('h1')[0]);
}
Description: Includes a header file for use inside a test group. Please use this include command instead of using #include. The reason is that all include files will be placed in the top of the generated test file, avoiding include duplicates.
Optionally there can be a conditional expression that causes the file to only be included if certain requirements (in addition to any toplevel requires in the file) are met.
The syntax for the conditional expression is if(CONDITIONAL [&& CONDITIONAL [...]]). Using '(' and ')' in the expressions is optional.
CONDITIONAL is one of the following:
Level: group
Syntax: include [CONDITIONAL] "header file";
Example:
group "Test group 2";
include "modules/util/simset.h"; // simset.h can be used in all of the group.
include if exists "legacy.h"; // include legacy.h if it exists
include if(exists) "legacy.h"; // Identical to the above
include if(defined(X)&&undefined(Y)) "legacy.h"; // include legacy.h if X is defined and Y is not defined
include
if(
exists
&& defined(X)
&& undefined(Y)
)
"legacy.h"; // same as above
// And, finally, since preprocessing is done before the include tokens
// are parsed, you can do some rather odd things:
constant include_condition=(defined(FOO) && undefined(BAR));
include if(include_condition) "include_file";
test("Test 1.1")
{
Head h;
}
Description: Allows you to import tests from another test group into the current group. Use it responsibly.
Level: group
Syntax: import "group[,group,...]";
Example import:
group "selftest.basic"; import "selftest.tables"; // No we will run 'selftest.tables' whenever 'selftest.basic' is run
Description: Iterates the contents in a table. See desciption of table for details of how the table content can look. This is basically a for() loop over table data. The table used must be typed.
Level: test_body
Syntax: iterate (IDLIST) from ID { BODY }
Example iterate:
group "Test group 1";
table Table1(int, int, char*)
{
{ 3, 3, "Hello" }
{ 4, 54, "Hi" }
{ 5, 65, "Hej" }
{ 1, 100, "Hejsan" }
}
test( "Test 1.1" )
{
iterate (x, y, tooltip) from Table1
{
verify(x == y); // This test will fail on the second table row, since 4 != 54.
}
}
Description: Defines which language that is used to write tests. The language can be one of c, c++, or ecmascript. Language can be defined for the following tests in a group, or defined for a specific test. Default language is c++.
Level: group, test
Syntax:
language lang; // On group level
test("test name") language lang; // On test level
Example language:
group "Test group 1";
language ecmascript;
test("Test 1.1"); // Written in ecmascript.
{
verify(2 == 3);
}
test("Test 1.2") // Written in ecmascript.
{
verify(2 == 2);
}
group "Test group 2";
test("Test 2.1") // Written in c++
{
verify(1 == 3);
}
language ecmascript;
test("Test 2.2") language c++; // Written in c++.
{
verify(1 == 2);
}
test("Test 2.3") // Written in ecmascript.
{
verify(1 == 2);
}
language c++;
test("Test 2.4") // Written in c++.
{
verify(1 == 2);
}
test("Test 2.5") // Written in c++.
{
verify(1 == 2);
}
Description: Indicates that leakchecking should be performed for the test
Level: common_test
Syntax: test("test name") leakcheck;
Example:
group "Test group 1";
test("Test 1.1") leakcheck; // Will report that this test leak memory.
{
char* str = new char[10];
}
test("Test 1.2") // Will not report that this test leak memory.
{
char* str = new char[10];
}
test("Test 1.3") leakcheck; // Will not report that this test leak memory, since it doesn't leak.
{
char* str = new char[10];
delete [] str;
}
Manual tests are always asynchronous, and thus can not be used when opera has not been initialized
Description: Indicates that a test requires manual interaction
Level: group, test
Syntax: test("test name") manual "message";
Syntax: manual("test name", "message")
Example, test level:
test( "Manual question" )
manual "Are you there?";
{
// We can place code here, and use modifiers above. The code is run
// before the question is put to the user.
}
Example, group level:
manual( "Manual question", "Are you there?" );
// Shorthand for the very common case where you have an empty test
// body. Most commonly this statement is preceded by html {}
Description: multi is a way to run multiple tests from a table without using foreach. It's basically identical to doing iterate on a table, but multiple tests are run instead of just one.
Level: test
Syntax: multi TABLENAME(IDLIST)
Example usage:
// Create the table test_file from a file, with char*, double and int columns.
// Multi requires that the columns are typed (since it's a runtime, not a macro, construct)
table test_file(char *, double, int) read "test_table.txt";
// All $(X), where X comes from the IDLIST in multi, are replaced with the current value.
test( "table test : $(text)" )
multi test_file(text,f,i);
{
// test, f and i will take their values from the table.
verify( text )
verify( i );
verify( i == op_ceil(f) );
}
Level: top
Syntax: nopch;
Description: Defines the number of times a test should run. Default is 1. If the test fails in the n:th run, the test will not run any more time.
Level: test
Syntax: test "test name" repeat INTEGER;
Example repeat:
group "Test group 1";
test("Test 1.1") repeat 3; // Will run 3 times.
{
verify(a == b);
}
test("Test 1.2"); // Will only run once.
{
verify(b == c);
}
Description: Defines a requirement for a test or a group.
require init; This is a test or group that needs to be run after Opera has started. If the command line argument given to selftest included now, this test will not run.
require noinit; This is a test or group that needs to be run before Opera has started. If the command line argument given to selftest did not include now, this test will not run.
require success "test name"; Requires that the test "test name" succeeded, otherwise this test will not run. Does not work on group level.
require failure "test name"; Requires that the test "test name" fails, otherwise this test will not run. Does not work on group level.
require undefined ID; Requires that the define ID is not defined, otherwise this test or group will not run.
require ID; Requires that the define ID is defined, otherwise this test or group will not run.
Level: common_test
Syntax:
group "Group";
require ID; // group level
test("testname") require ID; // test level
Example require:
group "Test group 1";
require init; // The whole group requires initialization
test("Test 1.1") require FOOBAR_SUPPORT;
// Will only run if FOOBAR_SUPPORT is defined.
{
verify(a == b);
}
test("Test 1.2") require undefined FOOBAR_SUPPORT;
// Will only run if FOOBAR_SUPPORT is not defined.
{
verify(b == c);
}
group "Test group 2";
test("Test 2.1") require init;
// Will only run if now is not defined on the command line argument given to selftest.
{
verify(b == c);
}
test("Test 2.2") require noinit;
// Will only run if now is defined on the command line argument given to selftest.
{
verify(b == c);
}
test("Test 2.3") require success "Test 2.1";
// Will only run if "Test 2.1" has been run and succeeded.
{
verify(b == c);
}
test("Test 2.4") failure "Test*:Test 1.2";
// Will only run if all tests named "Test 1.2" in groups whose
// name starts with Test has been run and failed.
{
verify(b == c);
}
test("Test 1.7") success "Test 2.1";
failure "Test 2.2";
// Will only run if "Test 2.1" was run and succeeded, and "Test 2.2" was not run or failed.
{
verify(b == c);
}
Description: A function that will run before all tests in a group. Should setup
all global things that the tests need. If you need to run setup code that might fail in a
way that affects the selftests after, like OOM, then use a test block instead, and make the
subsequent tests depend on the former, using require success "test name".
Level: group
Syntax: setup { c++ statements }
Example setup:
group "Test group 1";
global
{
int var;
}
setup
{
var = 3;
}
test("Test 1.1")
{
verify(var == 3); // Will most likely fail if setup section was removed.
}
Description: A subtest is code written in c++. It takes a number of
parameters and is used by tests as subtests. A subtest can as a test include
verify and iterate statements. If a subtest fails, it returns 0, if the
subtest passes, it returns 1. To use a subtest in a synchronous test, call it
as verify(MySubTest()); thus the test calling it will fail. A
subtest can have no modifiers, and must be called by a test written in c++.
Level: group
Syntax: subtest testname(parameters) { body }
Example subtest:
group "Test group 1";
subtest Subtest1(int a)
{
verify(a == 2);
}
subtest Subtest2(int a, int b)
{
verify(a == b);
}
subtest Subtest3(int a) // A subtest can call another subtest
{
verify(Subtest2(a, 2));
}
test("Test 1.1")
{
verify(Subtest1(2)); // This test will succeed, since SubTest1 succeeds.
}
test("Test 1.2")
{
verify(Subtest1(1)); // This test will fail, since SubTest1 fails.
}
test("Test 1.3")
{
verify(Subtest2(2, 2)); // This test will succeed, since SubTest2 succeeds.
}
test("Test 1.4")
{
verify(Subtest2(2, 3)); // This test will fail, since SubTest2 fails.
}
test("Test 1.5")
{
verify(Subtest3(2)); // This test will succeed, since SubTest3 succeeds.
}
Description: As html, but the type is image/svg+xml
Level: group
Syntax:
svg { DATA } | svg URL { DATA }
For a detailed description, see html
Example svg:
svg {
//! <?xml version='1.0' charset='utf-8'?>
//! <svg xmlns="www.w3.org/2000/svg">
//! <!--SVG document-->
//! </svg>
}
Description: A table is an array of value arrays. We can use foreach and iterate on a table. A table is very useful to supply data to tests. A table's content is normally an array of literal data. A table can also be a tab separated text file with table data (when using the read modifier). Last, but not least, it can be a simple list of directories and filename globs.
The type list is describing the types in the cells, and can be used in combination with all table variants. It's needed if iterate() will be used on the table, and provides hints for the file parser (e.g., a char* or uni_char* cell will always be a string, even if it's not a string in the table file, and uni_char * strings will have UNI_L() added)
The DIRECTORIES and GLOBS are a list of or-separated string ("foo" or "bar" or "gazonk")
Level: group
Syntax:
table ID[(TYPE_LIST)]? { TABLE_CELL_LIST }
table read STRING;
table filelist DIRECTORIES [name GLOBS]? [recursively]? [directories]?
Example table:
group "Test group 1";
table Table1(int, int, char*)
{
{ 3, 3, "Hello" }
{ 4, 54, "Hi" }
{ 5, 65, "Hej" }
{ 1, 100, "Hejsan" }
}
table Table2 // it is ok to use an untyped table.
{
{ 3, 3, "Hello" }
{ 4, 54, "Hi" }
{ 5, 65, "Hej" }
{ 1, 100, "Hejsan" }
}
// table.txt, content (the list only needs to be space separated, I
// used tab separated above in the text since it is a more common description.
// 3 3 "Hello"
// 4 54 "Hi"
// 5 65 "Hej"
// 1 100 "Hejsan"
table Table3(int, int, char*) read "table.txt";
// Read a table from table.txt. Will be the same table content as in Table1.
table Table4 read "table.txt";
// Read a table from table.txt. Will be the same table content as in
// Table2, but untyped. This table can not be used for iterate, but
// can be used for foreach
table Table5(char*) filelist "." name "*.h";
// Let the content be all file names ending with .h found in the current directory
table Table6 filelist "." name "*.h";
// Same content type as Table5 since char* is the default cell type for the file name.
table Table7 filelist "." name "*.h" or "*.cpp"
// Let the content be all file names ending with .h or .cpp found in the current directory.
table Table8 filelist "." name "*.h" recursively;
// Let the content be all file names ending with .h in the current directory
// and its subdirectories.
table Table9 filelist "." directories;
// Let the content be all file and directory names in the current directory.
table Table10 filelist "." or "../.." name "Makefile*";
// All files whose name starts with Makefile in the current directory
// or the directory two levels up in the directory hierarchy
Description: A syntax for writing tests. Depending on defined language, the test can include c, c++, or ecmascript code. The test can also include verify statements and iterate statements. The test can have a number of modifiers with syntax level test or common_test. Finally, the test can have a finally clause. A test can also call subtests.
Level: group
Syntax: test("test name") { body }
Example test:
group "Test group 1";
test("Test 1.1")
{
verify(2 == 2); // This test will succeed.
}
test("Test 1.2")
{
int i = 3;
verify(2 == i); // This test will fail.
}
Description: A syntax for writing simple tests that only verify that two expressions return the same value. test_equal can as modifiers have all keywords with level common_test.
Level: group
Syntax: test_equal("test name", expression_1, expression_2);
Example test_equal:
group "Test group 1";
test_equal("Test 1.1", 2, 2);
// compares if 2 and 2 is equal, will succeed
test_equal("Test 1.2", 2, 3);
// compares if 2 and 3 is equal, will fail
test_equal("Test 1.3", 2.5, 2.5);
// compares if 2.5 is equal to 2.5, will succeed
test_equal("Test 1.4", "Hello", "Hello");
// compares if the string "Hello" has the same content as "Hello", will succeed
test_equal("Test 1.6", GetValue1() + 1, GetValue2());
// Will call the functions GetValue1() and
// GetValue2(), and test if GetValue1() + 1 == GetValue2().
The extra ';' after the options is nessesary, since each option is
ended with a ;, as is the test declaration.
test_equal("Test 1.7", GetValue1() + 1, GetValue2())
leakcheck;
;
// an example of using a modifier on the test. Will report if we have
// leaked memory by running this test.
Description: A syntax for writing simple tests that only verify that two expressions return different values. test_nequal can only have the keywords allowed in common_test as modifiers.
Level: group
Syntax: test_nequal("test name", expression_1, expression_2);
Example test_nequal:
group "Test group 1";
test_nequal("Test 1.1", 2, 2); // compares if 2 and 2 is not equal, will fail
test_nequal("Test 1.2", 2, 3); // compares if 2 and 3 is not equal, will succeed
test_nequal("Test 1.3", 2.5, 2.5); // compares if 2.5 is not equal to 2.5, will fail
test_nequal("Test 1.4", "Hello", "Hello"); // compares if the string "Hello" has different
// content than "Hello", will fail
test_nequal("Test 1.5", p1, p2);
// compares if the pointer p1 points to a different address compared to p2
The extra ';' after the options is nessesary, since each option is
ended with a ;, as is the test declaration.
test_nequal("Test 1.5", p1, p2)
fails;;
// Silly way to write test_equal( "Test 1.5", p1, p2 )
test_nequal("Test 1.6", GetValue1() + 1, GetValue2()); // Will call the functions GetValue1() and
// GetValue2(), and test if GetValue1() + 1 != GetValue2().
Description: As html, but the type is text/plain
Level: group
Syntax:
text { DATA } | text URL { DATA }
For a detailed description, see html
Example text:
text {
//! If a<b, then b>a
}
Description: Measures the time a test takes to execute. This is usable with both asynchronous and synchronous test. Note that the time is measured with the system timer so don't expect to much accuracy.
Level: test
Syntax:
test("test name") timer; { ... }
Example text:
test("t")
repeat 1000;
timer;
{
/* Do something */
}
Description: Verify that an expression evaluates to true. A test will stop running the test body immediately if a verify statement fails and a subtest will return to its calling test if a verify statement fails. The optional message is appended to the output and allows the test to provide extra info about what should happen if the test fails. The message needs not to be wrapped in quotes, and if so, can span multiple lines.
Level: test_body
Syntax: test("test name") { verify(boolean_expression[, message]); }
Example:
group "Test group 1";
test("Test 1.1") // Will succeed and print "Success1" and "Success2"
{
verify(2 == 2);
output("Success1");
verify(3 == 3);
output("Success2");
}
test("Test 1.2") // Will fail and not print anything
{
verify(2 == 3);
output("Success1");
verify(3 == 3);
output("Success2");
}
test("Test 1.3") // Will fail after printing "Success1"
{
verify(2 == 2);
output("Success1");
verify(2 == 3);
output("Success2");
}
subtest SubTest(int a, int b)
{
verify(a == b);
}
test("Test 1.4") // Will succeed and print "Success", since SubTest() will succeed.
{
verify( SubTest(2, 2) );
output("Success");
}
test("Test 1.5") // Will fail and not print anything, since SubTest() will fail.
{
verify( SubTest(2, 3) );
output("Success");
}
test("Test 1.6") // Will fail if a != b and append the optional message in the error output
{
verify(a == b, a did not have the same value as b);
}
Description: verify_files_equal has the exact same semantics as verify. It checks if two files are equal, byte for byte. It means that the files' endings must also match.
The file paths should be absolute. In most cases the 'file' directive will be
used to provide full absolute paths to files in the selftest data directory.
Since the 'file' directive doesn't work in EcmaScripts you have to be inventive
there. One of the solutions is to create a pseudo test in C++ that adds variables to
EcmaScript runtime before the actual test is run.
Just like verify, verify_files_equal also supports an optional error message
Level: test_body
Syntax: test("test name") { verify_files_equal(reference_file_path, tested_file_path[, message]); }
Example:
group "Test group 1";
test("Test 1.1")
file uni reference_file_str "reference_file_contents.txt";
file uni test_file_str "output_file.txt";
{
/* generate some data in file output_file.txt */
verify_files_equal(reference_file_str, test_file_str); // succeeds if files have identical content
}
Description: verify_not_oom has the exact same semantics as verify. It checks if a pointer has a NULL value, and if so, reports an Out-of-memory error, similar to verify_status. This can be used to check that allocations during testing succeed.
Just like verify, verify_not_oom also supports an optional error message
Remarks: this function is only available in C++ tests.
Level: test_body
Syntax: test("test name") { verify_not_oom(expr[, message]); }
Example:
group "Test group 1";
test("Test 1.1")
{
Object x = new Object();
verify_not_oom(x); //succeeds if allocation does not fail
delete x;
}
test("Test 1.1") fails; //test succeeds
{
verify_not_oom(NULL);
}
Description: verify_string has the exact same semantics as verify, but allows easy and quick comparison of strings. When the strings are not equal, they are output, together with the error log. You can combine the following four types in any of the arguments.
Just like verify, verify_string also supports an optional error message
Remarks:
This function is only available in C++ tests.
This function treats an uninitialized OpString as equal to an empty string, and not equal to NULL.
Consider this:
verify_string(UNI_L(""), OpStringC()); // passes
verify_string(NULL, OpStringC()); // fails
Level: test_body
Syntax: test("test name") { verify_string(lhs, rhs[, message]); }
Example:
group "Test group 1";
test("Test 1.1") //succedds
{
verify_string("hello", "hello");
OpStringC16 str_1(UNI_L("hello"));
verify_string("hello", str_1);
verify_string(NULL, NULL);
}
test("Test 1.2") //fails and prints the two compared strings in the output
{
verify_string("not", "equal");
}
Description: verify_success(), verify_status() and verify_trap() have
the exact same semantics as verify, but slightly
different use cases. verify_success(expression) and
verify_status(expected_status, expression) test the result of an expression
with the return value of an OP_STATUS, verify_trap(FunctionL())
can be used to verify the success of function call that may
LEAVE.
verify_success() and verify_trap() take only one argument and test if the
value returned is success, resp. if the called function did not LEAVE (it
calls Opstatus::IsSuccess() implicitly).
verify_status(expected_status, expression) takes two arguments and tests if
the value returned by the expression (2nd argument) equals the reference value
(1st argument). If the conditions fail, the resulting error code, from the 2nd
argument, is output, and for some known values, like those in OpStatus, it's
converted into a readable string.
These functions, just like verify, also support an optional error message.
Remarks: these functions are only available in C++ tests. It's highly recommended to use this function to check for status codes, because foreign conditions like out of memory can be flagged in SPARTAN as ERRORs, instead of FAILs.
Level: test_body
Syntax: test("test name") { verify_success(expression[, message]); verify_trap(expression with a function that may leave[, message]); verify_status(error_code, expression[, message]); }
Example:
group "Test group 1";
global {
int MyFunctionL(const char* arg) { if (!arg) LEAVE(OpStatus::ERR_NULL_POINTER); return 17; }
OP_STATUS MyFunction(void* arg){ return arg ? OpStatus::OK : OpStatus::ERR_NULL_POINTER; }
OP_STATUS GetOOM(){ return OpStatus::ERR_NO_MEMORY; }
}
test("Test 1.1") //prints Success if Myfunction returns OpStatus::OK
{
int x;
verify_success(MyFunction(&x));
output("Success");
}
test("Test 1.2") //prints Success if Myfunction returns OpStatus::ERR_NULL_POINTER
{
verify_status(OpStatus::ERR_NULL_POINTER, MyFunction(NULL));
output("Success");
}
test("Test 1.3") //if this test runs on spartan, it can tell that it failed due to oom, and not some unknown error
{
verify_success(GetOOM());
}
test("Test 1.4")
{
int x;
verify_trap(x = MyFunctionL("foo")); // passes
verify_trap(x = MyFunctionL(0)); // fails, because MyFunctionL() leaves
}
Description: As html, but the type is text/xml
Level: group
Syntax:
xml { DATA } | xml URL { DATA }
For a detailed description, see html
Example xml:
xml {
//! <?xml version='1.0' charset='utf-8'?>
//! <document>
//! XML document
//! </document>
}