Copyright © 2010 W3C® (MIT, ERCIM, Keio), All Rights Reserved. W3C liability, trademark and document use rules apply.
This document provides an overview of the test suite for the Digital Signatures For Widgets specification. The Working Group has developed this test suite so to have a test case for every conformance requirement in the specification.
The Working Group encourages the public and members to contribute tests.
This is the first release of this test suite: this test suite is unstable and subject to change. The WebApps WG has sought review of the test cases from members of XMLSec and WebApps, the community at large, as well as any vendors who are implementing. The deadline for comment is the 30th of September. We are particularly interested in feedback on the completeness of testing (i.e., have we tested enough? do we need more tests)? Can the test suite actually be run on devices and runtimes? All and any comments are welcomed! Please send feedback to public-webapps.
This test suite is distributed under the W3C test suite licensing scheme. Note that claims of performance can only be made against unaltered tests. The Working Group encourages the public and W3C members to contribute tests. To contribute to a W3C Test Suite, see the policies and contribution forms.
The tests in this test suite, and supporting material, are all designed as much as possible to test only the features required for a user agent to conform to the specification.
This specification makes use of the guidelines described in A Method for Writing Testable Conformance Requirements. Please refer to that document for terminology and the approach that was taken to create this test suite.
Please note that this specification targets the three products identified in the specfication:
Each user agent is subtly different.
The test file template contains
a LICENSE file, which should be signed,
a simple config.xml file labelled with the test ID, and assertion ID,
an index.html also labelled with the test and assertion IDs, with the background
colour set to green is the test is expected to pass, red if it is expected to fail,
and grey if the result needs to be verified externally.
To create a test, download the tools from the tools/ directory,
the keys and certificates from the keys/ directory
and the template directory. Use the
create-test-case.sh script with the assertion ID and the new test ID,
for example: ./create-test-case.sh ta-1 1a. The tools are discussed
further in the next section.
For testing purposes the dsp:Identifier value for the signatures follows a
common format.
author-signature.xmlw3c-testsuite-id-<assertion id>-<test id>-author
signature1.xmlw3c-testsuite-id-<assertion id>-<test id>
signature<2..>.xmlw3c-testsuite-id-<assertion id>-<test id>-<2..>
For example the distributor signature identifier for signature1.xml
for ta-3 with test ID 3a would be w3c-testsuite-id-ta-3-3a.
Several tools are available to aid the creation and validation of widgets. They take the form of several bash scripts and have the following package dependencies:
At the time of writing this needs to be compiled from source, and requires the following libraries:
This tool will create a new test case by performing the following steps:
<ASSERTION ID>/<TEST ID>/3.rsa.p12If you wish to use a different key/certificate, then delete the automatically created signatures and use sign-widget.sh to create new signatures.
This tool will create author and distributor signatures for a widget (either zipped or in a directory).
This tool will validate a widget (zipped or not), both by checking that the XML is well formed, and the files that have a ds:Reference. It does not check that all files in the widget have a ds:Reference.
If you make a change to a widget in a directory this tool will zip it. It can also update the digests and signature in the signature file, and verify the widget's signatures.
This tool can regenerate many of the keys needed for this test suite, however
it is more useful for adding additional keys to the testCA by running
create-keys.sh add. When provided with a command to generate the
key (for example with OpenSSL) it will create a certificate and p12 file.
For this test suite a test certificate authority called "testCA" has been created. The commands used to create the certificates are available in tools/create-keys.sh and the resulting files are in the keys/ directory. The password for all of the p12 files is "secret".
The following two certificates are used to test that implementations can perform basic path validation, but are not used to sign any widgets:
The following keys and certificates are used to sign widgets:
Before you run the tests, please make sure you read this document thoroughly.
Tests can be downloaded using CVS using the following commands:
CVSROOT=:pserver:anonymous@dev.w3.org:/sources/public cvs login
Then enter the password anonymous when prompted, and then:
CVSROOT=:pserver:anonymous@dev.w3.org:/sources/public cvs checkout 2006/waf/widgets/test-suite/
The Working Group also provides an XML file that describes each test case, links to each test case, and identifies the assertion being tested.
The following assertions are not tested, because they are untestable:
A signer MUST place a unique identifier string for the signature in the
dsp:Identifiersignature property. This MUST be a unique signing string for all widget signatures created by the signer.