# Test Suite for the Digital Signatures For Widgets Specification

Lead test-case creator:
  Stuart Knightley, Opera Software ASA
Tools contributors:
  Kai Hendry, Aplix Corporation
  Stuart Knightley, Opera Software ASA
Maintainer:
  Marcos Cáceres, Opera Software ASA

Copyright © 2010 W3C® (MIT, ERCIM, Keio), All Rights Reserved. W3C liability, trademark and document use rules apply.

## Prologue

This document provides an overview of the test suite for the Digital Signatures For Widgets specification. The Working Group has developed this test suite so to have a test case for every conformance requirement in the specification.

The Working Group encourages the public and members to contribute tests.

## Status of this Test Suite

This is the first release of this test suite: this test suite is unstable and subject to change. The WebApps WG has sought review of the test cases from members of XMLSec and WebApps, the community at large, as well as any vendors who are implementing. The deadline for comment is the 30th of September. We are particularly interested in feedback on the completeness of testing (i.e., have we tested enough? do we need more tests)? Can the test suite actually be run on devices and runtimes? All and any comments are welcomed! Please send feedback to public-webapps.

## Licenses

This test suite is distributed under the W3C test suite licensing scheme. Note that claims of performance can only be made against unaltered tests. The Working Group encourages the public and W3C members to contribute tests. To contribute to a W3C Test Suite, see the policies and contribution forms.

## Introduction

The tests in this test suite, and supporting material, are all designed as much as possible to test only the features required for a user agent to conform to the specification.

### Method used for testing

This specification makes use of the guidelines described in A Method for Writing Testable Conformance Requirements. Please refer to that document for terminology and the approach that was taken to create this test suite.

### User Agent Requirements

Please note that this specification targets the three products identified in the specfication:

- Signer
- Verifier
- Implementaton

Each user agent is subtly different.

### Test file template

The test file template contains a `LICENSE` file, which should be signed, a simple `config.xml` file labelled with the test ID, and assertion ID, an `index.html` also labelled with the test and assertion IDs, with the background colour set to green is the test is expected to pass, red if it is expected to fail, and grey if the result needs to be verified externally.

To create a test, download the tools from the tools/ directory, the keys and certificates from the keys/ directory and the template directory. Use the `create-test-case.sh` script with the assertion ID and the new test ID, for example: `./create-test-case.sh ta-1 1a`. The tools are discussed further in the next section.

#### Signature IDs

For testing purposes the `dsp:Identifier` value for the signatures follows a common format.

`author-signature.xml`

      w3c-testsuite-id-<assertion id>-<test id>-author

`signature1.xml`

      w3c-testsuite-id-<assertion id>-<test id>

`signature<2..>.xml`

      w3c-testsuite-id-<assertion id>-<test id>-<2..>

For example the distributor signature identifier for `signature1.xml` for `ta-3` with test ID `3a` would be `w3c-testsuite-id-ta-3-3a`.

### Test suite tools

Several tools are available to aid the creation and validation of widgets. They take the form of several bash scripts and have the following package dependencies:

- uuid
- xmlsec >= 1.2.13

  At the time of writing this needs to be compiled from source, and requires the following libraries:

  - libxml2-dev
  - libxslt1-dev
  - libssl-dev
- xmlstarlet
- rnv
- openssl

#### create-test-case.sh

This tool will create a new test case by performing the following steps:

1. Create a new directory under the test-cases/ directory called `<ASSERTION ID>/<TEST ID>/`
2. Copy the files from test-cases/template/
3. Replace the ID place holders with the assertion ID and test ID
4. Create an author and two distributor signatures using the default key, `3.rsa.p12`

If you wish to use a different key/certificate, then delete the automatically created signatures and use sign-widget.sh to create new signatures.

#### sign-widget.sh

This tool will create author and distributor signatures for a widget (either zipped or in a directory).

#### validate-widget.sh

This tool will validate a widget (zipped or not), both by checking that the XML is well formed, and the files that have a ds:Reference. It does not check that all files in the widget have a ds:Reference.

#### repack-widget.sh

If you make a change to a widget in a directory this tool will zip it. It can also update the digests and signature in the signature file, and verify the widget's signatures.

#### create-keys.sh add

This tool can regenerate many of the keys needed for this test suite, however it is more useful for adding additional keys to the testCA by running `create-keys.sh add`. When provided with a command to generate the key (for example with OpenSSL) it will create a certificate and p12 file.

### Keys and certificates

For this test suite a test certificate authority called "testCA" has been created. The commands used to create the certificates are available in tools/create-keys.sh and the resulting files are in the keys/ directory. The password for all of the p12 files is "secret".

The following two certificates are used to test that implementations can perform basic path validation, but are not used to sign any widgets:

Root (key, cert, p12)
  This is the root certificate for the CA, and needs to be trusted for this test suite.
Second key/cert (key, cert, p12)
  This certificate is signed by the root certificate, and should be added as an untrusted certificate for this test suite.

The following keys and certificates are used to sign widgets:

RSA 2048 (key, cert, p12)
  Unless otherwise mentioned, this key should be used to sign/validate the test suite widgets.
DSA 2048 (key, cert, p12)
ECDSA (key, cert, p12)
RSA 512 (key, cert, p12)
RSA 2048, revoked (key, cert, p12, CRL)

## How to get the tests and test descriptions

Before you run the tests, please make sure you read this document thoroughly.

Tests can be downloaded using CVS using the following commands:

    CVSROOT=:pserver:anonymous@dev.w3.org:/sources/public cvs login

Then enter the password `anonymous` when prompted, and then:

    CVSROOT=:pserver:anonymous@dev.w3.org:/sources/public cvs checkout 2006/waf/widgets/test-suite/

The Working Group also provides an XML file that describes each test case, links to each test case, and identifies the assertion being tested.

## Tested assertions

The following assertions are not tested, because they are untestable:

> A signer MUST place a unique identifier string for the signature in the `dsp:Identifier` signature property. This MUST be a unique signing string for all widget signatures created by the signer.