Requirements for adding Specification assertions to TCK tests

Requirements for adding Specification assertions to TCK tests

We need more discussion around the requirements for adding assertions to new TCK tests. From current discussions up to now, no determination has been made yet as to whether new Specification TCK tests should include assertions IDs or if a different approach should be used.

References:

• Platform TCK Call #7 talked about test assertions (also watch recording of call).
• Platform TCK mailing list discussion on Assertion IDs.
• JCP TCK doc (see reference links near end of document).
• Oasis document on test assertions..
• W3C document on test assertions..

The historic purpose of maintaining a list of assertions is to identify the important test cases to cover with TCK tests (see the above JCP TCK doc link for a thorough description of the process). Such that the list of test assertions to cover writing tests for, is created before any TCK tests are added for a new Specification release.

Also remember that the TCK Process 1.1 document describes test assertions:

• The following scenarios are considered in scope for test challenges:
  • Claims that a test assertion conflicts with the specification.
  • Claims that a test asserts requirements over and above that of the specification.
  • Claims that an assertion of the specification is not sufficiently implementable.
  • Claims that a test is not portable or depends on a particular implementation.