From e4eff10438d41ebe7c5fb1cc7c8dbe7fd99b10e6 Mon Sep 17 00:00:00 2001 From: Greg Dennis Date: Sat, 12 Oct 2024 22:44:34 +1300 Subject: [PATCH 1/2] add a root object to the output --- .../jsonschema-validation-output-machines.md | 344 ++++++++++-------- specs/output/schema.json | 46 +-- 2 files changed, 207 insertions(+), 183 deletions(-) diff --git a/specs/output/jsonschema-validation-output-machines.md b/specs/output/jsonschema-validation-output-machines.md index b9d1787d..23751f10 100644 --- a/specs/output/jsonschema-validation-output-machines.md +++ b/specs/output/jsonschema-validation-output-machines.md @@ -3,74 +3,88 @@ JSON Schema is defined to be platform-independent. As such, to increase compatibility across platforms, implementations SHOULD conform to a standard -validation output format. This specification describes the minimum requirements -for machine consumers to properly interpret validation results. +validation output format. This specification describes a number of output +formats intended for machine consumers. ## Table of Contents ## Schema Identifiers -The output defined in this specification requires that the evaluation root be -defined with an absolute IRI, i.e. using the `$id` keyword. In the event an -absolute IRI has not been defined, the implementation MUST generate one. - -There are no requirements on the form of IRI itself, except that it MUST be -absolute. +The output defined in this specification requires that an initial IRI is defined +per [JSON Schema Core](./json-schema-core#initial-base) "Initial Base IRI" with an +[absolute IRI](#rfc3987). In the event an absolute IRI has not been defined, the +implementation MUST generate one. ## Textual Format and Encoding JSON Schema output is defined using the JSON Schema data instance model as -described in [JSON Schema](#json-schema) "Instance Data Model". Implementations -MAY deviate from this in their internal modelling, as supported by their -specific languages and platforms, however it is RECOMMENDED that the output be -convertible to the JSON format defined herein via serialization or other means. +described in [JSON Schema Core](./jsonschema-core#data-model) "Instance Data +Model". Implementations MAY deviate from this in their internal modelling, as +supported by their specific languages and platforms, however it is RECOMMENDED +that the output be convertible to the JSON format defined herein via +serialization or other means. + +## The Output Object + +The output of a subschema validation is a JSON object which contains general +information about the validation as well as a set of +[output units](#output-unit) which contain specifics. + +The root object MUST contain the following properties: + +| Property | Value | +|:-|:-| +| `dialect` | The `$schema` IRI of the evaluating schema. | +| `schema` | The identifier `$id` or otherwise used to identify the evaluating schema. | +| `valid` | A boolean indicating the overall validation result. | -## Minimum Information +For the "list" and "hierarchical" formats, the following is also required: -Beyond the simplistic "flag" output, additional information is useful to aid in -debugging evaluation of an instance by a schema. +| `details` | A collection of output units to provide additional details. Must be an array, even when containing a single output unit. | -The output of a subschema validation is considered an "output unit." The -contents of each output unit is specified by this section. +### Output Unit + +An output unit describes the evaluation of a single subschema against the appropriate instance location. Each output unit MUST contain the [validation result](#validation-result) for the associated subschema as well as the following information defined by -[JSON Schema](#json-schema) "Output Formatting": +[JSON Schema](./jsonschema-core#output) "Output Formatting": -- Evaluation Path -- Schema Location -- Instance Location +- Evaluation Path (as `evaluationPath`) +- Schema Location (as `schemaLocation`) +- Instance Location (as `instanceLocation`) The following information MAY be included conditionally: - When subschema validation has succeeded - - Annotations + - Annotations (as `annotations`) - When subschema validation has failed - - Errors - - Dropped Annotations + - Errors (as `errors`) + - Dropped Annotations (as `droppedAnnotations`) + +When including this information, the indicated property names MUST be used. Implementations MAY elect to provide additional information. -### Validation Result {#validation-result} +#### Validation Result {#validation-result} The validation result is a boolean that indicates whether the local instance passed validation by the local subschema. The JSON key for these additional results is `valid`. -### Results from Subschemas +#### Results from Subschemas -Evaluation results generated by applying a subschema to the instance or a child +Similar to the root object's `details` property, this property contains +evaluation results generated by applying a subschema to the instance or a child of the instance. Keywords which have multiple subschemas (e.g. `anyOf`) will generally generate an output unit for each subschema. In order to accommodate potentially multiple results, the value of this property MUST be an array of output units, even if only a single output unit is produced. -For "list", this property will appear only at the root output unit and will hold -all output units in a flat list. - -For "hierarchical", this property will contain results in a tree structure where -each output unit may itself have further nested results. +This property of the output unit is only used for the "hierarchical" format and +will contain results in a tree structure where each output unit may itself have +further nested results. The sequence of output units within this list is not specified and MAY be determined by the implementation. Sets of output units are considered equivalent @@ -230,6 +244,8 @@ omitted. ```json "Flag Results" { + "dialect": "https://json-schema.org/draft/next/schema", + "schema": "https://json-schema.org/schemas/example", "valid": false } ``` @@ -242,14 +258,8 @@ check the other three. The logic can simply return with success. ### List -The "list" structure is a flat list of output units contained within a root -output unit. - -The root output unit contains `valid` for the overall result and `details` for -the list of specific results. All other information is explicitly omitted from -the root output unit. If the root schema produces errors or annotations, then -the output node for the root MUST be present within the root output unit's -`details` list with those errors or annotations. +The "list" structure is a flat list of output units contained within the root +object's `details` array. Output units which do not contain errors or annotations SHOULD be excluded from this format, however implementations MAY choose to include them for @@ -257,6 +267,8 @@ completeness. ```json "Failing Results" { + "dialect": "https://json-schema.org/draft/next/schema", + "schema": "https://json-schema.org/schemas/example", "valid": false, "details": [ { @@ -292,6 +304,8 @@ completeness. ```json "Passing Results" { + "dialect": "https://json-schema.org/draft/next/schema", + "schema": "https://json-schema.org/schemas/example", "valid": true, "details": [ { @@ -369,85 +383,92 @@ The location properties of the root output unit MAY be omitted. ```json "failing Results { + "dialect": "https://json-schema.org/draft/next/schema", + "schema": "https://json-schema.org/schemas/example", "valid": false, - "evaluationPath": "", - "schemaLocation": "https://json-schema.org/schemas/example#", - "instanceLocation": "", "details": [ { "valid": false, - "evaluationPath": "/properties/foo", - "schemaLocation": "https://json-schema.org/schemas/example#/properties/foo", - "instanceLocation": "/foo", + "evaluationPath": "", + "schemaLocation": "https://json-schema.org/schemas/example#", + "instanceLocation": "", "details": [ { "valid": false, - "evaluationPath": "/properties/foo/allOf/0", - "schemaLocation": "https://json-schema.org/schemas/example#/properties/foo/allOf/0", + "evaluationPath": "/properties/foo", + "schemaLocation": "https://json-schema.org/schemas/example#/properties/foo", "instanceLocation": "/foo", - "errors": { - "required": "Required properties [\"unspecified-prop\"] were not present" - } - }, - { - "valid": false, - "evaluationPath": "/properties/foo/allOf/1", - "schemaLocation": "https://json-schema.org/schemas/example#/properties/foo/allOf/1", - "instanceLocation": "/foo", - "droppedAnnotations": { - "properties": [ "foo-prop" ], - "title": "foo-title" - }, "details": [ { "valid": false, - "evaluationPath": "/properties/foo/allOf/1/properties/foo-prop", - "schemaLocation": "https://json-schema.org/schemas/example#/properties/foo/allOf/1/properties/foo-prop", - "instanceLocation": "/foo/foo-prop", + "evaluationPath": "/properties/foo/allOf/0", + "schemaLocation": "https://json-schema.org/schemas/example#/properties/foo/allOf/0", + "instanceLocation": "/foo", "errors": { - "const": "Expected \"1\"" - }, - "droppedAnnotations": { - "title": "foo-prop-title" + "required": "Required properties [\"unspecified-prop\"] were not present" } }, { - "valid": true, - "evaluationPath": "/properties/foo/allOf/1/additionalProperties", - "schemaLocation": "https://json-schema.org/schemas/example#/properties/foo/allOf/1/additionalProperties", - "instanceLocation": "/foo/other-prop" + "valid": false, + "evaluationPath": "/properties/foo/allOf/1", + "schemaLocation": "https://json-schema.org/schemas/example#/properties/foo/allOf/1", + "instanceLocation": "/foo", + "droppedAnnotations": { + "properties": [ "foo-prop" ], + "title": "foo-title" + }, + "details": [ + { + "valid": false, + "evaluationPath": "/properties/foo/allOf/1/properties/foo-prop", + "schemaLocation": "https://json-schema.org/schemas/example#/properties/foo/allOf/1/properties/foo-prop", + "instanceLocation": "/foo/foo-prop", + "errors": { + "const": "Expected \"1\"" + }, + "droppedAnnotations": { + "title": "foo-prop-title" + } + }, + { + "valid": true, + "evaluationPath": "/properties/foo/allOf/1/additionalProperties", + "schemaLocation": "https://json-schema.org/schemas/example#/properties/foo/allOf/1/additionalProperties", + "instanceLocation": "/foo/other-prop" + } + ] } ] - } - ] - }, - { - "valid": false, - "evaluationPath": "/properties/bar", - "schemaLocation": "https://json-schema.org/schemas/example#/properties/bar", - "instanceLocation": "/bar", - "details": [ + }, { "valid": false, - "evaluationPath": "/properties/bar/$ref", - "schemaLocation": "https://json-schema.org/schemas/example#/$defs/bar", + "evaluationPath": "/properties/bar", + "schemaLocation": "https://json-schema.org/schemas/example#/properties/bar", "instanceLocation": "/bar", - "droppedAnnotations": { - "properties": [ "bar-prop" ], - "title": "bar-title" - }, "details": [ { "valid": false, - "evaluationPath": "/properties/bar/$ref/properties/bar-prop", - "schemaLocation": "https://json-schema.org/schemas/example#/$defs/bar/properties/bar-prop", - "instanceLocation": "/bar/bar-prop", - "errors": { - "minimum": "2 is less than or equal to 10" - }, + "evaluationPath": "/properties/bar/$ref", + "schemaLocation": "https://json-schema.org/schemas/example#/$defs/bar", + "instanceLocation": "/bar", "droppedAnnotations": { - "title": "bar-prop-title" - } + "properties": [ "bar-prop" ], + "title": "bar-title" + }, + "details": [ + { + "valid": false, + "evaluationPath": "/properties/bar/$ref/properties/bar-prop", + "schemaLocation": "https://json-schema.org/schemas/example#/$defs/bar/properties/bar-prop", + "instanceLocation": "/bar/bar-prop", + "errors": { + "minimum": "2 is less than or equal to 10" + }, + "droppedAnnotations": { + "title": "bar-prop-title" + } + } + ] } ] } @@ -455,94 +476,102 @@ The location properties of the root output unit MAY be omitted. } ] } + ``` ```json "Passing Results { - "valid": true, - "evaluationPath": "", - "schemaLocation": "https://json-schema.org/schemas/example#", - "instanceLocation": "", - "annotations": { - "title": "root", - "properties": [ - "foo", - "bar" - ] - }, + "dialect": "https://json-schema.org/draft/next/schema", + "schema": "https://json-schema.org/schemas/example", + "valid": false, "details": [ { "valid": true, - "evaluationPath": "/properties/foo", - "schemaLocation": "https://json-schema.org/schemas/example#/properties/foo", - "instanceLocation": "/foo", + "evaluationPath": "", + "schemaLocation": "https://json-schema.org/schemas/example#", + "instanceLocation": "", + "annotations": { + "title": "root", + "properties": [ + "foo", + "bar" + ] + }, "details": [ { "valid": true, - "evaluationPath": "/properties/foo/allOf/0", - "schemaLocation": "https://json-schema.org/schemas/example#/properties/foo/allOf/0", - "instanceLocation": "/foo" - }, - { - "valid": true, - "evaluationPath": "/properties/foo/allOf/1", - "schemaLocation": "https://json-schema.org/schemas/example#/properties/foo/allOf/1", + "evaluationPath": "/properties/foo", + "schemaLocation": "https://json-schema.org/schemas/example#/properties/foo", "instanceLocation": "/foo", - "annotations": { - "title": "foo-title", - "properties": [ - "foo-prop" - ], - "additionalProperties": [ - "unspecified-prop" - ] - }, "details": [ { "valid": true, - "evaluationPath": "/properties/foo/allOf/1/properties/foo-prop", - "schemaLocation": "https://json-schema.org/schemas/example#/properties/foo/allOf/1/properties/foo-prop", - "instanceLocation": "/foo/foo-prop", - "annotations": { - "title": "foo-prop-title" - } + "evaluationPath": "/properties/foo/allOf/0", + "schemaLocation": "https://json-schema.org/schemas/example#/properties/foo/allOf/0", + "instanceLocation": "/foo" }, { "valid": true, - "evaluationPath": "/properties/foo/allOf/1/additionalProperties", - "schemaLocation": "https://json-schema.org/schemas/example#/properties/foo/allOf/1/additionalProperties", - "instanceLocation": "/foo/unspecified-prop" + "evaluationPath": "/properties/foo/allOf/1", + "schemaLocation": "https://json-schema.org/schemas/example#/properties/foo/allOf/1", + "instanceLocation": "/foo", + "annotations": { + "title": "foo-title", + "properties": [ + "foo-prop" + ], + "additionalProperties": [ + "unspecified-prop" + ] + }, + "details": [ + { + "valid": true, + "evaluationPath": "/properties/foo/allOf/1/properties/foo-prop", + "schemaLocation": "https://json-schema.org/schemas/example#/properties/foo/allOf/1/properties/foo-prop", + "instanceLocation": "/foo/foo-prop", + "annotations": { + "title": "foo-prop-title" + } + }, + { + "valid": true, + "evaluationPath": "/properties/foo/allOf/1/additionalProperties", + "schemaLocation": "https://json-schema.org/schemas/example#/properties/foo/allOf/1/additionalProperties", + "instanceLocation": "/foo/unspecified-prop" + } + ] } ] - } - ] - }, - { - "valid": true, - "evaluationPath": "/properties/bar", - "schemaLocation": "https://json-schema.org/schemas/example#/properties/bar", - "instanceLocation": "/bar", - "details": [ + }, { "valid": true, - "evaluationPath": "/properties/bar/$ref", - "schemaLocation": "https://json-schema.org/schemas/example#/$defs/bar", + "evaluationPath": "/properties/bar", + "schemaLocation": "https://json-schema.org/schemas/example#/properties/bar", "instanceLocation": "/bar", - "annotations": { - "title": "bar-title", - "properties": [ - "bar-prop" - ] - }, "details": [ { "valid": true, - "evaluationPath": "/properties/bar/$ref/properties/bar-prop", - "schemaLocation": "https://json-schema.org/schemas/example#/$defs/bar/properties/bar-prop", - "instanceLocation": "/bar/bar-prop", + "evaluationPath": "/properties/bar/$ref", + "schemaLocation": "https://json-schema.org/schemas/example#/$defs/bar", + "instanceLocation": "/bar", "annotations": { - "title": "bar-prop-title" - } + "title": "bar-title", + "properties": [ + "bar-prop" + ] + }, + "details": [ + { + "valid": true, + "evaluationPath": "/properties/bar/$ref/properties/bar-prop", + "schemaLocation": "https://json-schema.org/schemas/example#/$defs/bar/properties/bar-prop", + "instanceLocation": "/bar/bar-prop", + "annotations": { + "title": "bar-prop-title" + } + } + ] } ] } @@ -556,6 +585,7 @@ The location properties of the root output unit MAY be omitted. For convenience, JSON Schema has been provided to validate output generated by implementations. Its IRI is: +and can also be found at this address. ## Filtering diff --git a/specs/output/schema.json b/specs/output/schema.json index b4cc9cec..b70fa270 100644 --- a/specs/output/schema.json +++ b/specs/output/schema.json @@ -3,12 +3,26 @@ "$id": "https://json-schema.org/draft/next/output/schema", "description": "A schema that validates the minimum requirements for validation output", - "anyOf": [ - { "$ref": "#/$defs/flag" }, - { "$ref": "#/$defs/list" }, - { "$ref": "#/$defs/hierarchical" } - ], + "type": "object", + "properties": { + "dialect": { + "type": "string", + "format": "iri" + }, + "schema": { + "type": "string", + "format": "iri" + }, + "valid": { "type": "boolean" }, + "details": { "$ref": "#/$defs/outputUnitArray" } + }, + "required": ["dialect", "schema", "valid"], + "$defs": { + "outputUnitArray": { + "type": "array", + "items": { "$ref": "#/$defs/outputUnit" } + }, "outputUnit":{ "properties": { "valid": { "type": "boolean" }, @@ -70,26 +84,6 @@ } } ] - }, - "outputUnitArray": { - "type": "array", - "items": { "$ref": "#/$defs/outputUnit" } - }, - "flag": { - "properties": { - "valid": { "type": "boolean" } - }, - "required": [ "valid" ] - }, - "list": { - "properties": { - "valid": { "type": "boolean" }, - "details": { - "$ref": "#/$defs/outputUnitArray" - } - }, - "required": [ "valid", "details" ] - }, - "hierarchical": { "$ref": "#/$defs/outputUnit" } + } } } From 6a95310e4f2bd6cb201dd0eb73389a88642bfb65 Mon Sep 17 00:00:00 2001 From: Greg Dennis Date: Tue, 29 Oct 2024 21:23:06 +1300 Subject: [PATCH 2/2] add explicit output-unit anchor --- specs/output/jsonschema-validation-output-machines.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/output/jsonschema-validation-output-machines.md b/specs/output/jsonschema-validation-output-machines.md index 23751f10..e080f64f 100644 --- a/specs/output/jsonschema-validation-output-machines.md +++ b/specs/output/jsonschema-validation-output-machines.md @@ -42,7 +42,7 @@ For the "list" and "hierarchical" formats, the following is also required: | `details` | A collection of output units to provide additional details. Must be an array, even when containing a single output unit. | -### Output Unit +### Output Unit {#output-unit} An output unit describes the evaluation of a single subschema against the appropriate instance location.