From 1d3e9a878f31467d7bb857d0ff3d864d7aab5bdc Mon Sep 17 00:00:00 2001 From: Alex Nelson Date: Wed, 4 May 2022 12:24:30 -0400 Subject: [PATCH 1/7] Add OWL-SKOS mix-in class This patch adds initial support for SKOS taxonomies as a hierarchy-enabled alternative to UCO's current string-vocabularies practice. While the typical SKOS namespace is prefixed, the import of SKOS is done by referencing an OWL-DL compatible subset of SKOS as a version IRI import. (See SKOS Reference C.3.) Paul Brandt identified the SKOS strategy used in this patch, though his initial draft was in another repository. I ported the property to UCO's ontology repository due to needing to satisfy references for SHACL. This patch is a reduction from the first definition of `core:TaxonomicConcept` drafted for UCO CP-99. References: * [OC-140] (CP-99) UCO should provide a SKOS taxonomy of device types for observable:deviceType * https://github.com/ucoProject/UCO/issues/363 * https://www.w3.org/TR/skos-reference/#namespace-documents Section C.3, "SKOS RDF Schema - OWL 1 DL Sub-set (informative)" Co-authored-by: Paul Brandt Signed-off-by: Alex Nelson --- ontology/core/core.ttl | 11 +++++++++++ ontology/master/uco.ttl | 2 ++ 2 files changed, 13 insertions(+) diff --git a/ontology/core/core.ttl b/ontology/core/core.ttl index de760d3b..4ae01501 100644 --- a/ontology/core/core.ttl +++ b/ontology/core/core.ttl @@ -1,8 +1,11 @@ +# imports: http://www.w3.org/TR/skos-reference/skos-owl1-dl.rdf + @prefix core: . @prefix owl: . @prefix rdf: . @prefix rdfs: . @prefix sh: . +@prefix skos: . @prefix types: . @prefix xsd: . @@ -10,6 +13,7 @@ a owl:Ontology ; rdfs:label "uco-core"@en ; rdfs:comment "This ontology defines classes and properties that are shared across the various UCO ontologies. At a high-level, the UCO core ontology provides base classes, relationship-oriented classes, content-aggregation classes, and shared classes."@en ; + owl:imports ; . core:Annotation @@ -317,6 +321,13 @@ core:Relationship sh:targetClass core:Relationship ; . +core:TaxonomicConcept + a owl:Class ; + rdfs:subClassOf skos:Concept ; + rdfs:label "TaxonomicConcept"@en ; + rdfs:comment "Taxonomic Concept is a mix-in class that enables UCO as an OWL ontology to use SKOS taxonomic individuals. [based on https://www.w3.org/2006/07/SWD/SKOS/skos-and-owl/master.html#Hybrids]"@en ; + . + core:UcoObject a owl:Class , diff --git a/ontology/master/uco.ttl b/ontology/master/uco.ttl index 0385239e..8f5977c8 100644 --- a/ontology/master/uco.ttl +++ b/ontology/master/uco.ttl @@ -1,3 +1,4 @@ +# imports: http://www.w3.org/TR/skos-reference/skos-owl1-dl.rdf # imports: https://ontology.unifiedcyberontology.org/uco/action # imports: https://ontology.unifiedcyberontology.org/uco/core # imports: https://ontology.unifiedcyberontology.org/uco/identity @@ -24,6 +25,7 @@ a owl:Ontology ; rdfs:label "uco-master"@en ; owl:imports + , , , , From 9cb7dc8b96ab27213e4d4371223d70012fd0b0f7 Mon Sep 17 00:00:00 2001 From: Alex Nelson Date: Wed, 4 May 2022 14:11:13 -0400 Subject: [PATCH 2/7] Extend Dublin Core Terms FileFormat class to be OWL Class Dublin Core's `dcterms:FileFormat` class is an existing class that represents file formats, including but not limited to IANA Media Types. (The reference to IANA Media Types is found on the `dcterms:format` property, where the IANA list is recommended, but not required.) `dcterms:FileFormat` is defined as an `rdfs:Class`. In order for UCO, as an OWL ontology, to use this class as a range of a property (especially `observable:mimeType`), it needs to be designated as an `owl:Class`. Note there are some things this patch does not do: * This patch does not import Dublin Core. It appears Dublin Core's model is incompatible with OWL 2 DL. For instance, the property `dcterms:format` appears to violate the OWL 2 DL separation of datatype properties (range of literals) from object properties (range of objects), on review of its sub-property `dcterms:extent` having a range of literals. * This patch does not extend `FileFormat`'s superclasses to also be OWL Classes. * This patch does not adapt the `dcterms:format` property noted above, because in addition to its OWL 2 DL compatibility issues, its semantic scope extends beyond data formats. Compatibility notes: Extending `dcterms:FileFormat` to be both an OWL Class and RDFS Class is compatible with OWL 2 DL. Reviewing this record: https://www.w3.org/TR/2012/REC-owl2-mapping-to-rdf-20121211/#Parsing_of_the_Ontology_Header_and_Declarations Table 5 Row 2 An ontology that also designates `dcterms:FileFormat` as an `rdfs:Class` would have the pair of `owl:Class` and `rdfs:Class` statements reduced to `owl:Class` only as part of its OWL parse. If an ontology has chosen to import Dublin Core as a whole, it has already implicitly made a commitment to use OWL 2 FULL instead of OWL 2 DL. References: * https://github.com/ucoProject/UCO/issues/363 * https://www.dublincore.org/specifications/dublin-core/dcmi-terms/#FileFormat * https://www.dublincore.org/specifications/dublin-core/dcmi-terms/#format Signed-off-by: Alex Nelson --- ontology/types/types.ttl | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/ontology/types/types.ttl b/ontology/types/types.ttl index ce5877b7..125cd738 100644 --- a/ontology/types/types.ttl +++ b/ontology/types/types.ttl @@ -1,6 +1,7 @@ # imports: https://ontology.unifiedcyberontology.org/uco/core # imports: https://ontology.unifiedcyberontology.org/uco/vocabulary +@prefix dcterms: . @prefix owl: . @prefix rdf: . @prefix rdfs: . @@ -18,6 +19,11 @@ ; . +dcterms:FileFormat + a owl:Class ; + rdfs:comment "UCO extends dcterms:FileFormat to be an OWL Class in addition to its native definition of an RDF Class."@en ; + . + types:ControlledDictionary a owl:Class , From 5c15b29af2ed918a227fde7ef30f7eff423ce030 Mon Sep 17 00:00:00 2001 From: Alex Nelson Date: Wed, 4 May 2022 14:19:52 -0400 Subject: [PATCH 3/7] Set observable:mimeType to be an ObjectProperty For compatibility with other graph data models that use IANA Media Types, the required range is set to `dcterms:FileFormat`. References: * https://github.com/ucoProject/UCO/issues/363 Signed-off-by: Alex Nelson --- ontology/observable/observable.ttl | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/ontology/observable/observable.ttl b/ontology/observable/observable.ttl index 9d141ecd..fbbe9c16 100644 --- a/ontology/observable/observable.ttl +++ b/ontology/observable/observable.ttl @@ -7,6 +7,7 @@ @prefix action: . @prefix core: . +@prefix dcterms: . @prefix identity: . @prefix location: . @prefix observable: . @@ -1728,6 +1729,11 @@ observable:ContentDataFacet rdfs:label "ContentDataFacet"@en ; rdfs:comment "A content data facet is a grouping of characteristics unique to a block of digital data."@en ; sh:property + [ + sh:class dcterms:FileFormat ; + sh:nodeKind sh:IRI ; + sh:path observable:mimeType ; + ] , [ sh:class observable:ObservableObject ; sh:maxCount "1"^^xsd:integer ; @@ -1775,11 +1781,6 @@ observable:ContentDataFacet sh:nodeKind sh:Literal ; sh:path observable:mimeClass ; ] , - [ - sh:datatype xsd:string ; - sh:nodeKind sh:Literal ; - sh:path observable:mimeType ; - ] , [ sh:datatype vocabulary:EndiannessTypeVocab ; sh:message "Value is outside the default vocabulary EndiannessTypeVocab." ; @@ -10944,10 +10945,10 @@ observable:mimeClass . observable:mimeType - a owl:DatatypeProperty ; + a owl:ObjectProperty ; rdfs:label "mimeType"@en ; - rdfs:comment "MIME type of the data. For example 'text/html' or 'audio/mp3'."@en ; - rdfs:range xsd:string ; + rdfs:comment "MIME type of the data. For interoperability with non-UCO resources, dcterms:FileFormat is this property's required range."@en ; + rdfs:range dcterms:FileFormat ; . observable:minorImageVersion From d75d30766825d04d7908f2df00ab2178ce0a5de2 Mon Sep 17 00:00:00 2001 From: Alex Nelson Date: Wed, 4 May 2022 15:55:39 -0400 Subject: [PATCH 4/7] Enable, suggest, and demonstrate mimeType values This patch adds a class hierarchy to distinguish between known IANA Media Types and Media Types known to not be registered with IANA. A unit test is added to demonstrate how mimeType being objects can also enable hierarchical searches, even between IANA and non-IANA types. A follow-on patch will generate validation result files. References: * https://github.com/ucoProject/UCO/issues/363 Signed-off-by: Alex Nelson --- ontology/observable/observable.ttl | 30 +++++- ontology/types/types.ttl | 27 +++++ tests/examples/Makefile | 4 + tests/examples/mime_PASS.json | 152 +++++++++++++++++++++++++++++ tests/examples/mime_XFAIL.json | 28 ++++++ tests/examples/test_mime.py | 98 +++++++++++++++++++ tests/examples/test_validation.py | 22 +++++ 7 files changed, 360 insertions(+), 1 deletion(-) create mode 100644 tests/examples/mime_PASS.json create mode 100644 tests/examples/mime_XFAIL.json create mode 100644 tests/examples/test_mime.py diff --git a/ontology/observable/observable.ttl b/ontology/observable/observable.ttl index fbbe9c16..40750d98 100644 --- a/ontology/observable/observable.ttl +++ b/ontology/observable/observable.ttl @@ -15,6 +15,7 @@ @prefix rdf: . @prefix rdfs: . @prefix sh: . +@prefix skos: . @prefix types: . @prefix vocabulary: . @prefix xsd: . @@ -10947,8 +10948,35 @@ observable:mimeClass observable:mimeType a owl:ObjectProperty ; rdfs:label "mimeType"@en ; - rdfs:comment "MIME type of the data. For interoperability with non-UCO resources, dcterms:FileFormat is this property's required range."@en ; + rdfs:comment "MIME type of the data. The text form of '${type}/${subtype}' (for example, 'text/html' or 'audio/mp3') can be used to find a UCO-provisioned set of types from the UCO Media Types Taxonomy. For interoperability with non-UCO resources, dcterms:FileFormat is this property's required range. The more-specific UCO classes types:IANAMediaType or types:NonIANAMediaType should be used when available."@en ; rdfs:range dcterms:FileFormat ; + rdfs:seeAlso + types:IANAMediaType , + types:NonIANAMediaType , + + ; + . + +observable:mimeType-class-types-MIMEFormat + a sh:PropertyShape ; + rdfs:comment "This shape is given an IRI in order to facilitate deactivation on request."@en ; + rdfs:seeAlso sh:deactivated ; + sh:class types:MIMEFormat ; + sh:message "Value is not an instance of types:MIMEFormat or one of its subclasses. Please consider using a value having type types:IANAMediaType or types:NonIANAMediaType."@en ; + sh:path observable:mimeType ; + sh:severity sh:Info ; + sh:targetSubjectsOf observable:mimeType ; + . + +observable:mimeType-notation + a sh:PropertyShape ; + rdfs:comment "This shape is intended to be deactivated by using overriding shapes in the UCO Mime Taxonomy."@en ; + rdfs:seeAlso ; + sh:message "The used mimeType value is not a concept with a skos:notation. Some consumers of this data expect a string with the MIME 'type/subtype' format to be supplied with skos:notation. Please either add the skos:notation to your input graph or incorporate the UCO MIME Taxonomy."@en ; + sh:minCount "1"^^xsd:integer ; + sh:path skos:notation ; + sh:severity sh:Warning ; + sh:targetObjectsOf observable:mimeType ; . observable:minorImageVersion diff --git a/ontology/types/types.ttl b/ontology/types/types.ttl index 125cd738..35106c81 100644 --- a/ontology/types/types.ttl +++ b/ontology/types/types.ttl @@ -1,7 +1,9 @@ # imports: https://ontology.unifiedcyberontology.org/uco/core # imports: https://ontology.unifiedcyberontology.org/uco/vocabulary +@prefix core: . @prefix dcterms: . +@prefix observable: . @prefix owl: . @prefix rdf: . @prefix rdfs: . @@ -160,16 +162,41 @@ types:Hash sh:targetClass types:Hash ; . +types:IANAMediaType + a owl:Class ; + rdfs:subClassOf types:MIMEFormat ; + rdfs:label "IANAMediaType"@en ; + owl:disjointWith types:NonIANAMediaType ; + . + types:Identifier a rdfs:Datatype ; rdfs:comment "An identifier is a string conformant to the specified UUID-based format for UCO object identifiers."@en ; . +types:MIMEFormat + a owl:Class ; + rdfs:subClassOf + dcterms:FileFormat , + core:TaxonomicConcept + ; + rdfs:label "MIMEFormat"@en ; + rdfs:comment "This class should be considered an intermediary, 'abstract' class, and should not have individuals defined that are not a member of one of its subclasses. This class has two purposes. First, to encode for interoperability with existing frameworks, without relying on RDFS inference, that a UCO MIMEFormat individual is both a SKOS Concept (via core:TaxonomicConcept) and Dublin Core Terms FileFormat. Second, to guarantee a concept to be used as a UCO MIME type has exactly one skos:notation. Note that strict SHACL enforcement of the skos:notation presence is not within the UCO Types ontology, but is instead a part of the shapes provided by the UCO MIME Taxonomy. The warning about skos:notation in the Observable namespace is deactivated on importing the taxonomy's shapes."@en ; + rdfs:seeAlso observable:mimeType-notation ; + . + types:NativeFormatString a rdfs:Datatype ; rdfs:comment "Specifies data in its native format of some external language. The data may be encoded in Base64 per [RFC4648]. Data encoded in Base64 must be denoted as such using the encoded property."@en ; . +types:NonIANAMediaType + a owl:Class ; + rdfs:subClassOf types:MIMEFormat ; + rdfs:label "NonIANAMediaType"@en ; + owl:disjointWith types:IANAMediaType ; + . + types:StructuredText a rdfs:Datatype ; rdfs:comment "Expresses string-based data in some information structuring format (e.g., HTML5)."@en ; diff --git a/tests/examples/Makefile b/tests/examples/Makefile index fd0a86df..d15ab098 100644 --- a/tests/examples/Makefile +++ b/tests/examples/Makefile @@ -25,6 +25,8 @@ all: \ hash_XFAIL_validation.ttl \ location_PASS_validation.ttl \ location_XFAIL_validation.ttl \ + mime_PASS_validation.ttl \ + mime_XFAIL_validation.ttl \ relationship_PASS_validation.ttl \ relationship_XFAIL_validation.ttl @@ -74,6 +76,8 @@ check: \ hash_XFAIL_validation.ttl \ location_PASS_validation.ttl \ location_XFAIL_validation.ttl \ + mime_PASS_validation.ttl \ + mime_XFAIL_validation.ttl \ relationship_PASS_validation.ttl \ relationship_XFAIL_validation.ttl source $(tests_srcdir)/venv/bin/activate \ diff --git a/tests/examples/mime_PASS.json b/tests/examples/mime_PASS.json new file mode 100644 index 00000000..ad0fbcea --- /dev/null +++ b/tests/examples/mime_PASS.json @@ -0,0 +1,152 @@ +{ + "@context": { + "core": "https://ontology.unifiedcyberontology.org/uco/core/", + "dcterms": "http://purl.org/dc/terms/", + "kb": "http://example.org/kb/", + "observable": "https://ontology.unifiedcyberontology.org/uco/observable/", + "rdfs": "http://www.w3.org/2000/01/rdf-schema#", + "skos": "http://www.w3.org/2004/02/skos/core#", + "types": "https://ontology.unifiedcyberontology.org/uco/types/" + }, + "@graph": [ + { + "@id": "urn:example:mime:dcterms:application/gzip", + "@type": "dcterms:FileFormat", + "rdfs:comment": "This is a custom media type individual, designed for just this unit test. A larger taxonomy of individuals should be used outside of testing contexts.", + "skos:notation": "application/gzip" + }, + { + "@id": "urn:example:mime:uco:application/gzip", + "@type": "types:IANAMediaType", + "rdfs:comment": "This is a custom media type individual, designed for just this unit test. A larger taxonomy of individuals should be used outside of testing contexts.", + "skos:exactMatch": { + "@id": "urn:example:mime:dcterms:application/gzip" + }, + "skos:notation": "application/gzip" + }, + { + "@id": "urn:example:mime:uco:application/tar", + "@type": "types:NonIANAMediaType", + "rdfs:comment": "This is a custom media type individual, designed for just this unit test. A larger taxonomy of individuals should be used outside of testing contexts.", + "skos:notation": "application/tar" + }, + { + "@id": "urn:example:mime:uco:application/tar+gzip", + "@type": "types:NonIANAMediaType", + "rdfs:comment": "This is a custom media type individual, designed for just this unit test. A larger taxonomy of individuals should be used outside of testing contexts.", + "skos:broader": [ + { + "@id": "urn:example:mime:uco:application/gzip" + }, + { + "@id": "urn:example:mime:uco:application/tar" + } + ], + "skos:notation": "application/tar+gzip" + }, + { + "@id": "urn:example:mime:uco:image/example.image.type.without.notation", + "@type": "types:NonIANAMediaType", + "rdfs:comment": "This is a custom media type individual, designed for just this unit test. A larger taxonomy of individuals should be used outside of testing contexts." + }, + { + "@id": "kb:file-1", + "@type": "observable:File", + "core:hasFacet": [ + { + "@type": "observable:ContentDataFacet", + "observable:mimeType": { + "@id": "urn:example:mime:dcterms:application/gzip" + } + }, + { + "@type": "observable:FileFacet", + "observable:fileName": "1.gz" + } + ], + "rdfs:comment": "This file node should trigger an sh:Info-severity result." + }, + { + "@id": "kb:file-2", + "@type": "observable:File", + "core:hasFacet": [ + { + "@type": "observable:ContentDataFacet", + "observable:mimeType": { + "@id": "urn:example:mime:uco:application/gzip" + } + }, + { + "@type": "observable:FileFacet", + "observable:fileName": "2.gz" + } + ] + }, + { + "@id": "kb:file-3", + "@type": "observable:File", + "core:hasFacet": [ + { + "@type": "observable:ContentDataFacet", + "observable:mimeType": { + "@id": "urn:example:mime:uco:application/tar" + } + }, + { + "@type": "observable:FileFacet", + "observable:fileName": "3.tar" + } + ] + }, + { + "@id": "kb:file-4", + "@type": "observable:File", + "core:hasFacet": [ + { + "@type": "observable:ContentDataFacet", + "observable:mimeType": { + "@id": "urn:example:mime:uco:application/tar+gzip" + } + }, + { + "@type": "observable:FileFacet", + "observable:fileName": "4.tar.gz" + } + ] + }, + { + "@id": "kb:file-5", + "@type": "observable:File", + "core:hasFacet": [ + { + "@type": "observable:ContentDataFacet", + "observable:mimeType": { + "@id": "urn:example:mime:uco:image/example.image.type.without.notation" + } + }, + { + "@type": "observable:FileFacet", + "observable:fileName": "5.dat" + } + ], + "rdfs:comment": "This file node should trigger an sh:Warning-severity result." + }, + { + "@id": "kb:file-6", + "@type": "observable:File", + "core:hasFacet": [ + { + "@type": "observable:ContentDataFacet", + "observable:mimeType": { + "@id": "urn:example:mime:uco:image/example.image.type.without.notation" + } + }, + { + "@type": "observable:FileFacet", + "observable:fileName": "6.dat" + } + ], + "rdfs:comment": "This file node should trigger an sh:Warning-severity result." + } + ] +} diff --git a/tests/examples/mime_XFAIL.json b/tests/examples/mime_XFAIL.json new file mode 100644 index 00000000..6f86ea64 --- /dev/null +++ b/tests/examples/mime_XFAIL.json @@ -0,0 +1,28 @@ +{ + "@context": { + "core": "https://ontology.unifiedcyberontology.org/uco/core/", + "dcterms": "http://purl.org/dc/terms/", + "kb": "http://example.org/kb/", + "observable": "https://ontology.unifiedcyberontology.org/uco/observable/", + "rdfs": "http://www.w3.org/2000/01/rdf-schema#", + "skos": "http://www.w3.org/2004/02/skos/core#", + "types": "https://ontology.unifiedcyberontology.org/uco/types/" + }, + "@graph": [ + { + "@id": "kb:file-7", + "@type": "observable:File", + "core:hasFacet": [ + { + "@type": "observable:ContentDataFacet", + "observable:mimeType": "application/gzip" + }, + { + "@type": "observable:FileFacet", + "observable:fileName": "7.gz" + } + ], + "rdfs:comment": "This file node errantly uses a literal instead of object reference." + } + ] +} diff --git a/tests/examples/test_mime.py b/tests/examples/test_mime.py new file mode 100644 index 00000000..822f5648 --- /dev/null +++ b/tests/examples/test_mime.py @@ -0,0 +1,98 @@ +#!/usr/bin/env python3 + +# This software was developed at the National Institute of Standards +# and Technology by employees of the Federal Government in the course +# of their official duties. Pursuant to title 17 Section 105 of the +# United States Code this software is not subject to copyright +# protection and is in the public domain. NIST assumes no +# responsibility whatsoever for its use by other parties, and makes +# no guarantees, expressed or implied, about its quality, +# reliability, or any other characteristic. +# +# We would appreciate acknowledgement if the software is used. + +import typing + +import pytest +import rdflib + +NS_UCO_CORE = rdflib.Namespace("https://ontology.unifiedcyberontology.org/uco/core/") +NS_UCO_OBSERVABLE = rdflib.Namespace("https://ontology.unifiedcyberontology.org/uco/observable/") + +NSDICT = { + "core": NS_UCO_CORE, + "observable": NS_UCO_OBSERVABLE, + "skos": rdflib.SKOS +} + +@pytest.fixture +def mime_pass_graph() -> rdflib.Graph: + graph = rdflib.Graph() + graph.parse("mime_PASS.json", format="json-ld") + return graph + + +def test_mime_file_names(mime_pass_graph: rdflib.Graph) -> None: + expected: typing.Set[str] = { + "1.gz", + "2.gz", + "3.tar", + "4.tar.gz", + "5.dat", + "6.dat", + } + computed: typing.Set[str] = set() + + query = rdflib.plugins.sparql.prepareQuery("""\ +SELECT ?lFileName +WHERE { + ?nFile core:hasFacet/observable:fileName ?lFileName . +} +""", initNs=NSDICT) + for result in mime_pass_graph.query(query): + computed.add(str(result[0])) + assert expected == computed + + +def test_mime_gzip_files(mime_pass_graph: rdflib.Graph) -> None: + expected: typing.Set[str] = { + "1.gz", + "2.gz", + "4.tar.gz", + } + computed: typing.Set[str] = set() + + query = rdflib.plugins.sparql.prepareQuery("""\ +SELECT ?lFileName +WHERE { + ?nFile core:hasFacet/observable:fileName ?lFileName . + ?nFile core:hasFacet/observable:mimeType ?nMimeType . + + ?nMimeType skos:exactMatch*/skos:broader*/skos:notation "application/gzip" . +} +""", initNs=NSDICT) + for result in mime_pass_graph.query(query): + computed.add(str(result[0])) + assert expected == computed + + + +def test_mime_tar_files(mime_pass_graph: rdflib.Graph) -> None: + expected: typing.Set[str] = { + "3.tar", + "4.tar.gz", + } + computed: typing.Set[str] = set() + + query = rdflib.plugins.sparql.prepareQuery("""\ +SELECT ?lFileName +WHERE { + ?nFile core:hasFacet/observable:fileName ?lFileName . + ?nFile core:hasFacet/observable:mimeType ?nMimeType . + + ?nMimeType skos:exactMatch*/skos:broader*/skos:notation "application/tar" . +} +""", initNs=NSDICT) + for result in mime_pass_graph.query(query): + computed.add(str(result[0])) + assert expected == computed diff --git a/tests/examples/test_validation.py b/tests/examples/test_validation.py index 51e08203..2c3e5da6 100644 --- a/tests/examples/test_validation.py +++ b/tests/examples/test_validation.py @@ -28,9 +28,11 @@ import rdflib.plugins.sparql NS_SH = rdflib.SH +NS_SKOS = rdflib.SKOS NS_UCO_ACTION = rdflib.Namespace("https://ontology.unifiedcyberontology.org/uco/action/") NS_UCO_CORE = rdflib.Namespace("https://ontology.unifiedcyberontology.org/uco/core/") NS_UCO_LOCATION = rdflib.Namespace("https://ontology.unifiedcyberontology.org/uco/location/") +NS_UCO_OBSERVABLE = rdflib.Namespace("https://ontology.unifiedcyberontology.org/uco/observable/") NSDICT = {"sh": NS_SH} @@ -190,6 +192,26 @@ def test_location_XFAIL_validation_XPASS_wrong_concept_name(): } ) +def test_mime_PASS_validation() -> None: + confirm_validation_results( + "mime_PASS_validation.ttl", + True, + expected_result_paths={ + str(NS_SKOS.notation), + str(NS_UCO_OBSERVABLE.mimeType) + } + ) + +def test_mime_XFAIL_validation() -> None: + confirm_validation_results( + "mime_XFAIL_validation.ttl", + False, + expected_result_paths={ + str(NS_SKOS.notation), + str(NS_UCO_OBSERVABLE.mimeType) + } + ) + def test_relationship_PASS_partial() -> None: """ This test should be replaced with test_relationship_XFAIL_full when the semi-open vocabulary design current as of UCO 0.8.0 is re-done. From 9902977ec84baab590fad278c53742c68285ca24 Mon Sep 17 00:00:00 2001 From: Alex Nelson Date: Wed, 4 May 2022 15:58:44 -0400 Subject: [PATCH 5/7] Generate Make-managed files References: * https://github.com/ucoProject/UCO/issues/363 Signed-off-by: Alex Nelson --- tests/examples/mime_PASS_validation.ttl | 37 ++++++++++++ tests/examples/mime_XFAIL_validation.ttl | 72 ++++++++++++++++++++++++ 2 files changed, 109 insertions(+) create mode 100644 tests/examples/mime_PASS_validation.ttl create mode 100644 tests/examples/mime_XFAIL_validation.ttl diff --git a/tests/examples/mime_PASS_validation.ttl b/tests/examples/mime_PASS_validation.ttl new file mode 100644 index 00000000..98fae107 --- /dev/null +++ b/tests/examples/mime_PASS_validation.ttl @@ -0,0 +1,37 @@ +@prefix observable: . +@prefix owl: . +@prefix rdf: . +@prefix rdfs: . +@prefix sh: . +@prefix skos: . +@prefix xsd: . + +[] + a sh:ValidationReport ; + sh:conforms "true"^^xsd:boolean ; + sh:result + [ + a sh:ValidationResult ; + sh:focusNode ; + sh:resultMessage "The used mimeType value is not a concept with a skos:notation. Some consumers of this data expect a string with the MIME 'type/subtype' format to be supplied with skos:notation. Please either add the skos:notation to your input graph or incorporate the UCO MIME Taxonomy."@en ; + sh:resultPath skos:notation ; + sh:resultSeverity sh:Warning ; + sh:sourceConstraintComponent sh:MinCountConstraintComponent ; + sh:sourceShape observable:mimeType-notation ; + ] , + [ + a sh:ValidationResult ; + sh:focusNode [ + a observable:ContentDataFacet ; + observable:mimeType ; + ] ; + sh:resultMessage "Value is not an instance of types:MIMEFormat or one of its subclasses. Please consider using a value having type types:IANAMediaType or types:NonIANAMediaType."@en ; + sh:resultPath observable:mimeType ; + sh:resultSeverity sh:Info ; + sh:sourceConstraintComponent sh:ClassConstraintComponent ; + sh:sourceShape observable:mimeType-class-types-MIMEFormat ; + sh:value ; + ] + ; + . + diff --git a/tests/examples/mime_XFAIL_validation.ttl b/tests/examples/mime_XFAIL_validation.ttl new file mode 100644 index 00000000..d4d56dcb --- /dev/null +++ b/tests/examples/mime_XFAIL_validation.ttl @@ -0,0 +1,72 @@ +@prefix dcterms: . +@prefix observable: . +@prefix owl: . +@prefix rdf: . +@prefix rdfs: . +@prefix sh: . +@prefix skos: . +@prefix xsd: . + +[] + a sh:ValidationReport ; + sh:conforms "false"^^xsd:boolean ; + sh:result + [ + a sh:ValidationResult ; + sh:focusNode "application/gzip" ; + sh:resultMessage "The used mimeType value is not a concept with a skos:notation. Some consumers of this data expect a string with the MIME 'type/subtype' format to be supplied with skos:notation. Please either add the skos:notation to your input graph or incorporate the UCO MIME Taxonomy."@en ; + sh:resultPath skos:notation ; + sh:resultSeverity sh:Warning ; + sh:sourceConstraintComponent sh:MinCountConstraintComponent ; + sh:sourceShape observable:mimeType-notation ; + ] , + [ + a sh:ValidationResult ; + sh:focusNode [ + a observable:ContentDataFacet ; + observable:mimeType "application/gzip" ; + ] ; + sh:resultMessage "Value does not have class dcterms:FileFormat" ; + sh:resultPath observable:mimeType ; + sh:resultSeverity sh:Violation ; + sh:sourceConstraintComponent sh:ClassConstraintComponent ; + sh:sourceShape [ + sh:class dcterms:FileFormat ; + sh:nodeKind sh:IRI ; + sh:path observable:mimeType ; + ] ; + sh:value "application/gzip" ; + ] , + [ + a sh:ValidationResult ; + sh:focusNode [ + a observable:ContentDataFacet ; + observable:mimeType "application/gzip" ; + ] ; + sh:resultMessage "Value is not an instance of types:MIMEFormat or one of its subclasses. Please consider using a value having type types:IANAMediaType or types:NonIANAMediaType."@en ; + sh:resultPath observable:mimeType ; + sh:resultSeverity sh:Info ; + sh:sourceConstraintComponent sh:ClassConstraintComponent ; + sh:sourceShape observable:mimeType-class-types-MIMEFormat ; + sh:value "application/gzip" ; + ] , + [ + a sh:ValidationResult ; + sh:focusNode [ + a observable:ContentDataFacet ; + observable:mimeType "application/gzip" ; + ] ; + sh:resultMessage "Value is not of Node Kind sh:IRI" ; + sh:resultPath observable:mimeType ; + sh:resultSeverity sh:Violation ; + sh:sourceConstraintComponent sh:NodeKindConstraintComponent ; + sh:sourceShape [ + sh:class dcterms:FileFormat ; + sh:nodeKind sh:IRI ; + sh:path observable:mimeType ; + ] ; + sh:value "application/gzip" ; + ] + ; + . + From 3fdba5eca2b9c1e8fe7076364fbc8336fb4b221c Mon Sep 17 00:00:00 2001 From: Alex Nelson Date: Wed, 4 May 2022 16:34:10 -0400 Subject: [PATCH 6/7] Add example of shorter JSON MIME object usage c/o Context Dictionary feature A follow-on patch will generate a Make-managed file. References: * https://github.com/ucoProject/UCO/issues/363 Signed-off-by: Alex Nelson --- tests/examples/Makefile | 2 ++ tests/examples/mime_prefix_PASS.json | 36 ++++++++++++++++++++++++++++ tests/examples/test_validation.py | 7 ++++++ 3 files changed, 45 insertions(+) create mode 100644 tests/examples/mime_prefix_PASS.json diff --git a/tests/examples/Makefile b/tests/examples/Makefile index d15ab098..1075937c 100644 --- a/tests/examples/Makefile +++ b/tests/examples/Makefile @@ -27,6 +27,7 @@ all: \ location_XFAIL_validation.ttl \ mime_PASS_validation.ttl \ mime_XFAIL_validation.ttl \ + mime_prefix_PASS_validation.ttl \ relationship_PASS_validation.ttl \ relationship_XFAIL_validation.ttl @@ -78,6 +79,7 @@ check: \ location_XFAIL_validation.ttl \ mime_PASS_validation.ttl \ mime_XFAIL_validation.ttl \ + mime_prefix_PASS_validation.ttl \ relationship_PASS_validation.ttl \ relationship_XFAIL_validation.ttl source $(tests_srcdir)/venv/bin/activate \ diff --git a/tests/examples/mime_prefix_PASS.json b/tests/examples/mime_prefix_PASS.json new file mode 100644 index 00000000..047ba475 --- /dev/null +++ b/tests/examples/mime_prefix_PASS.json @@ -0,0 +1,36 @@ +{ + "@context": { + "core": "https://ontology.unifiedcyberontology.org/uco/core/", + "ex-mime": "urn:example:mime:custom:", + "kb": "http://example.org/kb/", + "observable": "https://ontology.unifiedcyberontology.org/uco/observable/", + "observable:mimeType": { + "@type": "@id" + }, + "rdfs": "http://www.w3.org/2000/01/rdf-schema#", + "skos": "http://www.w3.org/2004/02/skos/core#", + "types": "https://ontology.unifiedcyberontology.org/uco/types/" + }, + "@graph": [ + { + "@id": "ex-mime:application/gzip", + "@type": "types:IANAMediaType", + "rdfs:comment": "This is a custom media type individual, designed for just this unit test. A larger taxonomy of individuals should be used outside of testing contexts.", + "skos:notation": "application/gzip" + }, + { + "@id": "kb:file-8", + "@type": "observable:File", + "core:hasFacet": [ + { + "@type": "observable:ContentDataFacet", + "observable:mimeType": "ex-mime:application/gzip" + }, + { + "@type": "observable:FileFacet", + "observable:fileName": "8.gz" + } + ] + } + ] +} diff --git a/tests/examples/test_validation.py b/tests/examples/test_validation.py index 2c3e5da6..eecebc7a 100644 --- a/tests/examples/test_validation.py +++ b/tests/examples/test_validation.py @@ -212,6 +212,13 @@ def test_mime_XFAIL_validation() -> None: } ) +def test_mime_prefix_PASS_validation() -> None: + confirm_validation_results( + "mime_prefix_PASS_validation.ttl", + True, + expected_focus_node_severities=set() + ) + def test_relationship_PASS_partial() -> None: """ This test should be replaced with test_relationship_XFAIL_full when the semi-open vocabulary design current as of UCO 0.8.0 is re-done. From d39c5441a81c162a80cfd591442760019aa291c5 Mon Sep 17 00:00:00 2001 From: Alex Nelson Date: Wed, 4 May 2022 16:34:29 -0400 Subject: [PATCH 7/7] Generate Make-managed file References: * https://github.com/ucoProject/UCO/issues/363 Signed-off-by: Alex Nelson --- tests/examples/mime_prefix_PASS_validation.ttl | 11 +++++++++++ 1 file changed, 11 insertions(+) create mode 100644 tests/examples/mime_prefix_PASS_validation.ttl diff --git a/tests/examples/mime_prefix_PASS_validation.ttl b/tests/examples/mime_prefix_PASS_validation.ttl new file mode 100644 index 00000000..33496ff0 --- /dev/null +++ b/tests/examples/mime_prefix_PASS_validation.ttl @@ -0,0 +1,11 @@ +@prefix owl: . +@prefix rdf: . +@prefix rdfs: . +@prefix sh: . +@prefix xsd: . + +[] + a sh:ValidationReport ; + sh:conforms "true"^^xsd:boolean ; + . +