Merge pull request #70 from linkml/add-oak

major refactor

Author: cmungall

.gitignore

@@ -1,5 +1,10 @@
+.idea
+.ipynb_checkpoints/
+docs/_build/
 wwwroot/*.js
 node_modules
 typings
 dist
 schema_automator.egg-info/*
+tmp/
+__pycache__

docs/cli.rst

@@ -0,0 +1,27 @@
+.. cli:
+
+Command Line
+============
+
+All Schema Automator functionality is available via the ``schemauto`` command
+
+Preamble
+--------
+
+.. warning ::
+
+   Previous versions had specific commands like ``tsv2linkml`` these are now deprecated.
+   Instead these are now *subcommands* of the main ``schemauto`` command, and have been renamed.
+
+.. note ::
+
+   we follow the `CLIG <https://clig.dev/>`_ guidelines as far as possible
+
+Main commands
+---------
+
+.. currentmodule:: schema_automator.cli
+
+.. click:: schema_automator.cli:main
+    :prog: schemauto
+    :show-nested:

docs/index.rst

@@ -1,17 +1,16 @@
-.. Schema Automator documentation master file, created by
-   sphinx-quickstart on Fri Apr 22 13:19:04 2022.
-   You can adapt this file completely to your liking, but it should at least
-   contain the root `toctree` directive.
-
-Welcome to Schema Automator's documentation!
+LinkML Schema Automator
 ============================================
 
+Schema Automator is a toolkit for bootstrapping and automatically enhancing LinkML schemas from a variety of sources
+
 .. toctree::
-   :maxdepth: 2
+   :maxdepth: 3
    :caption: Contents:
 
    introduction
-   intro/index
+   install
+   cli
+   packages/index
 
 
 Indices and tables

docs/intro/install.rst renamed to docs/install.rst

@@ -5,7 +5,11 @@ Installation
 
 .. code:: bash
 
-   chmod 755 environment.sh
-   . environment.sh
-   pip install -r requirements.txt
-   pip install -e . 
+   pip install schema-automator
+
+To check this works:
+
+.. code:: bash
+
+   schemauto --help
+

docs/intro/cli.rst

@@ -1,29 +1,38 @@
 LinkML Schema Automator
 =======================
 
-This is a toolkit that assists with:
+This is a toolkit that assists with generating and enhancing schemas and data models from a variety
+of sources.
 
-1. Bootstrapping LinkML models from instance data
+The primary end target is a `LinkML <https://linkml.io>`_ schema, but the framework can be used
+to generate JSON-Schema, SHACL, SQL DDL etc via the `LinkML Generator <https://linkml.io/linkml/generators>`_ framework.
 
-   -  TSVs and spreadsheets
-   -  SQLite databases
-   -  RDF instance graphs
+All functionality is available via a :ref:`cli`. In future there will be a web-based interface.
+The functionality is also available by using the relevant Python :ref:`packages`.
 
-2. Bootstrapping a LinkML model from a different schema representation
-   (i.e. opposite of a linkml.generator)
+Generalization from Instance Data
+---------------------------------
 
-   -  OWL (RDFS-like subset)
-   -  TODO: JSON-Schema, XSD, ShEx, SHACL, SQL DDL, FHIR, Python
-      dataclasses/pydantic, etc
+See :ref:`generalizers`
 
-3. Using automated methods to enhance a model
+Generalizers allow you to *bootstrap* a schema by generalizing from existing data files
 
-   -  Using text mining and concept annotator APIs to enrich semantic
-      enums
-   -  TODO: querying sparql endpoints to retrieve additional metadata
+-  TSVs and spreadsheets
+-  SQLite databases
+-  RDF instance graphs
 
-These can be composed together. For example, run ``tsvs2linkml``
-followed by ``annotate-enums``
+Importing from alternative modeling framework
+---------------------------------
+
+See :ref:`importers`
+
+- OWL (but this only works for schema-style OWL)
+- JSON-Schema
+
+In future other frameworks will be supported
+
+Annotating schemas
+---------------------------------
+
+See :ref:`annotators`
 
-The toolkit is still experimental. It is intended as an aid to schema
-creation rather than act as a formal conversion tool

docs/intro/index.rst

@@ -0,0 +1,21 @@
+.. annotators:
+
+Annotators
+=========
+
+Importers take an existing schema and *annotate* it with information
+
+.. warning::
+
+   some of these methods will be refactored to use OAK, in which case the way in which
+   the Bioportal API key is set will change
+
+.. currentmodule:: schema_automator.annotators
+
+.. autoclass:: SchemaAnnotator
+    :members: create
+
+.. autoclass:: JsonLdAnnotator
+    :members: create
+
+

docs/introduction.rst

@@ -0,0 +1,22 @@
+.. generalizers:
+
+Generalizers
+=========
+
+Generalizers take example data and *generalizes* to a schema
+
+.. warning::
+
+   Generalization is inherently a heuristic process, this should be viewed as a bootstrapping process
+   that *semi*-automates the creation of a new schema for you.
+
+.. currentmodule:: schema_automator.generalizers
+
+.. autoclass:: CsvDataGeneralizer
+    :members: create
+
+.. autoclass:: JsonDataGeneralizer
+    :members: create
+
+.. autoclass:: RdfDataGeneralizer
+    :members: create

docs/packages/annotators.rst

@@ -0,0 +1,27 @@
+.. importers:
+
+Importers
+=========
+
+Importers take either a representation of a schema in a different language or example data and bootstraps a schema.
+
+Importers are the opposite of `Generalizers <https://linkml.io/linkml/generators/index.html>`_ in the core LinkML framework
+
+.. warning::
+
+   Generally importers take a *less expressive* language than LinkML and attempts to create the corresponding
+   LinkML schema. This may be less optimal than a hand-crafted schema. For example, when converting to a
+   representation that lacks `inheritance <https://linkml.io/linkml/schemas/inheritance.html>`_, no ``is_a`` slots
+   will be created.
+
+.. currentmodule:: schema_automator.importers
+
+.. autoclass:: JsonSchemaImportEngine
+    :members: create
+
+.. autoclass:: OwlImportEngine
+    :members: create
+
+.. autoclass:: DOSDPImportEngine
+    :members: create
+