Added tutorial for consider and when. Better glossary, general clean-ups

IQSS · Oct 22, 2017 · 87d0c83 · 87d0c83
1 parent af1f810
commit 87d0c83
Show file tree

Hide file tree

Showing 35 changed files with 152 additions and 59 deletions.
diff --git a/DataTagsLib/DOCS/sphinx-source/source/code/policy-model-full.xml b/DataTagsLib/DOCS/sphinx-source/source/code/policy-model-full.xml
@@ -1,19 +1,19 @@
 <policy-model>
   <title>Model with Lots of Metadata</title>
   <subtitle>An example of a policy-model with full metadata.</subtitle>
-  <version doi="xyz.abc.eg5">2.0.4</version>
+  <version>2.0.4</version>
   <date>2017-05-29</date>
   <keywords>Policy model, metadata, full</keywords>
   <authors>
-    <person orcid="123-2345-67890">
+    <person orcid="123-2345-67890" url="http://www.example.com/jd">
       <name>Jane Doe</name>
       <affiliation>Institute A</affiliation>
       <email>[email protected]</email>
     </person>
     <person orcid="321-2345-67890">
       <name>Joe Smith</name>
     </person>
-    <group>
+    <group url="http://www.example.com/mwg">
       <name>Metadata Working Group</name>
       <contact>[email protected]</contact>
     </group>

diff --git a/DataTagsLib/DOCS/sphinx-source/source/decision-graphs/ask-node.rst b/DataTagsLib/DOCS/sphinx-source/source/decision-graphs/ask-node.rst
@@ -1,5 +1,7 @@
 :doc:`Home <../index>`
 
+.. index:: Ask Node
+
 Ask Node
 ========
 

diff --git a/DataTagsLib/DOCS/sphinx-source/source/decision-graphs/call-node.rst b/DataTagsLib/DOCS/sphinx-source/source/decision-graphs/call-node.rst
@@ -1,5 +1,7 @@
 :doc:`Home <../index>`
 
+.. index:: Call Node
+
 Call Node
 =========
 

diff --git a/DataTagsLib/DOCS/sphinx-source/source/decision-graphs/consider-when-node.rst b/DataTagsLib/DOCS/sphinx-source/source/decision-graphs/consider-when-node.rst
@@ -3,10 +3,12 @@
 Consider and When Nodes
 =======================
 
-``[consider]`` and ``[when]`` nodes are similar to :doc:`ask-node`, but instead of asking the user what to do, they look at the current :any:`Tags Value`. Use these nodes to avoid asking questions whose answers can be inferred from information the questionnaire already has. While both nodes look at the :any:`Tags Value`, ``when`` nodes are able to look at any combination of values, while ``consider`` nodes provide a more syntactically pleasent way of examining a single slot.
+``[consider]`` and ``[when]`` nodes are similar to :doc:`ask-node`, but instead of asking the user what to do, they look at the current :any:`Current Value`. Use these nodes to avoid asking questions whose answers can be inferred from information the questionnaire already has. While both nodes look at the :any:`Current Value`, ``when`` nodes are able to look at any combination of values, while ``consider`` nodes provide a more syntactically pleasent way of examining a single slot.
 
 Both nodes provide an ``else`` sub-node, which is selected when none of the other options matches. These nodes are similar to the ``if`` and ``switch`` keywords in other languages.
 
+.. index:: Consider Node
+
 Consider node
 -------------
 
@@ -27,7 +29,7 @@ Consider node
 
 In the above example, we use a ``consider`` node to infer the appropriate value of the ``Harm`` slot from the value at the ``SubjectType`` slot. If, for example, ``>cn<`` is arrived at when ``SubjectType`` contains ``animal``, the questionnaire declares the harm the dataset in question may cause is ``moderate``. If, on the other hand, ``>cn<`` is arrived at when the subject type is ``human``, the questionnaire goes into a special section for assessing the harm.
 
-
+.. index:: When Node
 
 When node
 ---------

diff --git a/DataTagsLib/DOCS/sphinx-source/source/decision-graphs/end-node.rst b/DataTagsLib/DOCS/sphinx-source/source/decision-graphs/end-node.rst
@@ -1,5 +1,7 @@
 :doc:`Home <../index>`
 
+.. index:: End Node
+
 End Node
 ========
 

diff --git a/DataTagsLib/DOCS/sphinx-source/source/decision-graphs/import-node.rst b/DataTagsLib/DOCS/sphinx-source/source/decision-graphs/import-node.rst
@@ -1,5 +1,7 @@
 :doc:`Home <../index>`
 
+.. index:: #import Node
+
 #Import Node
 =============
 

diff --git a/DataTagsLib/DOCS/sphinx-source/source/decision-graphs/index.rst b/DataTagsLib/DOCS/sphinx-source/source/decision-graphs/index.rst
@@ -37,10 +37,9 @@ Node types can be divided to three groups:
   Branching Node
     A node that can have more than a single "next" node.
     Example: :any:`ask-node`.
+
 
-.. index:: Tags Value
-
-Throughout the execution, the engine keeps a single value of the top-level type (currently, this type is called ``DataTags``), referred to as *Tags Value*. This value can be altered by :doc:`set nodes<set-node>`. At the end of the interview, this value holds the final tagging result. Additionally, the engine maintains a call stack of nodes. Pushing onto the stack is done by :doc:`call nodes<call-node>`. Popping is done by :doc:`end nodes<end-node>`.
+Throughout the execution, the engine keeps a single value of the top-level type, referred to as *Current Value*. This value can be altered by :doc:`set nodes<set-node>`. At the end of the interview, this value holds the final tagging result. Additionally, the engine maintains a call stack of nodes. Pushing onto the stack is done by :doc:`call nodes<call-node>`. Popping is done by :doc:`end nodes<end-node>`.
 
 
 .. toctree::

diff --git a/DataTagsLib/DOCS/sphinx-source/source/decision-graphs/reject-node.rst b/DataTagsLib/DOCS/sphinx-source/source/decision-graphs/reject-node.rst
@@ -1,5 +1,7 @@
 :doc:`Home <../index>`
 
+.. index:: Reject Node
+
 Reject Node
 ===========
 

diff --git a/DataTagsLib/DOCS/sphinx-source/source/decision-graphs/section-node.rst b/DataTagsLib/DOCS/sphinx-source/source/decision-graphs/section-node.rst
@@ -1,5 +1,7 @@
 :doc:`Home <../index>`
 
+.. index :: Section Node
+
 Section Node
 ============
 

diff --git a/DataTagsLib/DOCS/sphinx-source/source/decision-graphs/set-node.rst b/DataTagsLib/DOCS/sphinx-source/source/decision-graphs/set-node.rst
@@ -1,13 +1,16 @@
 :doc:`Home <../index>`
 
+
+.. index :: Set Node
+
 Set Node
 ========
 
-A node that sets sub values in the :any:`Tags Value`. Multiple sub values can be set by the same node.
+A node that sets sub values in the :any:`Current Value`. Multiple sub values can be set by the same node.
 
-Set nodes are through nodes - after updating the tags, the engine goes on to the next node.
+Set nodes are through nodes - after updating the current value, the engine goes on to the next node.
 
-The below examples use the following tagspace::
+The below examples use the following policy space::
 
   DataTags: consists of Mid1, Mid2.
   Mid1: consists of Bottom1, Bottom2.

diff --git a/DataTagsLib/DOCS/sphinx-source/source/decision-graphs/todo-node.rst b/DataTagsLib/DOCS/sphinx-source/source/decision-graphs/todo-node.rst
@@ -1,5 +1,9 @@
 :doc:`Home <../index>`
 
+
+.. index:: To-do Node
+
+
 To-Do Node
 ==========
 

diff --git a/DataTagsLib/DOCS/sphinx-source/source/folder-layout.rst b/DataTagsLib/DOCS/sphinx-source/source/folder-layout.rst
@@ -1,5 +1,9 @@
 :doc:`Home <index>`
 
+.. index :: Folder Layout
+.. index :: Folder Structure
+.. index :: Directory Structure
+
 Folder Layout
 ====================
 

diff --git a/DataTagsLib/DOCS/sphinx-source/source/language-general.rst b/DataTagsLib/DOCS/sphinx-source/source/language-general.rst
@@ -33,10 +33,13 @@ A block comment is contained between ``<*`` and ``*>``, like so::
   ]
 
 
+
+.. index:: Files
+
 Files
 -----
 
-Files are assumed to be encoded in Unicode. By convention, files describing tags spaces have a ``.ts`` suffix, while files describing decision graphs have a ``.dg`` one.
+Files are assumed to be encoded in Unicode. By convention, files describing policy spaces have a ``.pspace`` suffix, while files describing decision graphs have a ``.dg`` one.
 
 .. index:: Top-Down Support
 
@@ -64,13 +67,14 @@ In the below decision graph snippet, the ``EU-Compliance`` section is referenced
   [>EU-Compliance< todo: This section will ensure compliance with the EU regulations.]
 
 
-.. index: Text Editor Support
+.. index:: Text Editor Support
+.. index:: Atom Support
 
 
 Text-Editor Support
 -------------------
 
-It's always easier to code when the text editor helps. Tags has a `language support package`_
+It's always easier to code when the text editor helps. PolicyModels has a `language support package`_
 for the `Atom`_ text editor. It offers syntax colorizing and code templates. It could be installed from
 Atom itself or by using ``apm``::
 

diff --git a/DataTagsLib/DOCS/sphinx-source/source/localizations.rst b/DataTagsLib/DOCS/sphinx-source/source/localizations.rst
@@ -1,5 +1,9 @@
 :doc:`Home <index>`
 
+
+.. index:: Localization
+.. index:: External Texts
+
 Text Externalization and Localizations
 ========================================
 

diff --git a/DataTagsLib/DOCS/sphinx-source/source/policy-model.rst b/DataTagsLib/DOCS/sphinx-source/source/policy-model.rst
@@ -1,5 +1,9 @@
 :doc:`Home <index>`
 
+
+.. index:: policy-model.xml
+.. index:: Policy Model Metadata
+
 Policy Model Description File
 ==============================
 
@@ -32,7 +36,7 @@ subtitle
   The subtitle of the model.
 
 version
-  The current model version. Can have ``doi`` attribute, in case this version has a DOI.
+  The current model version.
 
 date
   Release date of this model version.

diff --git a/DataTagsLib/DOCS/sphinx-source/source/take-for-spin.rst b/DataTagsLib/DOCS/sphinx-source/source/take-for-spin.rst
@@ -3,7 +3,7 @@
 Take PolicyModels for a Spin
 =============================
 
-To start developing questionnaires using Tags and it's tools, you'll need Tags' console application, **CliRunner**. It is available from `binaries page`_ at the DataTaggingLibrary GitHub repository. There are two .zip files
+To start modeling policies using PolicyModels and it's tools, you'll need PolicyModels' console application, **CliRunner**. It is available from `binaries page`_ at the DataTaggingLibrary GitHub repository. There are two .zip files
 of interest in the release:
 
 .. _binaries page: https://github.com/IQSS/DataTaggingLibrary/releases

diff --git a/DataTagsLib/DOCS/sphinx-source/source/terms.rst b/DataTagsLib/DOCS/sphinx-source/source/terms.rst
@@ -7,21 +7,27 @@ Terms and Their Meaning
 
 
   DataTags
-    A set of dataset handling policies made to math the harm a given datset may cause.
+    A set of dataset handling policies made to match the harm a given datset may cause.
     See *Sweeney L, Crosas M, Bar-Sinai M. Sharing Sensitive Data with Confidence: The Datatags System. Technology Science. 2015101601. October 16, 2015.* http://techscience.org/a/2015101601
 
   Decision Graph
     A graph describing a decision process. Composed of various types of nodes.
     Defined over a tag space.
 
   Interview
-    The act of a user answering a questionnaire. The questionnaire equivalent of a program running.
+    The act of a user answering a questionnaire. The PolicyModels equivalent of a program running.
 
-  Questionnaire
-    A pair of matching decision graph and tag space. They have to match, since the decision graph's :doc:`set nodes</decision-graphs/set-node>` mention slots and values, which have to exist in the tag space.
+  PolicyModel
+    A pair of matching decision graph, a policy space, metadata, and possibly value inferrers.
 
-  Tag space
-    A multi-dimensional space defined by tags, which are discrete dimensions.
+  Policy Space
+    A multi-dimensional space defined by slots, which are discrete, ordinal dimensions.
 
-  Tags Value
+  Current Value
     The tags being set by the interview engine. An instance of the top-level compound type. Like all such instances, defines a point in the tag space of the questionnaire.
+
+  CliRunner
+    The model developer's main tool. This is a console application, that can be used to work with policy models. It contains many commands, e.g to create, analyze, query, visualize, inspect, and execute a policy model.
+
+  PolicyModelsServer
+    A web application that can host models online. PolicyModelsServer is an open-source project managed separatly from the main PolicyModels library and commandline environment. Its sources are available on `here <https://github.com/IQSS/DataTaggingServer>`_.
diff --git a/DataTagsLib/DOCS/sphinx-source/source/tutorial/call-and-end.rst b/DataTagsLib/DOCS/sphinx-source/source/tutorial/call-and-end.rst
@@ -1,10 +1,10 @@
-:doc:`index`
+:doc:`Home <../index>` / :doc:`index`
 
 =========
 Call/End
 =========
 
-As questionnaires grow, it is sometimes convenient to split the decision graph into sub graphs, so they are easier to manage. In order to do this, Tags offers the ``call`` node. But first, we need to look at node IDs.
+As questionnaires grow, it is sometimes convenient to split the decision graph into sub graphs, so they are easier to manage. In order to do this, PolicyModels offers the ``call`` node. But first, we need to look at node IDs.
 
 ---------
 Node IDs

diff --git a/...TagsLib/DOCS/sphinx-source/source/tutorial/code/consider-when/basic-tags-dg.png b/...TagsLib/DOCS/sphinx-source/source/tutorial/code/consider-when/basic-tags-dg.png
diff --git a/...tutorial/code/consider-when/basic-tags.dg → ...orial/code/consider-when/consider-when.dg b/...tutorial/code/consider-when/basic-tags.dg → ...orial/code/consider-when/consider-when.dg
@@ -24,7 +24,7 @@
 ]
 [end]
 
-[>open-data< set: Transfer=clear; Storage=clear]
+[>open-data< set: Transit=clear; Storage=clear]
 [end]
 
 <** We have human data, asses privacy risks *>
@@ -45,9 +45,9 @@
 [>cbg< consider:
   {slot: Granularity}
   {options:
-    {accurate:  [set: Transfer=encrypt; Storage=multipartyEncrypt]}
-    {aggregate: [set: Transfer=encrypt; Storage=clear]}
-    {noisy:     [set: Transfer=clear; Storage=clear]}
+    {accurate:  [set: Transit=encrypt; Storage=multipartyEncrypt]}
+    {aggregate: [set: Transit=encrypt; Storage=clear]}
+    {noisy:     [set: Transit=clear; Storage=clear]}
   }
   {else:
     [call: open-data]

diff --git a/...tutorial/code/consider-when/basic-tags.ts → ...orial/code/consider-when/consider-when.ts b/...tutorial/code/consider-when/basic-tags.ts → ...orial/code/consider-when/consider-when.ts
@@ -10,8 +10,8 @@ Domains: some of medical, education, geographic.
 
 Granularity: one of aggregate, noisy, accurate.
 
-Handling: consists of Transfer, Storage.
+Handling: consists of Transit, Storage.
 
-Transfer: one of clear, encrypt, none.
+Transit: one of clear, encrypt, none.
 
 Storage: one of clear, encrypt, multipartyEncrypt.
diff --git a/DataTagsLib/DOCS/sphinx-source/source/tutorial/code/consider-when/policy-model.xml b/DataTagsLib/DOCS/sphinx-source/source/tutorial/code/consider-when/policy-model.xml
@@ -0,0 +1,7 @@
+<policy-model>
+  <title>Consider and When</title>
+  <model>
+    <space root="DataTags">consider-when.ts</space>
+    <graph>consider-when.dg</graph>
+  </model>
+</policy-model>
diff --git a/DataTagsLib/DOCS/sphinx-source/source/tutorial/code/reject/education.dg b/DataTagsLib/DOCS/sphinx-source/source/tutorial/code/reject/education.dg
@@ -5,7 +5,8 @@ Sample toy interview
 [ask:
   {text: Do the data contains personally identifiable information?}
   {terms:
-    {Personally identifiable information: Any information about an individual... }
+    {Personally identifiable information: Any information
+       about an individual... }
   }
   {answers:
     {yes: [set: Storage=encrypt; Transfer=encrypt]}
@@ -17,5 +18,6 @@ Sample toy interview
   {text: Did you get parental consent?}
   {answers:
     {no:
-       [reject: Must get parental consent before collecting data from subjects under 18.]}}]
+       [reject: Must get parental consent before collecting
+          data from subjects under 18.]}}]
 [end]
diff --git a/DataTagsLib/DOCS/sphinx-source/source/tutorial/consider.rst b/DataTagsLib/DOCS/sphinx-source/source/tutorial/consider.rst
@@ -0,0 +1,37 @@
+:doc:`Home <../index>` / :doc:`index`
+
+===================================================
+``consider`` and ``when`` and Conditional Branching
+===================================================
+
+:doc:`[ask] nodes<../decision-graphs/ask-node>` specify a question to the user, and then further specify a list of possible answers, complete with a sub-graph to run for each answer. PolicyModels offers another way of branching between sub-graphs: based on the current values in the slots. This is done using the ``[consider]`` and ``[when]`` nodes. It might be useful to think about them as ``[ask]`` nodes that ask the runtime engine instead of the user. Or, for people familiar with programming, as the PolicyModels equivalent of an ``if`` statement.
+
+``Consider``\ing a Single Slot
+------------------------------
+
+Consider a questionnaire pretaining to medical datasets. The granularity of the dataset in question matters a lot when one needs to decide how protected it should be. After spending some time with the user, asserting the granularity of the data, the :download:`below questionnaire <code/consider-when/consider-when.dg>` uses a :doc:`consider<../decision-graphs/consider-when-node>` node to decide whether the data should be encrypted at rest and/or while in transit.
+
+
+.. literalinclude :: code/consider-when/consider-when.dg
+    :linenos:
+    :lines: 45-55
+
+A ``[consider]`` node has to specify the slot it is considering - this is done by the ``{slot:}`` sub-node (line 2 in the code above). It then lists the sub-graph to traverse for each option, under the ``{options:}`` sub-slot (lines 3-7). An optional ``{else:}`` sub-node (line 8) specifies what to do if none of values listed in ``{options:}`` is current when the considered slot is traversed.
+
+
+``when`` Considering More Than a Single Slot
+--------------------------------------------
+
+The ``[consider]`` node considers only a single slot. To consider the values of a few slots, we use the ``[when]`` node. The sample code below invokes the privacy section of the questionnaire, if the data subjects are living persons, or if they are deceased persons and the data contains medical information.
+
+.. literalinclude :: code/consider-when/consider-when.dg
+    :linenos:
+    :lines: 18-24
+
+
+The ``[when]`` node has a sub-node for each value combination that requires special action (lines 2 and 3 above). Again, an optional ``{else:}`` sub-nodes allows the model developer to specify what needs to be done in case none of the explicitly specified options match the current slot values.
+
+
+.. note:: :doc:`later<inference>` in this tutorial, we'll examine another way of achieving similar functionality, but in a very different way.
+
+In the :doc:`next tutorial<todo-node>`, we'll see how to create placeholder nodes.
diff --git a/DataTagsLib/DOCS/sphinx-source/source/tutorial/hello-question-world.rst b/DataTagsLib/DOCS/sphinx-source/source/tutorial/hello-question-world.rst
@@ -1,10 +1,10 @@
-:doc:`index`
+:doc:`Home <../index>` / :doc:`index`
 
 ===============
 Hello? World?
 ===============
 
-So far, we've put values into slots without asking the user for any information. That is, of course, not the common case. When we need to ask a user, Tags offers the ``ask`` node:
+So far, we've put values into slots without asking the user for any information. That is, of course, not the common case. When we need to ask a user, PolicyModels offers the ``ask`` node:
 
 .. include:: code/hello-question-world/hello-question-world.dg
    :code:
@@ -53,7 +53,7 @@ Run the questionnaire (here are the files for the :download:`tag space <code/hel
 Terms
 -------------
 
-Real-world questions are likely to contain unfamiliar terms, especially when the questions are about legal matters, and the intended audience have had little or no legal training. To clarify terms that appear in a question's ``{text:}``, Tags offers the ``{terms}`` sub-node.
+Real-world questions are likely to contain unfamiliar terms, especially when the questions are about legal matters, and the intended audience have had little or no legal training. To clarify terms that appear in a question's ``{text:}``, PolicyModels offers the ``{terms}`` sub-node.
 
 .. include:: code/hello-question-world/ask-with-terms.dg
    :code:
@@ -74,7 +74,7 @@ The terms are displayed to the user along with the question's text.
 Yes/No Special Case
 --------------------
 
-While working on Tags, we found a repeating interview pattern: The user is asked a yes/no question. On one option (typically a "yes"), the interview goes through a serious of questions before proceeding to the node pointed by the other option (typically, a "no"). To naturally support this, *if an ask node contains only a single "yes" or "no" answer, the other answer is implied, and leads to the next node following that ask node*.
+While working on PolicyModels, we found a repeating interview pattern: The user is asked a yes/no question. On one option (typically a "yes"), the interview goes through a serious of questions before proceeding to the node pointed by the other option (typically, a "no"). To naturally support this, *if an ask node contains only a single "yes" or "no" answer, the other answer is implied, and leads to the next node following that ask node*.
 
 In the :download:`below decision graph <code/hello-question-world/hello-implied-world.dg>`, the user may decide to greet other subjects.