Initial version

Author: tomqwpl

README.md

@@ -0,0 +1,96 @@
+# Program Package Development
+This program package guides you through how to develop
+programs for WPS Hub Deployment Services within a
+program package.
+
+Program packages are stored in git, and are edited using
+normal editors and a standard git client. The program package
+git repository can be hosted by the Hub itself, or it
+can he hosted on a site such as Github.
+
+When using program packages, the package becomes the unit
+of deployment. So all programs within a package are deployed
+together to an environment.
+
+## Package structure
+A package can be structured in any way you choose. Programs
+can be placed in directories for organisation if necessary.
+However, it is important to note that the complete git repository
+is deployed together. The package becomes the unit of deployment.
+So make sure that the git repository only contains resources
+that are necessary for deployment. It is not possible to deploy
+only part of a repository.
+
+In simple terms, if there is a file called "hello.sas" in the
+package, it can be invoked using a URL of `deployed-package-name/hello.sas`.
+
+
+## Package Descriptor File
+There is an optional file, `package.yaml` that can exist
+in the root directory of a package. This describes, amongst
+other things, the programs within the package that should be
+published to the directory. Note that publication of programs
+in the directory is not required for invocation. Programs can be
+invoked directly if the URL is known ('ex-directory' programs).
+Publication in the directory is necessary in order for the program
+to be invoked using an invocation interface such as the Excel plugin.
+
+### Syntax
+The package.yaml file is a YAML file with the following fields:
+
+| Field | Type | Description |
+| ----- | ---- | ----------- |
+| publishInDirectory | Array of string | A list of the relative paths to the programs to publish in the directory |
+
+## Program Descriptor
+In order to be able to invoke a program it is necessary to have a description of its interface and invocation options. This information is stored in the program descriptor that is associated with a program.
+
+The program descriptor is placed in a separate YAML file alongside the program source file. The program descriptor should have the same name as the program source file, with an additional ".yaml" extension. So for a program source file called "hello.sas", the program descriptor should be in a file called "hello.sas.yaml".
+
+Note that a program descriptor isn't strictly necessary. For a program that has no parameters, and uses all of the default execution options, no descriptor is required. In all but the simplest of cases though the descriptor is required.
+
+### Program Descriptor Syntax
+The descriptor is a YAML formatted file with the following base properties:
+
+| Field | Type | Description |
+| ----- | ---- | ----------- |
+| label | string | The label under which this program is published in the directory. This is only relevant if the program is included in the list of programs to be published within the package descriptor file. If not specified, the program's file name, without extension, is used.
+| description | string | A description of the program to place in the directory. This is only relevant if the program is included in the list of programs to be published within the package descriptor file.
+| categories | array of string | A list of categories under which the program will be listed in the directory, if it is to be published.
+| parameters | array of object | A list of the parameters for the program.
+
+#### SAS Language Programs
+In addition, for SAS language programs, the following properties can be specified:
+
+| Field | Type | Description |
+| ----- | ---- | ----------- |
+| resultType | string | One of the values `stream` or `dataset`.
+| resultContentType | string | The media type of the output in the case of `resultType` being set to `stream`
+| resultDatasetName | string | The name of the data set to be returned in the case of `resultType` being set to `dataset`
+| resultFormat | string | One of the values `json`, `csv` or `xml`. Determines how data sets are converted to text when generating the response.
+| parameterStyle | string | One of the values `macroVars` or `dataset`. Determines how parameters are passed into the program.
+
+These settings are described by way of the examples
+
+#### R Language Programs
+In addition, for R language programs, the following properties can be specified:
+
+| Field | Type | Description |
+| ----- | ---- | ----------- |
+| resultType | string | One of the values `stream`, `object` or `console`.
+| resultContentType | string | The media type of the output in the case of `resultType` being set to `stream`
+| resultObjectName | string | The name of the R object to be returned in the case of `resultType` being set to `object`
+| resultFormat | string | One of the values `json`, `csv`, `print` or `tab`. Determines how the result object is converted to text when generating the response.
+| parameterStyle | string | One of the values `macroVars` or `dataset`. Determines how parameters are passed into the program.
+| initText | string | A set of R program statements that are invoked once within the R runtime that will be used to invoke this program.
+
+These settings are described by way of the examples
+
+## Examples
+There are two sets of examples, one in the SAS language and one in the R language.
+
+* [SAS Language examples](sas/README.md)
+* [R Language examples](r/README.md)
+
+
+

package.yaml

@@ -0,0 +1,32 @@
+# This is a package descriptor file for the
+# package. It describes, amonst other things,
+# the list of programs to publish in the directory
+publishInDirectory:
+- sas/ExploringBooleanParameters.sas
+- sas/ExploringDateParameters.sas
+- sas/ExploringDatetimeParameters.sas
+- sas/ExploringIntParameters.sas
+- sas/ExploringTimeParameters.sas
+- sas/HelloWorld.sas
+- sas/HelloWithParameter.sas
+- sas/ParametersAsDataset.sas
+- sas/ReturnDataset.sas
+- sas/ReturningGraphics.sas
+- sas/ReturningHtml.sas
+- sas/ReturningListing.sas
+- sas/ReturningPdf.sas
+- sas/ReturnParametersAsDataset.sas
+- r/HelloWorld.r
+- r/ReturningDataFrame.r
+- r/ReturningDataFrameAsCsv.r
+- r/ReturningDataFrameAsPrint.r
+- r/ReturningConsoleOutput.r
+- r/ReturningStreamOutput.r
+- r/PassingParameters.r
+- r/BooleanParameters.r
+- r/IntegerParameters.r
+- r/DateParameters.r
+- r/TimeParameters.r
+- r/DatetimeParameters.r
+- r/ProcessingRequestBody.r
+- r/InitProgram.r

r/BooleanParameters.r

@@ -0,0 +1,7 @@
+result <- data.frame(
+    boolOptionalNoDefault,
+    boolOptionalDefaultFalse,
+    boolOptionalDefaultTrue,
+    boolRequiredNoDefault,
+    boolRequiredDefaultFalse,
+    boolRequiredDefaultTrue)

r/BooleanParameters.r.yaml

@@ -0,0 +1,47 @@
+label: PKGR07c - Exploring Boolean parameters
+categories:
+- Example
+- R Language Example
+- Package Example
+description: |
+  This example demonstrates use of a boolean parameter
+
+  The program has parameters that are required and optional,
+  and with default values of true and false.
+
+  The HTTP query parameters are converted from strings to
+  boolean values using the `as.logical` function.
+
+resultFormat: print
+resultObjectName: result
+parameters:
+  boolOptionalNoDefault:
+    label: Boolean Optional No Default
+    datatype: bool
+
+  boolOptionalDefaultFalse:
+    label: Boolean Optional Default False
+    datatype: bool
+    defaultValue: false
+
+  boolOptionalDefaultTrue:
+    label: Boolean Optional Default true
+    datatype: bool
+    defaultValue: true
+
+  boolRequiredNoDefault:
+    label: Boolean Required No Default
+    datatype: bool
+    required: true
+
+  boolRequiredDefaultFalse:
+    label: Boolean Required Default false
+    datatype: bool
+    defaultValue: false
+    required: true
+
+  boolRequiredDefaultTrue:
+    label: Boolean Required Default true
+    datatype: bool
+    defaultValue: true
+    required: true

r/DateParameters.r

@@ -0,0 +1,3 @@
+result <- data.frame(
+    dateRequired, dateOptional
+)

r/DateParameters.r.yaml

@@ -0,0 +1,36 @@
+label: PKGR07e - Exploring Date parameters
+categories:
+- Example
+- R Language Example
+- Package Example
+description: |
+  This example demonstrates use of date parameters.
+
+  Dates have different representations in different
+  programming languages. A client making a request
+  to execute a program (be that programatically through
+  something like cURL, or manually through an invocation
+  user interface) shouldn't have to worry about those
+  differences.
+
+  The format of a date value on a request is therefore
+  fixed and not dependent on the language in which the
+  program may have been written.
+
+  The HTTP query parameters are converted from strings to
+  date values by first using `as.numeric` to convert the
+  value into a numeric, then using the `as.Date` function.
+
+resultFormat: print
+resultObjectName: result
+
+parameters:
+  dateRequired:
+    label: Date Required
+    datatype: date
+    prompt: Enter a date
+    required: true
+  dateOptional:
+    label: Date Optional
+    datatype: date
+    prompt: Enter a date

r/DatetimeParameters.r

@@ -0,0 +1 @@
+result <- data.frame(datetimeParm)

r/DatetimeParameters.r.yaml

@@ -0,0 +1,21 @@
+label: PKGR07g - Exploring Datetime parameters
+categories:
+- Example
+- R Language Example
+- Package Example
+description: |
+  This example demonstrates use of a Datetime parameter.
+
+  The HTTP query parameters are converted
+  from strings by first using `as.numeric` to convert the
+  value to numeric, then using `asPOSIXct` to convert
+  the value into a datetime value.
+resultFormat: print
+resultObjectName: result
+
+parameters:
+  datetimeParm:
+    label: Datetime Parm
+    datatype: datetime
+    prompt: Enter a datetime
+    required: true

r/HelloWorld.r

@@ -0,0 +1 @@
+result <- 'Hello World!'

r/HelloWorld.r.yaml

@@ -0,0 +1,29 @@
+label: PKGR01 - Hello World
+categories:
+- Example
+- R Language Example
+- Package Example
+description: |
+  This is the simplest possible example, a program
+  that takes no input and produces a fixed output,
+  'Hello World!'.
+
+  By default the result that is returned is the value of
+  an R object, converted into text in one of a number of
+  ways: JSON (resultFormat=json), CSV (resultFormat=csv),
+  tab (tab separated, resultFormat=tab) and print (standard
+  R print format, resultFormat=print). Here we use JSON.
+  The object to return is named in the resultObjectName
+  property.
+
+  R objects are always arrays, so in the case of returning
+  JSON, you will always get a JSON array. In this case
+  the R object is a vector, so the output is an array of
+  strings.
+
+  The R object is converted to JSON using the jsonlite package,
+  invoking `jsonlite::toJSON`
+programText: |
+  result <- 'Hello World!'
+resultFormat: json
+resultObjectName: result

r/InitProgram.r

@@ -0,0 +1 @@
+result <- a*2

r/InitProgram.r.yaml

@@ -0,0 +1,40 @@
+label: PKGR09 - Use of initialisation program text
+categories:
+- Example
+- R Language Example
+- Package Example
+description: |
+  This example demonstrates use of the initialisation
+  program text.
+
+  The runtime server supports processing multiple requests
+  through the same R runtime for efficiency. The runtime
+  environment uses R environment objects to isolate each request
+  and to avoid invocation of a program poluting other invocations.
+
+  A program can specify an initialisation program text that will
+  be run only once per R runtime. This can be used to load
+  complex models into memory and avoid the overhead of having to
+  load a model on each request. The initialisation program text
+  is run within the context of a separate R environment, and this
+  environment is then locked to prevent further modification.
+
+  This example simply sets a variable in the initialisation text that
+  is then referenced in the main program text.
+
+  Note that the set up of the 'Demo' Environment object in the Hub
+  is such that a new worker process is created for each request. Since
+  a new worker process is created for each request, a new R runtime
+  is created per request anyway, so the initialisation program is
+  execute for each request. In order to properly see the effect of
+  the one-time execution of the initialisation program it is necessary
+  to set the `Idle Timeout` value on the `Default R Processing Engine`
+  engine of the Demo Environment to a value other than 0.
+
+  From the log it is possible to tell whether the initialisation
+  program has been run or not.
+
+resultFormat: json
+resultObjectName: result
+initProgramText: |
+  a <- 42

r/IntegerParameters.r

@@ -0,0 +1,8 @@
+result <- data.frame(
+    intOptionalNoDefault,
+    intOptionalDefault42,
+    intOptionalMax42,
+    intOptionalMin42,
+    intRequiredNoDefault,
+    intRequiredDefault42
+)

r/IntegerParameters.r.yaml

@@ -0,0 +1,43 @@
+label: PKGR07d - Exploring Integer parameters
+categories:
+- Example
+- R Language Example
+- Package Example
+description: |
+  This example demonstrates use of integer parameters
+
+  In the case of the SAS language there is no specific
+  integer type, so all numeric values are passed to
+  the program as doubles. However an invocation interface
+  will use the integer type to provide suitable input validation.
+
+  The HTTP query parameters are converted from strings to
+  integer values using the `as.integer` function.
+resultFormat: print
+resultObjectName: result
+parameters:
+  intOptionalNoDefault:
+    label: Int Optional No Default
+    datatype: int
+  intOptionalDefault42:
+    label: Int Optional Default 42
+    datatype: int
+    defaultValue: 42
+  intOptionalMax42:
+    label: Int Optional Max 42
+    datatype: int
+    max: 42
+  intOptionalMin42:
+    label: Int Optional Min 42
+    datatype: int
+    min: 42
+  intRequiredNoDefault:
+    label: Int Required No Default
+    datatype: int
+    required: true
+  intRequiredDefault42:
+    label: Int Required Default 42
+    datatype: int
+    defaultValue: 42
+    required: true
+

r/PassingParameters.r

@@ -0,0 +1 @@
+result <- paste('Hello', name)