Skip to content

Commit fd2ac01

Browse files
mr-cmr-c
authored
Merge branch 'master' into galaxy_deps
2 parents 5522e88 + 6e01be1 commit fd2ac01

16 files changed

+182
-15
lines changed

‎cwltool/schemas/README.md

+5-7
Original file line numberDiff line numberDiff line change
@@ -3,18 +3,20 @@ Common Workflow Language
33

44
[![GitHub stars](https://img.shields.io/github/stars/common-workflow-language/common-workflow-language.svg)](https://github.com/common-workflow-language/common-workflow-language/stargazers) [![Gitter](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/common-workflow-language/common-workflow-language?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) [**Support**](#Support)
55

6+
<a href="https://www.youtube.com/watch?v=86eY8xs-Vo8"><img align="right" src="https://github.com/common-workflow-language/logo/raw/master/intro_video_screenshot_413x193.png" alt="[Video] Common Workflow Language explained in 64 seconds"></a>
67
The Common Workflow Language (CWL) is a specification for describing analysis
78
workflows and tools in a way that makes them portable and scalable across a
89
variety of software and hardware environments, from workstations to cluster,
910
cloud, and high performance computing (HPC) environments. CWL is designed to
1011
meet the needs of data-intensive science, such as Bioinformatics, Medical
1112
Imaging, Astronomy, Physics, and Chemistry.
1213

14+
<a href="https://open-stand.org/about-us/principles"><img align="left" src="http://standards.ieee.org/images/openstand/128x128-blue2.png" alt="Open Stand badge"></a>
1315
CWL is developed by an informal, multi-vendor working group consisting of
1416
organizations and individuals aiming to enable scientists to share data
1517
analysis workflows. [The CWL project is on Github](https://github.com/common-workflow-language/common-workflow-language) and we follow the [Open-Stand.org principles for collaborative open standards development](https://open-stand.org/about-us/principles/)
1618

17-
[![Open Stand badge](http://standards.ieee.org/images/openstand/128x128-blue2.png)](https://open-stand.org/about-us/principles/).
19+
1820

1921
CWL builds on technologies such as [JSON-LD](http://json-ld.org)
2022
for data modeling and [Docker](http://docker.com) for portable runtime
@@ -30,12 +32,7 @@ descriptions.
3032

3133
For developers and advanced users, the current
3234
[CWL specification (v1.0)](http://www.commonwl.org/v1.0/) provides
33-
authoritative documentation of the execution of CWL documents. Links older
34-
drafts:
35-
[draft-1](https://github.com/common-workflow-language/common-workflow-language/tree/master/draft-1),
36-
[draft-2](http://www.commonwl.org/draft-2/),
37-
[draft-3](http://www.commonwl.org/draft-3/)
38-
35+
authoritative documentation of the execution of CWL documents.
3936
### Citation
4037

4138
Please cite https://dx.doi.org/10.6084/m9.figshare.3115156.v2
@@ -137,6 +134,7 @@ Conduct](https://github.com/common-workflow-language/common-workflow-language/bl
137134
* [Institute for Systems Biology](https://www.systemsbiology.org/)
138135
* [ELIXIR Europe](https://www.elixir-europe.org/)
139136
* [BioExcel CoE](http://bioexcel.eu/)
137+
* [BD2K](https://commonfund.nih.gov/bd2k)
140138

141139
## Individual Contributors
142140

‎cwltool/schemas/doc/cli-description-languages-comparison.md

+53
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
# Comparison of Command Line Interface description languages
2+
3+
## Compared languages
4+
5+
* Galaxy v17.05: https://github.com/galaxyproject/galaxy/blob/release_17.05/lib/galaxy/tools/xsd/galaxy.xsd
6+
* CTD v1.7.0 : https://github.com/WorkflowConversion/CTDSchema/blob/Version_1_7_0/CTD.xsd
7+
* CWL v1.0: http://www.commonwl.org/v1.0/CommandLineTool.html / https://github.com/common-workflow-language/common-workflow-language/blob/master/v1.0/CommandLineTool.yml
8+
9+
## Summary
10+
11+
### Tool description
12+
CTD reflects the perspective of a tool author targetting the KNIME processing unit approach ("nodes"), whereas
13+
Galaxy reflects the perspective of a platform centered around non-developer users interacting with a graphical interface.
14+
15+
### Parameters description
16+
17+
### Data/parameter types
18+
The typing system for parameters in CTD itself is simpler than the other two, providing mainly support for simple types and input/output files, but excluding arrays/lists and complex types. Galaxy adds GUI specific features (`select`, `drill_down`), domain specific features (`color`, `genome_build`, `data_column`), configuration of web services (`base_url`), and tighter integration with workbench internal data management (`library_data`).
19+
20+
### Command line generation
21+
All of the specifications include a flexible set of possibilities for the generation of command lines, reflecting the effective heterogeneity of the bioinformatics tools ecosystem ;)
22+
23+
## Comparison table
24+
25+
### Tool level
26+
27+
|Galaxy information |Optional |CTD information |Optional |CWL information |Optional |
28+
|---|---|---|---|---|---|
29+
|*description* |X |description |X |label |X |
30+
|*help* |X |*manual* |X |*doc* |X |
31+
|*citation* |X |*citation* (for the underlying tool, DOI or doc URL) |X |*SoftwareRequirement.name.specs* (URI) |X |
32+
| | |*ExecutableName* (overrides the name attribute) |X |*SoftwareRequirement.name.baseCommand[0]* |X |
33+
| | |*ExecutablePath* (specific path to the executable) |X | | |
34+
|*command*+*argument* under parameter | |*CLItype* list of input commands with elements mapping |X | *arguments* and/or *inputBindings* |X |
35+
| | |*logs* (retrospective on execution information - if so probably out of scope)|X | | |
36+
|*tool/outputs/data/@from_work_dir* (name of file to consume) |X |*relocators* (list of parameters, path pairs)|X |*outputBinding* |X |
37+
|*@version* (version of wrapper) |X |*@version* (apprarently version of the tool itself)| |*SoftwareRequirements.name.version* (list of known compatible versions of the underlying tool |X |
38+
|*@name* (just an identifier, not used to build the command itself)||*@name* (also used to build the command itself, unlesss *ExecutableName* is provided)| |baseCommand[0]|X|
39+
|often part of the *help* text|X|*docURL* (URL to documentation)||||
40+
|*edam_topics* and *edam_operations* - categories are external to tool definition, in per-server config (toolconf.xml)|X|*category* (any string)|X|can use EDAM or other 3rd party annotation or derive via identifier from external registry or datatype|
41+
|(often part of help text)|X|*tags*|X|||
42+
43+
### Parameter level
44+
45+
**Important remark** The structure of the CTD file is slightly different, because it explicitely allows the definition of "sub-tools" through a nested structure of NODE/ITEM elements. A node is a subgroup of parameters, and ITEM is a parameter.
46+
Therefore, to focus on Parameter-level information, we consider only the ITEM element here for parameter description in CTD.
47+
48+
|Galaxy information |Optional |CTD information |Optional |CWL information |Optional |
49+
|---|---|---|---|---|---|
50+
| *param* or *data* | | *ITEM* | | *inputs[n]* or *outputs[n]* | |
51+
|*@name*||*@name*||key of *inp|uts* or *outputs* entry||
52+
|*@type*||*@type* (Defines the possible types available in the type attribute of ITEM and ITEMLIST.)||type||
53+

‎cwltool/schemas/v1.0/CommandLineTool.yml

+3-2
Original file line numberDiff line numberDiff line change
@@ -36,10 +36,11 @@ $graph:
3636
- {$include: intro.md}
3737

3838
- |
39-
## Introduction to v1.0
39+
## Introduction to CWL Command Line Tool standard v1.0
4040
4141
This specification represents the first full release from the CWL group.
42-
Since draft-3, version 1.0 introduces the following changes and additions:
42+
Since draft-3, version 1.0 introduces the following changes and additions
43+
to the CWL Command Line Tool standard:
4344
4445
* The [Directory](#Directory) type.
4546
* Syntax simplifcations: denoted by the `map<>` syntax. Example: inputs

‎cwltool/schemas/v1.0/Process.yml

+3-1
Original file line numberDiff line numberDiff line change
@@ -563,7 +563,9 @@ $graph:
563563
564564
- name: default
565565
type: Any?
566-
jsonldPredicate: "cwl:default"
566+
jsonldPredicate:
567+
_id: cwl:default
568+
noLinkCheck: true
567569
doc: |
568570
The default value for this parameter if not provided in the input
569571
object.

‎cwltool/schemas/v1.0/Workflow.yml

+6-3
Original file line numberDiff line numberDiff line change
@@ -34,10 +34,11 @@ $graph:
3434

3535
- |
3636
37-
## Introduction to v1.0
37+
## Introduction to CWL Workflow standard v1.0
3838
3939
This specification represents the first full release from the CWL group.
40-
Since draft-3, this draft introduces the following changes and additions:
40+
Since draft-3, this draft introduces the following changes and additions
41+
to the CWL Workflow standard:
4142
4243
* The `inputs` and `outputs` fields have been renamed `in` and `out`.
4344
* Syntax simplifcations: denoted by the `map<>` syntax. Example: `in`
@@ -260,7 +261,9 @@ $graph:
260261
doc: |
261262
The default value for this parameter if there is no `source`
262263
field.
263-
jsonldPredicate: "cwl:default"
264+
jsonldPredicate:
265+
_id: "cwl:default"
266+
noLinkCheck: true
264267
- name: valueFrom
265268
type:
266269
- "null"

‎cwltool/schemas/v1.0/conformance_test_v1.0.yaml

+11
Original file line numberDiff line numberDiff line change
@@ -981,3 +981,14 @@
981981
size: 15
982982
tool: v1.0/imported-hint.cwl
983983
doc: Test hints with $import
984+
985+
- output: {}
986+
job: v1.0/default_path_job.yml
987+
tool: v1.0/default_path.cwl
988+
doc: Test warning instead of error when default path is not found
989+
990+
- output:
991+
args: [-A,'2',-B,baz,-C,'10','9','8','7','6','5','4','3','2','1']
992+
job: v1.0/empty.json
993+
tool: v1.0/inline-js.cwl
994+
doc: Test InlineJavascriptRequirement with multiple expressions in the same tool

‎cwltool/schemas/v1.0/v1.0/default_path.cwl

+10
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
cwlVersion: v1.0
2+
class: CommandLineTool
3+
inputs:
4+
- id: "file1"
5+
type: File
6+
default:
7+
class: File
8+
path: default.txt
9+
outputs: []
10+
arguments: [cat,$(inputs.file1.path)]

‎cwltool/schemas/v1.0/v1.0/default_path_job.yml

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
file1:
2+
class: File
3+
path: hello.txt

‎cwltool/schemas/v1.0/v1.0/inline-js.cwl

+38
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
cwlVersion: v1.0
2+
class: CommandLineTool
3+
baseCommand: echo
4+
5+
requirements:
6+
- class: InlineJavascriptRequirement
7+
8+
inputs:
9+
- id: args.py
10+
type: File
11+
default:
12+
class: File
13+
location: args.py
14+
inputBinding:
15+
position: -1
16+
17+
outputs:
18+
- id: args
19+
type:
20+
type: array
21+
items: string
22+
23+
baseCommand: python
24+
25+
arguments:
26+
- prefix: -A
27+
valueFrom: $(1+1)
28+
- prefix: -B
29+
valueFrom: $("/foo/bar/baz".split('/').slice(-1)[0])
30+
- prefix: -C
31+
valueFrom: |
32+
${
33+
var r = [];
34+
for (var i = 10; i >= 1; i--) {
35+
r.push(i);
36+
}
37+
return r;
38+
}

‎cwltool/schemas/v1.1.0-dev1/Process.yml

+3-1
Original file line numberDiff line numberDiff line change
@@ -564,7 +564,9 @@ $graph:
564564
565565
- name: default
566566
type: Any?
567-
jsonldPredicate: "cwl:default"
567+
jsonldPredicate:
568+
_id: cwl:default
569+
noLinkCheck: true
568570
doc: |
569571
The default value for this parameter if not provided in the input
570572
object.

‎cwltool/schemas/v1.1.0-dev1/Workflow.yml

+3-1
Original file line numberDiff line numberDiff line change
@@ -267,7 +267,9 @@ $graph:
267267
doc: |
268268
The default value for this parameter if there is no `source`
269269
field.
270-
jsonldPredicate: "cwl:default"
270+
jsonldPredicate:
271+
_id: "cwl:default"
272+
noLinkCheck: true
271273
- name: valueFrom
272274
type:
273275
- "null"

‎cwltool/schemas/v1.1.0-dev1/conformance_test_v1.1.0-dev1.yaml

+5
Original file line numberDiff line numberDiff line change
@@ -992,3 +992,8 @@
992992
size: 15
993993
tool: v1.1.0-dev1/imported-hint.cwl
994994
doc: Test hints with $import
995+
996+
- output: {}
997+
job: v1.0/default_path_job.yml
998+
tool: v1.0/default_path.cwl
999+
doc: Test warning instead of error when default path is not found

‎cwltool/schemas/v1.1.0-dev1/v1.1.0-dev1/default_path.cwl

+10
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
cwlVersion: v1.0
2+
class: CommandLineTool
3+
inputs:
4+
- id: "file1"
5+
type: File
6+
default:
7+
class: File
8+
path: default.txt
9+
outputs: []
10+
arguments: [cat,$(inputs.file1.path)]

‎cwltool/schemas/v1.1.0-dev1/v1.1.0-dev1/default_path_job.yml

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
file1:
2+
class: File
3+
path: hello.txt

‎tests/test_default_path.py

+15
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
import unittest
2+
from cwltool.load_tool import fetch_document, validate_document
3+
from .util import get_data
4+
from schema_salad.ref_resolver import Loader
5+
6+
class TestDefaultPath(unittest.TestCase):
7+
# Testing that error is not raised when default path is not present
8+
def test_default_path(self):
9+
document_loader, workflowobj, uri = fetch_document(
10+
get_data("tests/wf/default_path.cwl"))
11+
document_loader, avsc_names, processobj, metadata, uri = validate_document(
12+
document_loader, workflowobj, uri)
13+
14+
self.assertIsInstance(document_loader,Loader)
15+
self.assertIn("cwlVersion",processobj)

‎tests/wf/default_path.cwl

+11
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
cwlVersion: v1.0
2+
class: CommandLineTool
3+
inputs:
4+
- id: "file1"
5+
type: File
6+
default:
7+
class: File
8+
path: default.txt
9+
outputs: []
10+
arguments: [cat, $(inputs.file1.path)]
11+

0 commit comments

Comments
 (0)