updating agave cli basics sections

Author: wjallen

docs/systems_basics.md renamed to docs/02.01.systems_basics.md

@@ -10,7 +10,7 @@ hostname. They may be public or private, and they may either be **storage** syst
 
 In SD2E, we provide all users with private storage and execution systems attached to
 TACC resources. If you would like to create your own systems attached to different
-resources, skip ahead to [Create Private Systems](create_systems.md).
+resources, skip ahead to [Create Private Systems](02.06.create_systems.md).
 
 
 <br>
@@ -26,9 +26,10 @@ data-sd2e-projects-users
 data-sd2e-projects-etldev
 data-sd2e-app-assets
 ```
-Here is a quick summary of the default storage systems:
+Here is a quick summary of the default storage systems and how they relate to
+the [SD2E portal](https://sd2e.org/):
 
-* `data-tacc-work-username`: Your personal directory on the TACC `$WORK` filesystem. You have +rw and sharing access. This will show up in the portal as **My Data**.
+* `data-tacc-work-username`: Your personal directory on the TACC `$WORK` filesystem. You have `+rw` and sharing access. This will show up in the portal as **My Data**.
 
 * `data-sd2e-community`: Project-wide shared data. Unless you are specifically authorized, it is read-only. This will show up in the portal as **Community Data**.
 
@@ -38,7 +39,7 @@ Here is a quick summary of the default storage systems:
 
 <br>
 Systems where you are able to execute commands and run jobs (called execution systems)
-can be found with the following command: 
+can be found with the following command:
 ```
 % systems-list -E
 hpc-tacc-maverick-username
@@ -65,7 +66,6 @@ Here is a quick summary of the default execution systems:
 * `hpc-tacc-jetstream`: Public CLI execution system for the TACC [Jetstream](https://portal.tacc.utexas.edu/user-guides/jetstream) cloud resource. Most SD2E community members have `USER` access and will be able to run public applications set up to run on these systems.
 
 
-
 *Tip: use '`systems-list -h`' to see more usage options*
 
 <br>
@@ -257,10 +257,10 @@ information about queue types and availability.
 <br>
 #### Set a default system
 
-You may find that most of the [file operations](data_management.md) you perform
-are on your private TACC storage system - `data-tacc-work-username`. To avoid
-typing the full name of the system with every file command, you may set it as the
-default with:
+You may find that most of the [file operations](02.02.data_management.md) you
+perform are against your private TACC storage system - `data-tacc-work-username`.
+To avoid typing the full name of the system with every file command, you may set
+it as the default system with:
 ```
 % systems-setdefault data-tacc-work-username
 Successfully set data-tacc-work-username as the default storage system for username

docs/data_management.md renamed to docs/02.02.data_management.md

@@ -5,12 +5,12 @@ tagline:
 ---
 
 Data is managed using the `files` service of the Agave CLI. With many parallels in
-Linux file commands (`ls, mv, cp, rm,` etc.), Agave `files-*` commands can be used to 
+Linux file commands (`ls, mv, cp, rm,` etc.), Agave `files-*` commands can be used to
 list files, upload and download data, remotely manage and organize data, and import
 external data from the web.
 
-Agave `files-*` commands act on Agave STORAGE `systems`. See the [systems basics](systems_basics.md)
-documentation for more details.
+Agave `files-*` commands act on Agave storage systems. See the
+[systems basics](02.01.systems_basics.md) documentation for more details.
 
 This page is broken down into the following sections:
 
@@ -23,15 +23,15 @@ This page is broken down into the following sections:
 <br>
 #### Listing files and navigating
 
-To list the files available to you on a STORAGE system, use:
+To list the files available to you on a storage system, use:
 ```
 % files-list -S data-tacc-work-username -L /
 drwx------  username  4096  13:07  .
 ```
 
 The `-L` flag provides a long listing (similar to `ls -l` in Linux), and the
 optional forward slash (`/`) at the end of the line indicates the root path -
-your `$WORK` directory. From the listing above, we have no files or folders 
+your `$WORK` directory. From the listing above, we have no files or folders
 available to us yet.
 
 To make a new folder, then list the contents of that folder, do:
@@ -49,23 +49,24 @@ drwx------  username  4096  15:49  .
 
 To remove a folder, use the `files-delete` command:
 ```
-% files-delete -S data-tacc-work-username sd2e-data/ 
+% files-delete -S data-tacc-work-username sd2e-data/
 Successfully deleted sd2e-data/
 ```
 
-WARNING: The `files-delete` command will delete folders with or without content.
+WARNING: The `files-delete` command will delete folders with or without contents.
 
-*Tip: if you set `data-tacc-work-username` as your default system, then you 
-do not need to provide it on the command line. More info [here](systems_basics.md).*
+*Tip: if you set `data-tacc-work-username` as your default system, then you
+do not need to provide it on the command line. More info
+[here](02.01.systems_basics.md).*
 
 <br>
 #### Uploading and downloading via the CLI
 
-Files can be transferred from your local machine to the remote STORAGE system
-using the `files-upload` command, and from the remote STORAGE system to your
+Files can be transferred from your local machine to the remote storage system
+using the `files-upload` command, and from the remote storage system to your
 local machine using the `files-get` command.
 
-First, find or create a local file and upload it to the STORAGE system (recreate
+First, find or create a local file and upload it to the storage system (recreate
 the sd2e-data folder if you deleted it in the previous example):
 ```
 % touch my_file.txt
@@ -79,7 +80,7 @@ drwx------  username  4096  15:53  .
 -rw-------  username  0     15:53  my_file.txt
 ```
 
-Use the `files-copy` command to make a copy of the file on the remote STORAGE system:
+Use the `files-copy` command to make a copy of the file on the remote storage system:
 ```
 % files-copy -S data-tacc-work-username -D sd2e-data/my_copy.txt sd2e-data/my_file.txt
 Successfully copied sd2e-data/my_file.txt to sd2e-data/my_copy.txt
@@ -91,10 +92,10 @@ drwx------  username  4096  15:57  .
 ```
 
 The `-D` flag specifies the destination of the copied file, including path and
-new file name. Experienced Linux command line users may find the order of options 
+new file name. Experienced Linux command line users may find the order of options
 given to be counterintuitive - first the name of the target is given, then the
 name of the original file is given. This is a common convention for many of the
-Agave CLI file commands. 
+Agave CLI file commands.
 
 Then, download the result:
 ```
@@ -109,7 +110,7 @@ my_copy.txt  my_file.txt
 #### Other file operations
 
 Using the Agave CLI, files and folders can also be renamed, moved, and deleted remotely on
-the STORAGE system. The syntax for these operations is very similar to the 
+the storage system. The syntax for these operations is very similar to the
 `files-copy` command syntax. Here are some common examples:
 ```
 # Rename a file in place
@@ -130,15 +131,16 @@ $ files-move -S data-tacc-work-username -D sd2e-data/subfolder/renamed.txt sd2e-
 ```
 
 Be cautious with `files-move` and `files-delete` commands. Just like a Linux
-filesystem, files inadvertently deleted or written over are probably unrecoverable.
+filesystem, files inadvertently deleted or written over are most likely
+unrecoverable.
 
 
 <br>
 #### File or folder history
 
 You can list the history of events for a specific file or folder. This will give
 more descriptive information (when applicable) related to number of retries, permission
-grants and revocations, reasons for failure, and hiccups that may have occurred in 
+grants and revocations, reasons for failure, and hiccups that may have occurred in
 the transfer process.
 ```
 % files-history -S data-tacc-work-username sd2e-data/my_copy.txt                  
@@ -150,7 +152,7 @@ File item copied from https://api.sd2e.org/files/v2/media/system/data-tacc-work-
 
 Reminder: at any time, you can issue an Agave command with the `-h` flag to
 find more information on the function and usage of the command. Extensive Agave
-documentation can be found [here](https://tacc.github.io/developer.tacc.cloud/).
+documentation can be found [here](http://developer.tacc.cloud/).
 
 ---
 Return to the [API Documentation Overview](../index.md)

docs/share_import.md renamed to docs/02.03.share_import.md

@@ -35,7 +35,7 @@ View your own user profile by issuing:
   "phone": "",
   "mobile_phone": "",
   "status": "",
-  "create_time": "20150223162108Z", 
+  "create_time": "20150223162108Z",
   "username": "juser"
 }
 ```
@@ -44,24 +44,19 @@ Each of the fields stored in the JSON object is queryable using the profiles-sea
 command. Some more common examples include:
 ```
 # Search for another user by first name
-% profiles-search --rich 'first_name=Joe'
+% profiles-search --rich --filter first_name,last_name,email,username 'first_name=Joe'
 
 # Search for another user by last name
-% profiles-search --rich 'last_name=User'
+% profiles-search --rich --filter first_name,last_name,email,username 'last_name=User'
 
 # Search for another user by e-mail address
-% profiles-search --rich '[email protected]'
-
-# Search for another user by guessing all or part of their username
-% profiles-search --rich 'username.like=*juser*'
+% profiles-search --rich --filter first_name,last_name,email,username '[email protected]'
 ```
 
-The `--rich` flag summarizes part of the JSON response in a human-friendly table.
+The `--rich` flag summarizes part of the JSON response in a human-friendly
+table, and the `--filter` flag specifies which of the available fields you would
+like to see.
 
-Note: There is a file called 'richtext' in the CLI bin directory that needs to
-be updated for `profiles-search`. Fields were changed as some point from `firstName`
-and `lastName` to `first_name` and `last_name`. Also, `institution` is no longer
-a field so it must be removed.
 
 <br>
 #### Share files with another user
@@ -79,8 +74,8 @@ username READ WRITE EXECUTE
 vaughn READ WRITE EXECUTE
 ```
 
-To add permissions for another user to view the file, use the `files-pems-update`
-command:
+To add permissions for another user (in this case with username '`collaborator`')
+to view the file, use the `files-pems-update` command:
 ```
 % files-pems-update -U collaborator -P ALL sd2e-data/my_file.txt
 % files-pems-list -S data-tacc-work-username sd2e-data/my_file.txt
@@ -107,7 +102,7 @@ username ADMIN
 
 To add your collaborator to your system, use the `systems-roles-addupdate` command:
 ```
-% systems-roles-addupdate -u collaborator -r GUEST data-tacc-work-username 
+% systems-roles-addupdate -u collaborator -r GUEST data-tacc-work-username
 Successfully updated roles for user collaborator on data-tacc-work-username
 
 % systems-roles-list data-tacc-work-username
@@ -118,7 +113,7 @@ username ADMIN
 ```
 
 Now, a user with username `collaborator` can see files with the appropriate
-permissions on your STORAGE system. Valid values for setting a role with the `-r` 
+permissions on your STORAGE system. Valid values for setting a role with the `-r`
 flag are GUEST, USER, PUBLISHER, ADMIN, and OWNER.
 
 Finally, ask your collaborator to download the file with the exact same command
@@ -134,14 +129,15 @@ you use to download the file:
 Another convenient way to share data is the Agave postits service. Postits
 generate a short URL with a user-specified lifetime and limited number of uses.
 Anyone with the URL can paste it into a web browser, or curl against it on the
-command line. To create a postit: 
+command line. To create a postit:
 ```
-% postits-create -m 5 -l 3600 https://api.sd2e.org/files/v2/media/system/data-tacc-work-username/sd2e-data/my_file.txt
+% postits-create -m 5 -l 3600 \
+    https://api.sd2e.org/files/v2/media/system/data-tacc-work-username/sd2e-data/my_file.txt
 ```
 
-The json response from this command is the URL, e.g.:
+The json response from this command is a URL, e.g.:
 
-``` 
+```
 https://api.sd2e.org/postits/v2/866d55b36a459e8098173655e916fa15
 ```
 
@@ -168,11 +164,11 @@ for that purpose.
 ```
 
 With the above syntax, the file located at the root directory on the
-`data-sd2e-community` STORAGE system will be imported to your private STORAGE
+`data-sd2e-community` storage system will be imported to your private storage
 system, and placed in your directory `sd2e-data/`.
 
 Please also note that even though you are *able* to import files from other
-Agave STORAGE systems, you may not always *need* to import those files. Most
+Agave storage systems, you may not always *need* to import those files. Most
 applications of Agave will allow you to provide a complete URI path to the file,
 e.g. `agave://data-sd2e-community/public_data.dat`. This is useful, for example,
 in the case of large reference libraries. Pointing to the remote libraries

docs/find_application.md renamed to docs/02.04.find_application.md

@@ -4,7 +4,7 @@ title: Find an Application
 tagline:
 ---
 
-The SD2E catalog has several pre-built public applications (or "apps" for short) 
+The SD2E catalog has several pre-built public applications (or "apps" for short)
 that are available to run right now. Each public application is tied to a public
 HPC system.
 
@@ -14,6 +14,7 @@ HPC system.
 The `apps-list` command can be used to list all applications you have access to:
 ```
 % apps-list
+
 fastqc-0.11.5u2
 fcs-etl-0.3.3u3
 fcs-tasbe-0.2.0u6
@@ -24,6 +25,7 @@ kallisto-0.43.1u3
 lcms-maverick-0.1.0u2
 lcms-maverick-0.1.0u3
 lcms-pyquant-0.1.0u1
+lcms-pyquant-0.1.1u1
 lcms-wrangler-0.1.0u2
 lcms-wrangler-0.1.0u3
 msf-0.1.0u3
@@ -214,9 +216,9 @@ The `inputs` section describe what input data are required for running this app:
   ],
 ```
 
-It appears this app is expecting three files as inputs, identified by the `id`
-field: `transcripts`, `fastq1`, and `fastq2`. Each has a short description, and
-all three are required to run this app.
+This app is expecting three files as inputs, identified by the `id` field:
+`transcripts`, `fastq1`, and `fastq2`. Each has a short description, and all
+three are required (`"required": true`) to run this app.
 
 <br>
 #### The `parameters` section
@@ -306,7 +308,7 @@ Next, inspect the `parameters` section of the app json file:
 ```
 
 It is also expecting three parameters, identified by the `id` field: `bootstrap`,
-`seed`, and `output`. This time, only the `output` parameter is requried. The 
+`seed`, and `output`. This time, only the `output` parameter is requried. The
 purpose and acceptable values for each parameter are provided in the description.
 
 <br>
@@ -363,9 +365,9 @@ Here, we see two outputs are created when running this app: `abundance` and
 However, this is an important clue (along with the inputs, parameters, and app
 description) to indicate what is the function of this app.
 
-Once the appropriate app has been identified, the next steps are to 
-[stage the input data](data_management.md), and to
-[prepare and submit a job](submit_job).
+Once the appropriate app has been identified, the next steps are to
+[stage the input data](02.02.data_management.md), and to
+[prepare and submit a job](02.05.submit_job).
 
 ---
 Return to the [API Documentation Overview](../index.md)