Add docs for importer service, per review

Signed-off-by: Jim Crossley <[email protected]>

Author: jcrossley3

docs/env-vars.md

@@ -20,6 +20,8 @@
 | `HTTP_SERVER_TLS_ENABLED`                | Enable TLS                                                                          | `false`                                 |
 | `HTTP_SERVER_TLS_KEY_FILE`               | Path to the TLS key file in PEM format                                              |                                         |
 | `HTTP_SERVER_WORKERS`                    | Number of worker threads, defaults to zero, which falls back to the number of cores | `0`                                     |
+| `IMPORTER_CONCURRENCY`                   | The maximum number of jobs run simultaneously by the importer                       | `1`                                     |
+| `IMPORTER_WORKING_DIR`                   | Where the importer downloads documents prior to ingesting them                      | `tempdir`                               |
 | `OIDC_PROVIDER_CLIENT_ID`                | OIDC client ID used for retrieving access tokens                                    |                                         |
 | `OIDC_PROVIDER_CLIENT_SECRET`            | Secret matching the OIDC client ID                                                  |                                         |
 | `OIDC_PROVIDER_ISSUER_URL`               | OIDC issuer to request access tokens from                                           |                                         |

modules/importer/README.md

@@ -1,46 +1,67 @@
 # Importer
 
-## Create a new CSAF importer
+## Server configuration
+
+The `importer` service is started using `trustd`, e.g.
+
+```shell
+trustd importer --concurrency=4 --working-dir=".trustify/importer"
+```
+
+The default value for `concurrency` is 1. This is the maximum number
+of importer jobs run simultaneously by the service.
+
+The `importer` should be started with the same database and storage
+options as the associated `api` service. Run the following to see what
+those are:
+
+```shell
+trustd importer --help
+```
+
+## Client API
+
+### Create a new CSAF importer
 
 ```shell
 http POST localhost:8080/api/v2/importer/redhat-csaf csaf[source]=https://redhat.com/.well-known/csaf/provider-metadata.json csaf[disabled]:=false csaf[onlyPatterns][]="^cve-2023-" csaf[period]=30s csaf[v3Signatures]:=true
 ```
 
-## Create a new OSV importer
+### Create a new OSV importer
 
 ```shell
 http POST localhost:8080/api/v2/importer/osv-r osv[source]=https://github.com/RConsortium/r-advisory-database osv[path]=vulns osv[disabled]:=false osv[period]=30s
 ```
 
-## Create a new SBOM importer
+### Create a new SBOM importer
 
 Quarkus & RHEL 9 data:
 
 ```shell
 http POST localhost:8080/api/v2/importer/redhat-sbom sbom[source]=https://access.redhat.com/security/data/sbom/beta/ sbom[keys][]=https://access.redhat.com/security/data/97f5eac4.txt#77E79ABE93673533ED09EBE2DCE3823597F5EAC4 sbom[disabled]:=false sbom[onlyPatterns][]=quarkus sbom[onlyPatterns][]=rhel-9 sbom[period]=30s sbom[v3Signatures]:=true
 ```
 
-## Get all importers
+### Get all importers
 
 ```shell
 http GET localhost:8080/api/v2/importer
 ```
 
-## Get a specific importer
+### Get a specific importer
 
 ```shell
 http GET localhost:8080/api/v2/importer/redhat-csaf
 http GET localhost:8080/api/v2/importer/redhat-sbom
 ```
 
-## Get reports
+### Get reports
 
 ```shell
 http GET localhost:8080/api/v2/importer/redhat-csaf/report
 http GET localhost:8080/api/v2/importer/redhat-sbom/report
 ```
 
-## Update an importer configuration
+### Update an importer configuration
 
 ```shell
 http PUT localhost:8080/api/v2/importer/redhat-csaf csaf[source]=https://redhat.com/.well-known/csaf/provider-metadata.json csaf[disabled]:=false csaf[period]=30s csaf[v3Signatures]:=true csaf[fetchRetries]:=50
@@ -58,29 +79,27 @@ To execute:
 http GET localhost:8080/api/v2/importer/redhat-csaf | jq .configuration | jq .csaf.fetchRetries=50 | http PUT localhost:8080/api/v2/importer/redhat-csaf
 ```
 
-## Patch an importer configuration
+### Patch an importer configuration
 
 ```shell
 http PATCH localhost:8080/api/v2/importer/redhat-csaf "Content-Type:application/merge-patch+json" csaf[fetchRetries]:=50
 ```
 
-## Delete an importer
+### Delete an importer
 
 ```shell
 http DELETE localhost:8080/api/v2/importer/redhat-csaf
 http DELETE localhost:8080/api/v2/importer/redhat-sbom
 ```
 
-## Set the enabled state of an importer
+### Set the enabled state of an importer
 
 ```shell
 echo true | http PUT localhost:8080/api/v2/importer/redhat-sbom/enabled
 ```
 
-## Force an importer run
+### Force an importer run
 
 ```shell
 http PUT localhost:8080/api/v2/importer/redhat-sbom/force
 ```
-
-