chore: remove open-vulnerability-client

resolves #275

.github/workflows/gradle.yml

@@ -33,4 +33,3 @@ jobs:
         retention-days: 7
         path: |
           vulnz/build/reports/tests
-          open-vulnerability-clients/build/reports/tests

README.md

@@ -1,25 +1,161 @@
-# Open Vulnerability Project
+# vulnz cli
 
-The Open Vulnerability Project is a collection of Java libraries and a CLI to work
-with various vulnerability data-sources (NVD, GitHub Security Advisories, CISA Known Exploited Vulnerablity Catalog, FIRST Exploit Prediction Scoring System (EPSS), etc.).
+The cli is a spring-boot command line tool built with picocli. The example
+below does run the setup - which creates both the `vulnz` symlink (in `/usr/local/bin`)
+and a completion script. If using zsh, the completion will be added to
+`/etc/bash_completion.d` or `/usr/local/etc/bash_completion.d` (depending
+on if they exist); see [permanently installing completion](https://picocli.info/autocomplete.html#_installing_completion_scripts_permanently_in_bashzsh)
+for more details. We may add a brew formula in the future.
 
-- [open-vulnerability-clients](/open-vulnerability-clients) is a collection of clients to retrieve vulnerability data from various data-feeds and APIs.
-- [vulnz](/vulnz) a simple CLI that can be used to access the vulnerability sources and persist the data using the open-vulnerability-store.
+After running `install` you may need to restart your shell for the completion to work.
 
-## Caching the NVD CVE API Data
+```bash
+./gradlew vulnz:build
+cd vulnz/build/libs
+./vulnz-7.2.2.jar install
+vulnz cve --cveId CVE-2021-44228 --prettyPrint
+```
 
-One of the primary uses of the vulnz CLI is to be able to create a local cache of
-the NVD CVE Data from their API. See the [vulnz/README.md](/vulnz/README.md#caching-the-nvd-cve-data)
-for instructions on how to create and maintain the local cache.
+Example of using the CLI with an API key stored in [1password](https://1password.com/) using
+the `op` CLI (see [getting started with op](https://developer.1password.com/docs/cli/get-started/)):
 
-## Upgrading from vuln-tools
+```bash
+export NVD_API_KEY=op://vaultname/nvd-api/credential
+eval $(op signin)
+op run -- vulnz cve --threads 4 > cve-complete.json
+```
 
-The project started off called vuln-tools and the various APIs were seperated into
-standalone JAR files. The project has been renamed to the Open Vulnerability Project.
+## Caching the NVD CVE Data
 
-- All of the client libraries are now in the [open-vulnerability-clients](/open-vulnerability-clients).
-- Packages have been renamed/moved:
-    - `io.github.jeremylong.ghsa.*` -> `io.github.jeremylong.openvulnerability.client.ghsa.*`
-    - `io.github.jeremylong.nvdlib.*` -> 'io.github.jeremylong.openvulnerability.client.nvd.*'
-    - `io.github.jeremylong.nvdlib.nvd` -> 'io.github.jeremylong.openvulnerability.client.nvd.*'
-- The `NvdCveApi` class has been renamed to `NvdCveClient`.
+The vulnz cli can create a cache of the NVD CVE data obtained from the API. The
+data is stored in `json` files with the data saved in the traditional yearly groupings
+starting with 2002 and going to the current year. In addition, a `cache.properties` is
+created that contains the `lastModifiedDate` datetime as well as the prefix used for the
+generated JSON files (by default `nvdcve-` is used). Additionally, a `modified` JSON file
+is created that will hold the CVEs that have been modified in the last 7 days. After running
+the below command you will end up with a directory with:
+
+- `cache.properties`
+- `nvdcve-modified.json.gz`
+- `nvdcve-modified.meta`
+- `nvdcve-2002.json.gz`
+- `nvdcve-2002.meta`
+- `nvdcve-2003.json.gz`
+- `nvdcve-2003.meta`
+- ...
+- `nvdcve-2024.json.gz`
+- `nvdcve-2024.meta`
+
+### API Key is used and a 403 or 404 error occurs
+
+If an API Key is used and you receive a 404 error:
+
+```
+ERROR
+io.github.jeremylong.openvulnerability.client.nvd.NvdApiException: NVD Returned Status Code: 404
+```
+
+There is a good chance that the API Key is set incorrectly or is invalid. To check if the API Key works
+the following `curl` command should return JSON:
+
+```
+curl -H "Accept: application/json" -H "apiKey: ########-####-####-####-############" -v https://services.nvd.nist.gov/rest/json/cves/2.0\?cpeName\=cpe:2.3:o:microsoft:windows_10:1607:\*:\*:\*:\*:\*:\*:\*
+```
+
+If no JSON is returned and you see a 404 error the API Key is invalid and you should request a new one.
+
+### Out-of-Memory Errors
+
+Create the local cache may result in an out-of-memory error. To resolve the
+error simply increase the available memory for Java:
+
+```bash
+export JAVA_OPTS="-Xmx2g"
+```
+
+Alternatively, run the CLI using the `-Xmx2g` argument:
+
+```bash
+java -Xmx2g -jar ./vulnz-7.2.2.jar
+```
+
+An option to save memory would be: `-XX:+UseStringDeduplication`:
+```bash
+export JAVA_OPTS="-Xmx2g -XX:+UseStringDeduplication"
+```
+
+### Creating the Cache
+
+To create a local cache of the NVD CVE Data you can execute the following command
+via a daily schedule to keep the cached data current:
+
+```bash
+vulnz cve --cache --directory ./cache 
+```
+
+Alternatively, without using the above install command:
+
+```bash
+./vulnz-7.2.2.jar cve --cache --directory ./cache
+```
+
+When creating the cache all other arguments to the vulnz cli
+will still work except the `--lastModEndDate` and `--lastModStartDate`.
+As such, you can create `--prettyPrint` the cache or create a cache
+of only "application" CVE using the `--virtualMatchString=cpe:2.3:a`.
+
+## Docker image
+
+### Configuration
+
+There are a couple of ENV vars
+
+- `NVD_API_KEY`: define your API key
+- `DELAY`: override the delay - given in milliseconds. If you do not set an API KEY, the delay will be `10000`
+- `MAX_RETRY_ARG` Using max retry attempts
+- `MAX_RECORDS_PER_PAGE_ARG` Using max records per page
+- `METRICS_ENABLE` If is set to `true`, OpenMetrics data for the vulnz cli can be retrieved via the endpoint http://.../metrics
+- `METRICS_WRITE_INTERVAL` Sets the update interval for generating metrics, in milliseconds. Default: `5000`
+- `METRICS_WRITE_FORMAT` Sets the output format for the metrics. Either openmetrics or Prometheus format. Default: `openmetrics`
+
+### Run
+
+```bash
+# replace the NVD_API_KEY with your NVD api key
+docker run --name vulnz -e NVD_API_KEY=myapikey jeremylong/open-vulnerability-data-mirror:v7.2.2 
+
+# if you like use a volume 
+docker run --name vulnz -e NVD_API_KEY=myapikey -v cache:/usr/local/apache2/htdocs jeremylong/open-vulnerability-data-mirror:v7.2.2
+
+# adjust the memory usage
+docker run --name vulnz -e JAVA_OPT=-Xmx2g jeremylong/open-vulnerability-data-mirror:v7.2.2
+
+# you can also adjust the delay 
+docker run --name vulnz -e NVD_API_KEY=myapikey -e DELAY=3000 jeremylong/open-vulnerability-data-mirror:v7.2.2 
+
+```
+
+If you like, run this to pre-populate the database right away
+
+```bash
+docker exec -u mirror vulnz /mirror.sh
+```
+
+### Build
+
+Assuming the current version is `7.2.2`
+
+```bash
+export TARGET_VERSION=7.2.2
+./gradlew vulnz:build -Pversion=$TARGET_VERSION
+docker build vulnz/ -t ghcr.io/jeremylong/vulnz:$TARGET_VERSION --build-arg BUILD_VERSION=$TARGET_VERSION
+```
+
+### Release
+
+```bash
+# checkout the repo
+git tag -a 'v7.2.2'' -m 'release 7.2.2'
+git push --tags
+# this will build vulnz 7.2.2 on publish the docker image tagged 7.2.2 
+```