Merge pull request #13 from shashank-jfrog/feature/RTDEV-52485

Enhanced worker samples

Author: yashprit-jfrog

samples/artifactory/AFTER_DOWNLOAD/report-download-to-external-endpoint/README.md

@@ -1,3 +1,72 @@
-# Log download activity to an external endpoint
+# Log Download activity to an external endpoint
 
-This simple worker will send a log containing the name of the artifact that has been downloaded, the repository and the token/userid that requested the download.
+## Overview  
+This worker listens to the `AFTER_DOWNLOAD` event in JFrog Artifactory and notifies an external endpoint when a file is downloaded. The worker logs the artifact's details, the repository, and the user responsible for the download.  
+
+## Functionality  
+
+1. **Trigger:** Activated after a download event occurs in Artifactory.  
+2. **Notification:** Sends a log containing details about the downloaded artifact to a specified external endpoint.  
+3. **Authentication:** Uses a bearer token stored in a secret to authenticate with the external endpoint.  
+4. **Error Handling:** Logs and updates the response message if the notification fails.  
+
+## Configuration  
+
+To customize the worker, you need to update two constants in the source code:  
+
+- `URL`: The endpoint where the download log will be sent.  
+  Example:  
+  ```typescript  
+  const URL = 'https://<external_endpoint>';  
+  ```  
+
+- `SECRET_NAME`: The name of the secret that contains the bearer token for authentication.  
+  Example:  
+  ```typescript  
+  const SECRET_NAME = 'myBearerToken';  
+  ```  
+
+## How It Works  
+
+1. **Download Event:** The worker is triggered when an artifact is downloaded.  
+2. **Log Creation:** The worker creates a log message with the following details:  
+    - Artifact path  
+    - Repository key  
+    - User ID or token used for the download  
+3. **Notification:** Sends the log message as a POST request to the external endpoint.  
+4. **Response Handling:**  
+    - Logs success if the notification is successfully sent.  
+    - Logs a warning or error if the notification fails and updates the response message accordingly.  
+
+## Example Log Message  
+
+```plaintext  
+The artifact '<artifact_path>' has been downloaded by the <userid/token>: <user_id> from the repository '<repo_key>'.  
+```  
+
+## Response Messages  
+
+- **Successful Notification:**  
+  - Message: `Download activity successfully logged`  
+
+- **Failed Notification:**  
+  - Message: `Failed to log download activity`  
+
+## Dependencies  
+
+This worker relies on the following JFrog Workers APIs:  
+- `PlatformContext` for accessing platform resources (e.g., secrets, HTTP clients).  
+- `AfterDownloadRequest` and `AfterDownloadResponse` for handling download event payloads and responses.  
+
+## Recommendations  
+
+1. **Endpoint Configuration:** Ensure the external endpoint is correctly set and accessible.  
+2. **Bearer Token Secret:** Store the bearer token securely as a secret in the JFrog platform.  
+3. **Monitoring:** Regularly monitor logs for failed notifications to identify issues.  
+4. **Testing:** Test the worker in a staging environment before deploying it in production.  
+
+## Error Scenarios  
+
+- **Network Errors:** Occur if the external endpoint is unreachable.  
+- **Authentication Failures:** Occur if the bearer token is invalid or missing.  
+- **Invalid URL:** Occurs if the provided endpoint URL is incorrect.  

samples/artifactory/AFTER_DOWNLOAD/report-download-to-external-endpoint/worker.ts

@@ -1,6 +1,5 @@
 import { PlatformContext, AfterDownloadRequest, AfterDownloadResponse } from 'jfrog-workers';
 
-
 export default async (context: PlatformContext, data: AfterDownloadRequest): Promise<AfterDownloadResponse> => {
     // MODIFY THOSE TWO CONSTANTS TO FIT YOUR NEEDS
     const URL = 'https://<external_endpoint>';

samples/artifactory/BEFORE_DOWNLOAD/resctrict-download-if-no-xray-scan-since/README.md

@@ -1,4 +1,71 @@
-# Block download if last Xray scan is older than 30 seconds
+# Worker for Restricting Downloads Based on Xray Scan Recency
 
-This worker will block the download if the artifact has not been scanned by Xray within the last 30 seconds and send a report to an external endpoint. For the example the expiry is quite short, but you may edit the variables EXPIRY and SECONDS to change the expiry duration.
+## Overview
 
+This worker is triggered by the `BEFORE_DOWNLOAD` event of Artifactory. Its primary purpose is to restrict downloads of artifacts that have not undergone an Xray scan within a specified threshold duration. This ensures artifacts comply with security policies by enforcing timely scans.
+
+## Functionality
+
+### Key Features
+- **Xray Scan Check:** Verifies the most recent Xray scan date of an artifact.
+- **Threshold Enforcement:** Restricts download if the last scan exceeds the specified threshold days or date.
+- **Error Handling:** Issues a warning and allows download if Xray is unavailable or errors occur during the check.
+
+### Worker Logic
+1. **Xray Availability Check:** Verifies if Xray is operational by pinging the Xray system.
+2. **Artifact Scan Details Fetching:** Retrieves artifact scan details, including the last scan date, using the artifact name and repository key.
+3. **Decision Making:**
+   - Stops the download if the last scan is older than the allowed threshold.
+   - Allows download with a warning if scan details cannot be retrieved or Xray is unavailable.
+   - Proceeds with the download if the scan is recent.
+
+## Configuration
+
+### Threshold Configuration
+- **Threshold Days:** Defines the maximum allowed age (in days) of the last Xray scan. Default: 2 days.
+- **Threshold Date:** Defines the latest allowable date for the last scan. Default: 7 days prior to the current date.
+
+## Response Types
+
+### Download Proceed
+```json
+{
+  "status": "DOWNLOAD_PROCEED",
+  "message": "Proceeding with download"
+}
+```
+- **Explanation:** Indicates the artifact complies with the scan recency requirements.
+
+### Download Stopped
+```json
+{
+  "status": "DOWNLOAD_STOP",
+  "message": "Stopping Download, because last scan date is older than allowed threshold duration of <threshold_days> days."
+}
+```
+- **Explanation:** Indicates the artifact's last scan is outdated, and the download is blocked.
+
+### Warning Response
+```json
+{
+  "status": "DOWNLOAD_WARN",
+  "message": "Could not check for Xray scans because Xray is not available. Proceeding download with warning."
+}
+```
+- **Explanation:** Indicates that the worker encountered an error (e.g., Xray unavailability), but the download is allowed with a warning.
+
+## Error Handling
+
+- **Xray Unavailability:** Issues a warning and allows download if Xray is not operational.
+- **Scan Check Failure:** Logs the error and proceeds with a warning.
+
+## Recommendations
+
+1. **Threshold Adjustment:**
+   - Update the threshold days or date as per organizational requirements.
+
+2. **Monitoring and Logging:**
+   - Regularly review logs for `DOWNLOAD_WARN` responses to address recurring issues.
+
+3. **Testing:**
+   - Validate the worker functionality in a staging environment before deploying to production.

samples/artifactory/BEFORE_DOWNLOAD/resctrict-download-if-no-xray-scan-since/worker.ts

@@ -1,30 +1,124 @@
 import { PlatformContext, BeforeDownloadRequest, BeforeDownloadResponse, DownloadStatus } from 'jfrog-workers';
 
 export default async (context: PlatformContext, data: BeforeDownloadRequest): Promise<BeforeDownloadResponse> => {
+    const thresholdDate = new Date(new Date().getUTCDate() - 7);
+    const thresholdDays = 2;
+    const isXrayAvailable = await checkIfXrayAvailable();
+    if (!isXrayAvailable) {
+        return {
+            status: DownloadStatus.DOWNLOAD_WARN,
+            message: "Could not check for xray scans because xray is not available. Proceeding download with warning.",
+            headers: {} // This can be populated if response headers are required to be added/overriden. 
+        }
+    }
 
     let status: DownloadStatus = DownloadStatus.DOWNLOAD_UNSPECIFIED;
+    let message = '';
+
+    let responseData: {
+        "data": [
+            {
+                "name": string,
+                "repo_path": string,
+                "package_id": string,
+                "version": string,
+                "sec_issues": {
+                    "critical": number,
+                    "high": number,
+                    "low": number,
+                    "medium": number,
+                    "total": number
+                },
+                "scans_status": {
+                    "overall": {
+                        "status": string,
+                        "time": string
+                    }
+                }
+                "size": string,
+                "violations": number,
+                "created": string,
+                "deployed_by": string,
+                "repo_full_path": string
+            }
+        ],
+        "offset": number
+    } = null;
 
     try {
-        // The in-browser HTTP client facilitates making calls to the JFrog REST APIs
-        //To call an external endpoint, use 'await context.clients.axios.get("https://foo.com")'
-        const res = await context.clients.platformHttp.get('/artifactory/api/v1/system/readiness');
+        const artifactName = data.metadata.repoPath.path.substr(data.metadata.repoPath.path.lastIndexOf('/') + 1);
+        const repoKey = data.metadata.repoPath.key;
+
+        const res = await context.clients.platformHttp.get(`/xray/api/v1/artifacts?repo=${repoKey}&search=${artifactName}&num_of_rows=1`);
+        responseData = res.data;
 
-        // You should reach this part if the HTTP request status is successful (HTTP Status 399 or lower)
         if (res.status === 200) {
-            status = DownloadStatus.DOWNLOAD_PROCEED;
-            console.log("Artifactory ping success");
+            // console.log(responseData);
+            console.log(responseData.data);
+            const lastScanDetails = responseData.data[0].scans_status.overall;
+            if (lastScanDetails.status !== "DONE") {
+                return {
+                    status: DownloadStatus.DOWNLOAD_WARN,
+                    message: "Scan is on-going, proceeding with download with a warning.",
+                    headers: {} // This can be populated if response headers are required to be added/overriden. 
+                }
+            } else {
+                const lastScanDate = new Date(lastScanDetails.time);
+                const currentDate = new Date();
+                console.log(differenceInDays(lastScanDate, currentDate));
+                if (thresholdDays < differenceInDays(lastScanDate, currentDate)) {
+                    return {
+                        status: DownloadStatus.DOWNLOAD_STOP,
+                        message: `Stopping Download, because last scan date is older than allowed threshold duration of ${thresholdDays} days.`,
+                        headers: {}
+                    }
+                } else if(lastScanDate<thresholdDate) {
+                    return {
+                        status: DownloadStatus.DOWNLOAD_STOP,
+                        message: `Stopping Download, because last scan date is older than allowed threshold last date of ${thresholdDate.toUTCString()}.`,
+                        headers: {}
+                    }
+                } 
+                return {
+                    status: DownloadStatus.DOWNLOAD_PROCEED,
+                    message: `Proceeding with download`,
+                    headers: {}
+                }
+            }
         } else {
             status = DownloadStatus.DOWNLOAD_WARN;
-            console.warn(`Request is successful but returned status other than 200. Status code : ${ res.status }`);
+            message = 'Request returned unexpected result. Download will proceed with warning.'
         }
-    } catch(error) {
-        // The platformHttp client throws PlatformHttpClientError if the HTTP request status is 400 or higher
-        status = DownloadStatus.DOWNLOAD_STOP;
-        console.error(`Request failed with status code ${ error.status || '<none>' } caused by : ${ error.message }`)
+    } catch (error) {
+        message = `Encountered error: ${error.message} during scan check. Download will proceed with warning.`;
+        status = DownloadStatus.DOWNLOAD_WARN;
+        console.error(`Request failed: ${error.message}`);
     }
 
     return {
         status,
-        message: 'Overwritten by worker-service if an error occurs.',
+        message,
+        headers: {} // This can be populated if response headers are required to be added/overriden.
+    }
+
+
+    async function checkIfXrayAvailable(): Promise<boolean> {
+        let response;
+        try {
+            response = await context.clients.platformHttp.get('/xray/api/v1/system/ping');
+            if (response.data.status !== "pong") {
+                throw new Error("Xray not available");
+            }
+            return true;
+        } catch (error) {
+            console.log(`Encountered error ${error.message} while checking for xray readiness. Allowing download with a warning`);
+            return false;
+        }
+
+    }
+
+    function differenceInDays(lastScanDate: Date, currentDate: Date): number {
+        const differenceInTime = currentDate.getTime() - lastScanDate.getTime();
+        return Math.round(differenceInTime / (1000 * 3600 * 24));
     }
-}
+}

samples/artifactory/BEFORE_DOWNLOAD/restrict-download-by-property-value/README.md

@@ -1,4 +1,88 @@
-# Restrict Download By Property Value
+Restrict Download by Property Value
+=====================================
 
-This worker will block the download on artifacts on which the property EXPECTED_PROPERTY_KEY is set with the value EXPECTED_PROPERTY_VALUE.
+This worker is triggered by the `BEFORE_DOWNLOAD` event of Artifactory. Its primary purpose is to block artifact downloads if the artifact contains certain forbidden properties with restricted values. This ensures compliance with organizational policies by preventing downloads of restricted artifacts.
 
+Functionality
+-------------
+- **Forbidden Properties Check:** Blocks download if an artifact contains a forbidden property with restricted values.
+- **Error Handling:** Issues a warning and allows download if the properties cannot be verified due to an error.
+- **Headers:** Optionally modifies response headers if required.
+
+Worker Logic
+------------
+1. **Property Fetching**: Queries Artifactory for artifact properties using the repository key and artifact path.
+2. **Decision Making**:
+    - Blocks the download if a forbidden property with restricted values is present.
+    - Allows the download if no forbidden properties are found.
+    - Issues a warning and allows download for errors or unexpected results.
+
+Payload
+-------
+The worker operates on the `BEFORE_DOWNLOAD` event payload provided by Artifactory. It uses metadata such as the artifact's repository path and key to fetch properties.
+
+Configuration
+-------------
+- **Forbidden Properties**: Define the list of forbidden properties and their restricted values in the worker script. Example:
+  ```typescript
+  let forbiddenProperties: ForbiddenProperty[] = [];
+  forbiddenProperties.push({
+      key: "FORBIDDEN",
+      values: ["true"]
+  });
+  ```
+
+Possible Responses
+------------------
+
+### Download Proceed
+- **Structure:**
+  ```json
+  {
+    "status": "DOWNLOAD_PROCEED",
+    "message": "Allowing Download"
+  }
+  ```
+- **Explanation:**
+  - Indicates the artifact does not contain any forbidden properties, and the download is allowed.
+
+### Download Stopped
+- **Structure:**
+  ```json
+  {
+    "status": "DOWNLOAD_STOP",
+    "message": "Stopping Download because forbiddenProperty <key> is present with forbidden values"
+  }
+  ```
+- **Explanation:**
+  - Indicates the artifact contains a forbidden property, and the download is blocked.
+
+### Warning Response
+- **Structure:**
+  ```json
+  {
+    "status": "DOWNLOAD_WARN",
+    "message": "Download proceed with a warning. Could not check if artifact is forbidden or not."
+  }
+  ```
+- **Explanation:**
+  - Indicates that the worker encountered an error (e.g., API failure), but the download is allowed with a warning.
+
+### Headers
+- The worker response can optionally include modified headers. The `headers` field in the response is currently empty but can be populated if required.
+
+Error Handling
+--------------
+- **Property Fetch Failure:** Allows download with a warning message.
+- **Unexpected Errors:** Logs errors and proceeds with a warning.
+
+Recommendations
+---------------
+1. **Forbidden Properties Definition**:
+   - Update the list of forbidden properties and their restricted values as per organizational policies.
+
+2. **Monitoring and Logging**:
+   - Review logs for `DOWNLOAD_WARN` responses to identify and address recurring issues.
+
+3. **Testing**:
+   - Validate the worker functionality in a staging environment before deploying to production.