chore: sync with upstream goharbor/harbor

CONTRIBUTING.md

@@ -261,8 +261,48 @@ go list ./... | grep -v -E 'tests' | xargs -L1 fgt golint
 
 ```
 
-Unit test cases should be added to cover the new code. Unit test framework for backend services is using [go testing](https://golang.org/doc/code.html#Testing). The UI library test framework is built based on [Jasmine](https://jasmine.github.io/) and [Karma](https://karma-runner.github.io/1.0/index.html), please refer to [Angular Testing](https://angular.io/guide/testing) for more details.
+## Recommended Make Commands
 
+Harbor provides a Makefile-driven developer workflow. Use these commands during development and testing.
+
+### Testing & Validation
+```sh
+make go_check      # Run tests, API generation, lint, vet, race, spell checks
+```
+
+### Build Specific Services
+```sh
+make compile_core        # Build the core Harbor service binary
+make compile_jobservice  # Build the jobservice binary (for background jobs)
+make compile_registryctl # Build the registryctl binary (for registry management)
+```
+
+### TLS / Cert Generation
+```sh
+make gen_tls                     # Only generate TLS certificates
+```
+
+### Cleanup & Reset
+```sh
+make cleanall       # Remove all binaries, images, and generated configs
+make cleanbinary    # Remove only compiled binaries
+make cleanimage     # Remove only built Docker images
+make cleanconfig    # Remove only generated configuration files
+```
+
+---
+
+### Running Tests
+
+Before submitting a pull request, you should ensure that your changes are well-tested.  
+Harbor uses separate testing frameworks for backend services and the web UI:
+
+- **Backend (Go) services**: Use the built-in `go testing` framework.  
+- **Web UI (Angular/Clarity)**: Use [Jasmine](https://jasmine.github.io/) and [Karma](https://karma-runner.github.io/1.0/index.html).
+
+It is recommended to run all tests locally to catch issues early before creating a PR.
+
+Unit test cases should be added to cover the new code. 
 Run go test cases:
 ```sh
 #cd #working_dir/src/[package]

Makefile

@@ -111,8 +111,8 @@ PREPARE_VERSION_NAME=versions
 
 #versions
 REGISTRYVERSION=v2.8.3-patch-redis
-TRIVYVERSION=v0.65.0
-TRIVYADAPTERVERSION=v0.34.0-rc.1
+TRIVYVERSION=v0.68.2
+TRIVYADAPTERVERSION=v0.34.2
 NODEBUILDIMAGE=node:16.18.0
 
 # version of registry for pulling the source code
@@ -407,7 +407,8 @@ build:
 	 -e TRIVY_DOWNLOAD_URL=$(TRIVY_DOWNLOAD_URL) -e TRIVY_ADAPTER_DOWNLOAD_URL=$(TRIVY_ADAPTER_DOWNLOAD_URL) \
 	 -e PULL_BASE_FROM_DOCKERHUB=$(PULL_BASE_FROM_DOCKERHUB) -e BUILD_BASE=$(BUILD_BASE) \
 	 -e REGISTRYUSER=$(REGISTRYUSER) -e REGISTRYPASSWORD=$(REGISTRYPASSWORD) \
-	 -e PUSHBASEIMAGE=$(PUSHBASEIMAGE) -e GOBUILDIMAGE=$(GOBUILDIMAGE)
+	 -e PUSHBASEIMAGE=$(PUSHBASEIMAGE) -e GOBUILDIMAGE=$(GOBUILDIMAGE) \
+	 -e RELEASEVERSION=$(RELEASEVERSION) -e GITCOMMIT=$(GITCOMMIT)
 
 build_standalone_db_migrator: compile_standalone_db_migrator
 	make -f $(MAKEFILEPATH_PHOTON)/Makefile _build_standalone_db_migrator -e BASEIMAGETAG=$(BASEIMAGETAG) -e VERSIONTAG=$(VERSIONTAG)

README.md

@@ -49,7 +49,6 @@ For learning the architecture design of Harbor, check the document [Architecture
   * Part 1: [New or changed APIs](https://editor.swagger.io/?url=https://raw.githubusercontent.com/goharbor/harbor/main/api/v2.0/swagger.yaml)
 
 ## Install & Run
-
 **System requirements:**
 
 **On a Linux host:** docker 20.10.10-ce+ and docker-compose 1.18.0+ .
@@ -59,6 +58,26 @@ Download binaries of **[Harbor release ](https://github.com/goharbor/harbor/rele
 If you want to deploy Harbor on Kubernetes, please use the **[Harbor chart](https://github.com/goharbor/harbor-helm)**.
 
 Refer to the **[documentation](https://goharbor.io/docs/)** for more details on how to use Harbor.
+### Verifying Release Signatures
+Starting with v2.15.0, Harbor release artifacts are cryptographically signed using Cosign to ensure authenticity and integrity.
+
+Download the installers and signature bundles from the Harbor releases page.
+
+#### Quick Verification
+```bash
+# Install Cosign (v2.0+)
+brew install sigstore/tap/cosign
+
+# Verify signature
+cosign verify-blob \
+  --bundle harbor-offline-installer-v2.15.0.tgz.sigstore.json \
+  --certificate-oidc-issuer https://token.actions.githubusercontent.com \
+  --certificate-identity-regexp '^https://github.com/goharbor/harbor/.github/workflows/publish_release.yml@refs/tags/v.*$' \
+  harbor-offline-installer-v2.15.0.tgz
+```
+- *Expected output:* Verified OK
+
+- *Full verification guide:* [docs/signature-verification.md](docs/signature-verification.md)
 
 ## OCI Distribution Conformance Tests
 

api/v2.0/swagger.yaml

@@ -1276,7 +1276,7 @@ paths:
           description: The JSON object of tag.
           required: true
           schema:
-            $ref: '#/definitions/Tag'
+            $ref: '#/definitions/NewTag'
       responses:
         '201':
           $ref: '#/responses/201'
@@ -6696,6 +6696,16 @@ definitions:
         type: boolean
         x-omitempty: false
         description: The immutable status of the tag
+  NewTag:
+    type: object
+    description: The request body for creating a tag
+    required:
+      - name
+    properties:
+      name:
+        type: string
+        x-nullable: false
+        description: The name of the tag
   ExtraAttrs:
     type: object
     additionalProperties:
@@ -7532,6 +7542,10 @@ definitions:
       insecure:
         type: boolean
         description: Whether or not the certificate will be verified when Harbor tries to access the server.
+      ca_certificate:
+        type: string
+        description: The PEM-encoded CA certificate for this registry endpoint. If provided, this CA will be used to verify the registry's certificate instead of the system CA pool.
+        x-nullable: true
       description:
         type: string
         description: Description of the registry.
@@ -7577,6 +7591,10 @@ definitions:
         type: boolean
         description: Whether or not the certificate will be verified when Harbor tries to access the server.
         x-nullable: true
+      ca_certificate:
+        type: string
+        description: The PEM-encoded CA certificate for this registry endpoint.
+        x-nullable: true
   RegistryPing:
     type: object
     properties:
@@ -7609,6 +7627,10 @@ definitions:
         type: boolean
         description: Whether or not the certificate will be verified when Harbor tries to access the server.
         x-nullable: true
+      ca_certificate:
+        type: string
+        description: The PEM-encoded CA certificate for this registry endpoint.
+        x-nullable: true
   RegistryInfo:
     type: object
     description: The registry info contains the base info and capability declarations of the registry

contrib/backup-restore/README.md

@@ -0,0 +1,107 @@
+# Harbor Backup and Restore Scripts (Contrib)
+
+**Warning:** These scripts are provided as-is in the `contrib/backup-restore` directory. They are not officially maintained or supported by the Harbor project. Use them at your own risk and ensure you understand their functionality before running them in a production environment.
+
+These scripts (`harbor-backup` and `harbor-restore`) are provided as a convenience for backing up and restoring your Harbor instance. They aim to back up the following components:
+
+* Harbor Database (PostgreSQL)
+* Container Registry Data
+* Chart Museum Data (if enabled)
+* Redis Data (if enabled)
+* Secret Keys
+* Harbor Configuration (`harbor.yml`)
+
+### Features
+Compared to the scripts the harbor project used to have in their repo this set of scripts is more robust in its error handling and also offers features
+for not packing the backup into a tarball. This makes it easy to rsync the whole backup directory to a secondary/standby node and restore there.
+
+rsync is used extensively by the script. by leaving the files in the backup directory between runs the downtime for backup is greatly reduced at the
+expense of disk space usage.
+
+Supports logging of status messages directly to syslog
+
+## Prerequisites
+
+* **Docker:** These scripts rely on the `docker` command-line interface to interact with Harbor's containers. Ensure Docker is installed and accessible on the machine where you run these scripts.
+* **Sufficient Permissions:** You'll need appropriate permissions (e.g., `sudo` or being in the `docker` group) to run Docker commands and perform file system operations.
+* **Stopped Harbor Instance:** You must stop your Harbor instance completely before running the `harbor-backup` or `harbor-restore` script to avoid data inconsistencies.
+
+## Usage
+
+### Backup (`harbor-backup`)
+
+1.  **Download the Scripts:** Place the `harbor-backup` script in a location accessible from your Harbor instance. Within the Harbor repository, this would typically be under `contrib/backup-restore/`.
+
+2.  **Make it Executable:**
+    ```bash
+    chmod +x harbor-backup
+    ```
+
+3.  **Run the Backup Script:**
+    ```bash
+    ./harbor-backup [OPTIONS]
+    ```
+3.  **Stop Harbor:** Ensure your Harbor instance is completely stopped before proceeding with the backup.
+
+4.  **Options:**
+    * `--docker-cmd <command>`: Specify the Docker command to use (default: `docker`).
+    * `--db-image <image>`: Specify the Harbor database image to use for the temporary backup container (default: auto-detected). It's generally recommended to let it auto-detect.
+    * `--db-path <path>`: Harbor DB data path (default: `/data/database`). Adjust if your deployment uses a different path.
+    * `--registry-path <path>`: Registry data path (default: `/data/registry`). Adjust if your deployment uses a different path.
+    * `--chart-museum-path <path>`: Chart Museum data path (default: `/data/chart_storage`). Adjust if your deployment uses a different path.
+    * `--redis-path <path>`: Redis data path (default: `/data/redis`). Adjust if your deployment uses a different path.
+    * `--secret-path <path>`: Secret data path (default: `/data/secret`). Adjust if your deployment uses a different path.
+    * `--config-path <path>`: Harbor configuration file path (default: `/etc/harbor/harbor.yml`). Adjust if your deployment uses a different path.
+    * `--backup-dir <path>`: Directory where the backup will be stored (default: `harbor_backup`).
+    * `--no-archive`: Do not create a `tar.gz` archive of the backup directory. The backup will remain as a directory structure in `$BACKUP_DIR/harbor`.
+    * `--use-syslog`: Use syslog for logging output.
+    * `--log-level <level>`: Set the logging level (default: `INFO`, options: `DEBUG`, `INFO`, `NOTICE`, `WARNING`, `ERROR`, `CRITICAL`, `ALERT`, `EMERGENCY`).
+    * `--help`: Display this help message.
+
+5.  **Backup Location:** By default, the backup will be created in a directory named `harbor_backup` in the current working directory. If the `--no-archive` option is not used, the final backup will be a compressed tarball named `harbor_backup.tar.gz` within the `harbor_backup` directory.
+
+### Restore (`harbor-restore`)
+
+1.  **Download the Scripts:** Place the `harbor-restore` script in a location accessible from your Harbor instance. Within the Harbor repository, this would typically be under `contrib/backup-restore/`.
+
+2.  **Make it Executable:**
+    ```bash
+    chmod +x harbor-restore
+    ```
+
+3.  **Stop Harbor:** Ensure your Harbor instance is completely stopped before proceeding with the restore.
+
+4.  **Run the Restore Script:**
+    ```bash
+    ./harbor-restore [OPTIONS]
+    ```
+
+5.  **Options:** The restore script accepts similar options to the backup script, allowing you to specify the Docker command, database image, data paths, and the backup directory.
+
+    * `--backup-dir <path>`: **Crucially**, this should point to the directory containing your Harbor backup (either the `harbor` subdirectory extracted from the tarball or the `harbor_backup` directory if `--no-archive` was used).
+    * `--no-archive`: Use this option if your backup is already extracted into the `$BACKUP_DIR/harbor` directory. If your backup is a `tar.gz` file, **do not** use this option; the script will attempt to extract it.
+
+    *(Other options like `--docker-cmd`, `--db-image`, `--db-path`, `--registry-path`, `--chart-museum-path`, `--redis-path`, `--secret-path`, `--config-path`, `--use-syslog`, and `--log-level` function similarly to the backup script.)*
+
+6.  **Restore Process:** The script will:
+    * Start a temporary database container.
+    * Extract the backup archive (if not using `--no-archive`).
+    * Drop and recreate existing Harbor databases.
+    * Restore the database content from the backed-up SQL files.
+    * Synchronize the registry, chart museum, Redis, and secret data directories.
+    * Restore the Harbor configuration file.
+    * Clean up the temporary database container.
+
+7.  **Restart Harbor:** Once the restore script completes successfully, you can restart your Harbor instance.
+
+## Important Notes
+
+* **Backup Consistency:** For a consistent backup, it's recommended to stop your Harbor instance or at least ensure minimal write activity during the backup process.
+* **Database Image Tag:** In production environments, it's advisable to use a specific tag for the `--db-image` option in both the backup and restore scripts to ensure consistency.
+* **Custom Deployments:** If you have a highly customized Harbor deployment with data stored in non-default locations, you **must** use the appropriate command-line options to point the scripts to the correct paths.
+* **Testing:** Always test the backup and restore process in a non-production environment before relying on it for critical data.
+* **Unsupported:** Remember that these scripts are provided in the `contrib/backup-restore/` directory. They may not be actively maintained, and you might encounter issues. Contributions and improvements from the community are welcome.
+
+## Contributing
+
+If you find issues or have improvements to these scripts, feel free to submit pull requests to the Harbor project in the `contrib/backup-restore/` directory.