Update API spec and format with Ruff (#79)

* Do all formatting and linting with Ruff

* Rename "audit log rotation" to "purge"

* Import custom root models in `parser.py`

* Format fragments with Ruff manually

* Update API spec

* Remove unused `type: ignore` comments

* Run linting and formatting on all files

* Add Python 3.12 support

* Add `SBOMOverview` to changelog

* Remove all unnecessary "# type: ignore" comments

* Format and lint tests

Author: pederhan

.github/workflows/test.yml

@@ -30,7 +30,7 @@ jobs:
       fail-fast: false
       matrix:
         os: [ubuntu-latest]
-        python-version: ["3.8", "3.9", "3.10", "3.11"]
+        python-version: ["3.8", "3.9", "3.10", "3.11", "3.12"]
 
     steps:
       - uses: actions/checkout@v3

.pre-commit-config.yaml

@@ -8,16 +8,13 @@ repos:
       - id: end-of-file-fixer
       - id: check-yaml
       - id: check-added-large-files
-  - repo: https://github.com/asottile/reorder_python_imports
-    rev: v3.12.0
-    hooks:
-      - id: reorder-python-imports
-        args: [--py38-plus, --add-import, "from __future__ import annotations"]
-  - repo: https://github.com/charliermarsh/ruff-pre-commit
-    rev: "v0.1.7"
+  - repo: https://github.com/astral-sh/ruff-pre-commit
+    # Ruff version.
+    rev: v0.4.8
     hooks:
       # Run the linter.
       - id: ruff
-        args: [--fix]
+        args: [ --fix ]
+        exclude: '^codegen/ast/fragments/.*'
       # Run the formatter.
       - id: ruff-format

CHANGELOG.md

@@ -12,7 +12,21 @@ While the project is still on major version 0, breaking changes may be introduce
 
 <!-- changelog follows -->
 
-<!-- ## Unreleased -->
+## Unreleased
+
+### Added
+
+- Python 3.12 support.
+- `harborapi.models.SBOMOverview` which represents the SBOM overview for an artifact. Can be accessed via `Artifact.sbom_overview`.
+
+### Changed
+
+- Added compatibility methods for purge endpoint methods that were renamed in HarborAPI v0.11.0.
+  - The `*audit_log_rotation*` methods are now deprecated and will be removed in a future release.
+  - It never made sense to create an opinionated naming for these methods.
+- Project is now formatted and linted with Ruff.
+- Code generation now explicitly imports `StrDictRootModel` and `StrRootModel`.
+- Updated API spec to [ec8d692](https://github.com/goharbor/harbor/blob/6a38ed3d7769e3598c6cf829aae4e0e152f93a83/api/v2.0/swagger.yaml)
 
 ## [0.23.4](https://github.com/unioslo/harborapi/tree/harborapi-v0.23.4) - 2024-03-01
 
@@ -52,6 +66,7 @@ While the project is still on major version 0, breaking changes may be introduce
 The big Pydantic V2 update. This is a major update in terms of both scope and API compatibility.
 
 ### Changed
+
 - Model validation has become more strict.
   - Most importantly, `str` fields will no longer coerce `int` values to strings.
   - See [Pydantic docs](https://docs.pydantic.dev/latest/api/standard_library_types/) for more information.
@@ -111,10 +126,8 @@ The big Pydantic V2 update. This is a major update in terms of both scope and AP
 - Pydantic version capped at <2.0.0.
   - Migration to Pydantic V2 will begin soon.
 
-
 ## [0.21.0](https://github.com/unioslo/harborapi/tree/harborapi-v0.21.0) - 2023-06-08
 
-
 ### Added
 
 - Missing optional `limit` argument for methods that fetch multiple resources:
@@ -139,12 +152,10 @@ The big Pydantic V2 update. This is a major update in terms of both scope and AP
 
 ## [0.20.0](https://github.com/unioslo/harborapi/tree/harborapi-v0.20.0) - 2023-05-30
 
-
 ### Changed
 
 - `HarborAsyncClient.delete_scanner` now raises `HarborAPIException` if no scanner response is returned from the API (was `UnprocessableEntity` before).
 
-
 ### Removed
 
 - Loguru dependency. The library now uses the standard Python logging library for logging purposes. See [Logging](https://unioslo.github.io/harborapi/usage/logging/) in the docs for more information.
@@ -175,15 +186,12 @@ The big Pydantic V2 update. This is a major update in terms of both scope and AP
   - Remove model:`SearchResult` (deprecated)
   - Remove field `Project.chart_count`(deprecated)
 
-
 ## [0.18.1](https://github.com/unioslo/harborapi/tree/harborapi-v0.18.1) - 2023-05-12
 
-
 ### Added
 
 - `BaseModel.convert_to`, which allows converting a model to another model type that has a similar schema, such as `Project` to `ProjectReq`.
 
-
 ### Fixed
 
 - Bool to string converter on `BaseModel` now correctly handles assignments.
@@ -192,7 +200,6 @@ The big Pydantic V2 update. This is a major update in terms of both scope and AP
 
 ### Added
 
-
 - New context manager for temporarily disabling retrying: `HarborAsyncClient.no_retry`
 - New context manager for temporarily disabling validation: `HarborAsyncClient.no_validation`
 - New context manager for temporarily enabling raw mode: `HarborAsyncClient.raw_mode`
@@ -227,7 +234,6 @@ The big Pydantic V2 update. This is a major update in terms of both scope and AP
 
 - Disclaimer for `harborapi.HarborAsyncClient.delete_retention_policy`, warning that it can break a project due to a bug in Harbor.
 
-
 ### Changed
 
 - Move `harborapi.client.ResponseLog` and `harborapi.client.ResponseLogEntry` to `harborapi.responselog` module.
@@ -238,7 +244,6 @@ The big Pydantic V2 update. This is a major update in terms of both scope and AP
 - `harborapi.ext.regex.match` return type annotation is now correctly marked as `Optional[Match[str]]` instead of `Match[str]`.
 - `harborapi.HarborAsyncClient.get_retention_tasks` missing `limit` parameter in docstring.
 
-
 ## [0.16.1](https://github.com/unioslo/harborapi/tree/harborapi-v0.16.1) - 2023-04-24
 
 ### Fixed
@@ -267,7 +272,6 @@ Until the official API spec is fixed, this is the best we can do.
 
 - `harbor` being added as an executable script installed by the project. This was a mistake, as the `harbor` executable script is intended to be exposed by [harbor-cli](https://github.com/unioslo/harbor-cli).
 
-
 ## [0.15.1](https://github.com/unioslo/harborapi/tree/harborapi-v0.15.1) - 2023-04-17
 
 ### Added
@@ -279,9 +283,8 @@ Until the official API spec is fixed, this is the best we can do.
 - `HarborAsyncClient.get_artifact_vulnerabilities()` now always returns a `harborapi.models.HarborVulnerabilityReport` object. If the artifact has no vulnerabilities or the report cannot be processed, an exception is raised.
 
 ### Removed
-- `config` argument from `HarborAsyncClient.__init__()`. The `config` argument was never implemented.
-
 
+- `config` argument from `HarborAsyncClient.__init__()`. The `config` argument was never implemented.
 
 ## [0.15.0](https://github.com/unioslo/harborapi/tree/harborapi-v0.15.0) - 2023-04-13
 
@@ -330,7 +333,6 @@ Until the official API spec is fixed, this is the best we can do.
 
 - `HarborAsyncClient.update_project_member_role()` now accepts integer arguments for its `role_id` parameter, since `RoleRequest` only has a single field (`role_id`).
 
-
 ### Fixed
 
 - Potential bug with `models.VulnerabilitySummary` if `summary` is `None`.
@@ -366,7 +368,6 @@ Until the official API spec is fixed, this is the best we can do.
 - `HarborAsyncClient.get_system_certificate()`
   - Returns the system certificate. (`GET /api/v2.0/systeminfo/getcert`)
 
-
 ### Changed
 
 - **BREAKING**: Methods that download files, now return `FileResponse` instead of a bytes object. `FileResponse` contains the file contents along with its metadata. The object can be passed to `bytes()` to get the response contents, otherwise it can be accessed via the `FileResponse.content` attribute.
@@ -435,12 +436,10 @@ Until the official API spec is fixed, this is the best we can do.
 - DEPRECATED: Using `credentials` as a parameter for `HarborAsyncClient.__init__` is deprecated. Use `basicauth` instead.
 - `HarborAsyncClient.credentials` is now a Pydantic SecretStr, which prevents it from being printed in clear text when locals are dumped, such as when printing the client object. To access the value, use `HarborAsyncClient.credentials.get_secret_value()`.
 
-
 ### Removed
 
 - Explicit logging calls from `HarborAsyncClient.set_user_cli_secret()` and `HarborAsyncClient.set_user_password()`. The exception handler handles logging if configured.
 
-
 ## [0.9.0](https://github.com/unioslo/harborapi/tree/harborapi-v0.9.0) - 2023-02-21
 
 ### Changed
@@ -451,19 +450,16 @@ Until the official API spec is fixed, this is the best we can do.
 
 ## [0.8.6](https://github.com/unioslo/harborapi/tree/harborapi-v0.8.6) - 2023-02-20
 
-
 ### Fixed
 
 - Models with `harborapi.models.ScheduleObj` fields are now correctly validated when the Harbor API responds with a value of `"Schedule"` for the field `ScheduleObj.type`, which is not a valid value for the enum according to their own spec.
 
-
 ## [0.8.5](https://github.com/unioslo/harborapi/tree/harborapi-v0.8.5) - 2023-02-20
 
 ### Added
 
 `NativeReportSummary.severity_enum` which returns the severity of the report as a `harborarpi.scanner.Severity` enum, which can be used for comparisons between reports.
 
-
 ### Fixed
 
 `harborarpi.scanner.Severity` enum not having a `None` value, which is observed when a report has no vulnerabilities.
@@ -475,7 +471,6 @@ Until the official API spec is fixed, this is the best we can do.
 - Certain resource enumeration methods missing the `limit` parameter.
 - `HarborAsyncClient.get_gc_jobs()` ignoring user parameters.
 
-
 ## [0.8.3](https://github.com/unioslo/harborapi/tree/harborapi-v0.8.3) - 2023-02-14
 
 ### Changed
@@ -493,7 +488,6 @@ Until the official API spec is fixed, this is the best we can do.
 
 - `HarborAsyncClient.get_registry_providers` now returns a `RegistryProviders` object, which is a model whose only attribute `providers` is a dict of `RegistryProviderInfo` objects. Previously this method attempted to return a list of `RegistryProviderInfo` objects, but this was incorrect.
 
-
 ## [0.8.1](https://github.com/unioslo/harborapi/tree/harborapi-v0.8.1) - 2023-02-09
 
 ### Changed
@@ -506,7 +500,6 @@ Until the official API spec is fixed, this is the best we can do.
 
 - `limit` parameter for all methods that return a list of items. This parameter is used to limit the number of items returned by the API. See the [docs](https://unioslo.github.io/harborapi/usage/limit/) for more details.
 
-
 ### Removed
 
 - `retrieve_all` parameter for all methods that return a list of items. Use the new `limit` parameter to control the number of results to retrieve. Passing `retrieve_all` to these methods will be silently ignored. In the future this will raise a DeprecationWarning.
@@ -517,7 +510,6 @@ Until the official API spec is fixed, this is the best we can do.
 
 - New parameters `raw` and `validate` to `HarborAsyncClient` and `HarborClient` to control whether the client returns the raw data from the API, and whether the client validates the data from the API, respectively. See the [docs](https://unioslo.github.io/harborapi/usage/validation/) for more details.
 
-
 ## [0.7.0](https://github.com/unioslo/harborapi/tree/harborapi-v0.7.0) - 2023-02-06
 
 ### Added
@@ -533,7 +525,6 @@ Until the official API spec is fixed, this is the best we can do.
 
 - `HarborAsyncClient.search()` raising an error when finding Helm Charts with an empty `engine` field.
 
-
 ### Removed
 
 - **BREAKING**: `HarborAsyncClient.get_internal_config()`. This endpoint is meant for internal usage only, and the new model definitions don't seem to play well with it. If you need this endpoint, please open an issue.

codegen/ast/fragments/main/_common.py

@@ -4,7 +4,6 @@
 from typing import Dict
 from typing import Optional
 
-
 # Changed: change params field type
 # Reason: params is a dict of Any, not a dict of dicts
 

codegen/ast/fragments/main/immutablerule.py

@@ -6,8 +6,8 @@
 from typing import Union
 
 from pydantic import Field
-from pydantic import field_validator
 from pydantic import ValidationInfo
+from pydantic import field_validator
 
 
 class ProjectMetadata(BaseModel):

codegen/ast/fragments/main/projectmetadata.py

@@ -4,7 +4,6 @@
 from typing import Dict
 from typing import Optional
 
-
 # Changed: change params field type
 # Reason: params is a dict of Any, not a dict of dicts
 # TODO: add descriptions

codegen/ast/fragments/main/retentionrule.py

@@ -1,7 +1,7 @@
 from __future__ import annotations
 
-from ..version import get_semver
 from ..version import SemVer
+from ..version import get_semver
 
 
 class Scanner(BaseModel):

codegen/ast/fragments/scanner/_common.py

@@ -1,16 +1,15 @@
 from __future__ import annotations
 
-from typing import Dict
 from typing import Iterable
 from typing import Optional
 from typing import Union
 
-from pydantic import field_validator
 from pydantic import ValidationInfo
+from pydantic import field_validator
 
 from ..log import logger
-from ..version import get_semver
 from ..version import SemVer
+from ..version import get_semver
 
 
 class VulnerabilityItem(BaseModel):

codegen/ast/fragments/scanner/scanner.py

@@ -321,7 +321,9 @@ def modify_module(tree: ast.Module, fragment_dir: FragmentDir) -> ast.Module:
 
 
 # Imports that should be added to every file
-ADD_IMPORTS = {}  # type: dict[str, list[str]] # module: list[import_name]
+ADD_IMPORTS = {
+    "harborapi.models.base": ["StrDictRootModel", "StrRootModel"],
+}  # type: dict[str, list[str]] # module: list[import_name]
 
 
 def add_imports(tree: ast.Module) -> ast.Module:
@@ -498,11 +500,11 @@ def insert_or_update_classdefs(
                     ) and getattr(
                         stmt.target,
                         "id",
-                        None,  # type: ignore
+                        None,
                     ) == getattr(
                         class_stmt.target,
                         "id",
-                        None,  # type: ignore
+                        None,
                     ):
                         node.body[i] = class_stmt
                         break

codegen/ast/fragments/scanner/vulnerabilityitem.py

@@ -51,7 +51,6 @@ if [ "$fetch_spec" = true ]; then
 fi
 
 python codegen/ast/parser.py "$output_file" "$final_file" "$source_type"
-reorder-python-imports "$final_file"
-ruff --fix "$final_file"
+ruff check --fix "$final_file"
 ruff format "$final_file"
 cp "$final_file" "./harborapi/models/$(basename "$final_file")"

codegen/ast/parser.py

@@ -4,10 +4,3 @@
     options:
         members:
         - get_audit_logs
-        - get_audit_log_rotation
-        - stop_audit_log_rotation
-        - get_audit_log_rotation_log
-        - get_audit_log_rotation_history
-        - get_audit_log_rotation_schedule
-        - create_audit_log_rotation_schedule
-        - update_audit_log_rotation_schedule

codegen/generate.sh

@@ -0,0 +1,12 @@
+# Purge
+
+::: harborapi.client.HarborAsyncClient
+    options:
+        members:
+        - get_purge_job
+        - stop_purge_job
+        - get_purge_job_log
+        - get_purge_job_schedule
+        - create_purge_job_schedule
+        - update_purge_job_schedule
+        - get_purge_job_history