✨ Support remote URLs in core.sqlite_zip storage

[giovannipizzi]

The filepath of the core.sqlite_zip storage backend now also accepts an http:// or https:// URL of an archive hosted online, e.g.:

verdi profile setup core.sqlite_zip -n --profile-name archive \
    --filepath https://example.com/export.aiida

The archive is never downloaded in full. Using the remotezip library (added as a new dependency), only the SQLite database is fetched via HTTP range requests (to a temporary file, once per Python session, on first query), and repository files are streamed individually from the remote zip on demand. Each storage instance opens a single connection to the remote archive, reused for validation, database extraction and repository access. The server hosting the archive must support range requests, and the timeout of the HTTP requests can be changed via the new storage.remote_archive_timeout configuration option (default: 60 seconds).

Since a remote archive cannot be migrated in place, profile setup fails with a clear error if the archive is not at the target schema version, instructing the user to download the file and run verdi archive migrate on the local copy. Similarly, resetting a remote archive raises, and verdi profile delete --delete-data reports that the remote archive is left untouched instead of warning that the file does not exist.

The -p/--profile option of verdi now also accepts the location of a .aiida archive, as a file:///absolute/path URL of a local archive or an http(s):// URL of a remote one, e.g.:

verdi -p file:///path/to/export.aiida process list -a
verdi -p https://example.com/export.aiida shell

In this case an ephemeral profile is created that mounts the archive with the read-only core.sqlite_zip storage backend, for the duration of the command only: the profile is never added to the configuration file and any temporary files are cleaned up when the command finishes. Archive locations are only accepted by parameters that explicitly opt in via ProfileParamType(accept_archive_location=True), currently only the top-level -p/--profile option: other profile parameters, such as the argument of verdi profile delete, keep treating such values as invalid profile names. As part of this,
aiida.manage.configuration.load_profile now also accepts a Profile instance in addition to a profile name.

Also fix a latent bug in FolderBackendRepository.open, which passed encoding to a binary-mode open() call, raising ValueError for any file access on unpacked folder-format archives.

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 9e4958cf-afee-4b37-96b0-2888f112486f

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[codecov]

Codecov Report

❌ Patch coverage is 82.25108% with 41 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.50%. Comparing base (a5b2bc4) to head (6e647f2).
⚠️ Report is 4 commits behind head on main.

Files with missing lines	Patch %	Lines
src/aiida/storage/sqlite_zip/backend.py	84.85%	20 Missing ⚠️
src/aiida/storage/sqlite_zip/utils.py	76.37%	13 Missing ⚠️
src/aiida/storage/sqlite_zip/migrator.py	44.45%	5 Missing ⚠️
src/aiida/cmdline/params/types/profile.py	92.86%	2 Missing ⚠️
src/aiida/manage/configuration/config.py	75.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #7432      +/-   ##
==========================================
+ Coverage   80.44%   80.50%   +0.07%     
==========================================
  Files         578      579       +1     
  Lines       46253    46523     +270     
==========================================
+ Hits        37204    37451     +247     
- Misses       9049     9072      +23     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[giovannipizzi]

Note: Developed with support of Claude Fable 5.

Potential design discussion items:

1. remotezip as a hard dependency. It is a small third-party package (single maintainer,
   last release Feb 2024, outside the aiidateam org).
   Easy to keep as dependency, but could also become an extra
   (e.g. aiida-core[remote]) with a lazy, actionable ImportError.

2. UX of overloading -p/--profile. -p now means "profile name or archive location".
   Seems to me the simplest option. An alternative possible: a dedicated
   top-level option (e.g. verdi --archive <loc>), or accepting bare local paths
   (verdi -p ./export.aiida) without the file:// scheme (which helps disambiguate filenames
   from profile names, but is unusual for a CLI). Another option: bare paths could be accepted when
   the value contains a / or ends in .aiida (to double check if profile labels cannot contain /;
   probably they shouldn't as they are used also as folder names in .aiida, etc.). Related:
   shell completion completes only profile names, not paths. This should be probably updated.

3. Performance expectations for remote repository access. Every repository file open is a
   separate HTTP range request through a single kept-open connection; commands that traverse many
   repository files (verdi archive-style dumps, provenance exports) might/will be very slow.
   To do @GeigerJ2: test some verdi process dump commands directly from a remote file,
   on a workflow with many subworkflows. I would compare with the time to download the .aiida
   and run still an ephemeral profile, but from disk.