Skip to content

fix(hub): platform-aware artifact selection for binary hub agents#2086

Merged
itomek-amd merged 5 commits into
mainfrom
fix/2084-hub-email-artifact-selection
Jul 15, 2026
Merged

fix(hub): platform-aware artifact selection for binary hub agents#2086
itomek-amd merged 5 commits into
mainfrom
fix/2084-hub-email-artifact-selection

Conversation

@itomek

@itomek itomek commented Jul 15, 2026

Copy link
Copy Markdown
Collaborator

Closes #2084

Installing the Email Triage agent from the Agent UI hub failed on every platform: the hub manifest's legacy singular artifact field always points at the first-published platform binary (email-agent-darwin-arm64), the installer read only that field, and language: python routed the macOS executable into uv pip install — the exit-2 error in #2084. The installer now selects the right entry from versions[v].artifacts[] for the host platform and installs a binary as a binary (checksum-verified, atomic, chmod +x), so the only agent in the live hub catalog actually installs — verified end-to-end on a Windows 11 + Radeon machine (the reporter's environment class) and on macOS.

Also in the change, because the email install dir doubles as the running sidecar's own binary cache: the UI shuts a running sidecar down before install/uninstall/rollback, a Windows locked-file failure becomes an actionable error instead of a stack trace, updates replace the binary in place (no dir-level backup that would yank a live process's directory), and health/status report a correct binary install as healthy instead of demanding the gaia.agent entry point binaries don't ship. Wheel- and C++-agent behavior is byte-identical (regression-tested). Deferred follow-ups (binary-agent trust gate, wheel-publishing hazards, small install hardening) are tracked in #2085 with the full analysis.

Test plan

  • python -m pytest tests/unit/test_hub_installer.py tests/unit/test_hub_router.py tests/unit/test_hub_lifecycle.py tests/unit/test_hub_compatibility.py -q121 passed (includes the new T1–T14 contract: platform selection, wheel/cpp regression guards, loud no-match errors, sentinel back-compat, locked-file/update paths, sidecar-shutdown hook, kind-aware health, platform-key parity with the sidecar's own mapping)
  • Regression net: tests/unit/test_hub_catalog.py tests/unit/test_agent_hub_api.py → 21 passed, 4 skipped
  • python util/lint.py --all → ALL QUALITY CHECKS PASSED
  • Real-world before→after on Windows 11 + Radeon and on macOS (evidence below)

Evidence (real hub, cold state — ~/.gaia/agents/email deleted before each phase)

Before — main (a02ae714), Windows 11: POST /api/agents/install {"id":"email"} → install-status:

{"id":"email","status":"failed","error":"'uv pip install' failed (exit 2): error: Failed to parse: `...\\gaia-hub-k1pdt_rr\\email-agent-darwin-arm64` ... Expected path to end in a supported file extension: `.whl`, ..."}

After — this branch (924c8449), same box, cold: same request →

{"id":"email","status":"completed","phase":"completed","percent":100,"version":"0.4.0","error":null}

email-agent.exe SHA-256 = 206a9c12…93f8 — exact match with the live manifest's win32-x64 artifact; GET /api/agents/email/health{"state":"healthy"}.

🔍 More evidence: locked-file error, update-in-place, macOS

Windows, install while the agent binary is running (holds the exe open) — actionable error instead of a stack trace, and no .backup/ dir is created:

{"status":"failed","error":"Could not write email-agent.exe to C:\\Users\\...\\agents\\email: [WinError 5] Access is denied ... The agent appears to be running — close it and retry."}

After closing the process, the same request completes and replaces the binary in place (SHA re-verified).

macOS (darwin-arm64), cold: same install → completed, ~/.gaia/agents/email/email-agent written with -rwx--x--x, SHA-256 = 26bcf05b…bf (exact manifest match), health healthy, ./email-agent --help runs.

Tested by driving the real POST /api/agents/install / install-status / health endpoints of gaia.ui.server against the live hub on both machines.

Tomasz Iniewicz added 5 commits July 14, 2026 19:57
…alls (#2084)

The hub installer only reads the manifest's legacy singular artifact field
(always the first-published macOS binary), so Windows/Linux hosts get fed a
macOS executable into uv pip install. These tests pin the fix: platform-aware
selection from versions[v].artifacts[], no wheel fallback for binary agents,
and health/rollback/uninstall/router handling for the new binary artifact kind.

All new tests fail against current code (missing install(platform_key=...),
compatibility.current_platform_key(), and the sentinel artifact_kind field),
confirming they exercise the not-yet-implemented fix rather than passing
vacuously.
The installer only read the manifest's legacy singular artifact field,
which is always the first-published macOS binary — so Windows and
Linux hosts got fed a macOS executable into uv pip install and the
install failed every time (#2084).

installer.py now selects the matching entry from versions[v].artifacts[]
by platform key (win32-x64/darwin-arm64/darwin-x64/linux-x64), writes it
as a binary rather than routing it through pip, and never falls back to
a wheel even if one happens to be present. compatibility.py gains
current_platform_key() to resolve the host's key, mirroring the email
sidecar's own mapping so both agree on the platform namespace.
…ts[] cases (#2084)

The hub worker writes artifacts: [artifact] for every publish, so a modern
wheel-only agent carries a single-element .whl artifacts[]. Presence of
artifacts[] must not imply a binary install: a wheel/sdist-only list routes
through pip regardless of platform_key, and a cpp manifest (classified first)
extracts its singular zip exactly like the legacy route. Both new tests are
red against the current increment, which over-eagerly treats any non-empty
artifacts[] as platform-keyed binary selection.
The hub worker writes artifacts: [artifact] on every publish, so a
modern wheel-only agent carries an artifacts[] list too — treating the
list's mere presence as "binary agent" would loudly fail every wheel
install the current worker publishes. Classify the set instead: cpp
first (byte-identical legacy route), then platform-match among the
binary-like entries only, and a wheel/sdist-only list takes the pip
route regardless of platform.

Also generalizes the .exe suffix match to any platform key (win32-arm64
binaries are planned, #1898) and names the version in the no-match
error.
… mutating its dir

A correct binary install of the email agent would read "broken" forever:
health/status demanded a gaia.agent entry point, which binary agents
don't ship by design. health_check now probes binary-kind installs by
executable presence (+x on POSIX) via the executable name the sentinel
records, leaving wheel agents' entry-point probe untouched.

The email agent's install dir doubles as the sidecar's own binary
cache, and the warm manager holds the exe open — so install/uninstall/
rollback of email now shut a running sidecar down first (router-level
hook; gaia.hub still never imports gaia.ui). Absent or stopped manager
is a no-op.
@github-actions github-actions Bot added the tests Test changes label Jul 15, 2026
@itomek itomek self-assigned this Jul 15, 2026
@itomek
itomek marked this pull request as ready for review July 15, 2026 00:29
@itomek
itomek requested a review from kovtcharov-amd as a code owner July 15, 2026 00:29
@itomek
itomek enabled auto-merge July 15, 2026 00:30
@github-actions

Copy link
Copy Markdown
Contributor

Verdict: Approve — clean, well-tested, scope-tight.

This fixes the Email Triage agent failing to install from the hub on every platform: the installer read only the manifest's legacy singular artifact (always the macOS binary) and fed it into uv pip install. It now classifies the artifact set, selects the right per-platform binary, and installs a binary as a binary (checksum-verified, atomic write, chmod +x), while leaving wheel and C++ install paths byte-identical. The three failure modes that fall out of treating an install dir as a live sidecar's cache — locked-file writes, dir-level backups, and health probes demanding an entry point binaries don't ship — are each handled and turned into actionable errors. No blocking issues found; the fail-loudly discipline (loud no-match error, no silent wheel substitution) is exactly right.

🔍 Technical details

Strengths

  • Correctness-first classification order (installer.py:253-288): cpp classified first, then artifacts[] set membership, then legacy singular — and critically it classifies the set, not artifacts[] presence, so the worker's habit of writing artifacts: [artifact] on every publish doesn't misfire binaries. The wheel-only-artifacts[] regression (T4b) and cpp-with-artifacts[] case (T4c) both have dedicated guards.
  • No silent fallback: a binary manifest with no platform match raises a loud InstallError naming the platform and every available filename (installer.py:257-264), never substituting a wheel from elsewhere in the list. Matches CLAUDE.md's fail-loudly rule.
  • Path-traversal guard on the artifact filename before any path-join (_sanitize_artifact_filename, installer.py:188-195), covered for both singular and artifacts[] shapes (T9).
  • Cold-state + real-flow testing: tests drive the real installer.install() and the real router endpoints rather than stubbing the selection, and the sidecar/platform-key parity is asserted against gaia.ui.email_sidecar.platform directly (T10) — so the duplicated current_platform_key can't silently drift. This is the "test the call is valid, from the user's real state" bar CLAUDE.md asks for.
  • Layer boundary respected: gaia.hub never imports gaia.ui; the sidecar-shutdown bridge lives only in the router (hub.py:541-555), and the shutdown fires synchronously before the background install task is queued (hub.py:190-203) — correct ordering, no race with the worker.

Minor observations (non-blocking, no change requested)

  • 🟢 _shutdown_email_sidecar hardcodes agent_id != "email" (hub.py:549). Fine while email is the only sidecar agent and it's clearly commented, but worth a registry-driven "is this a sidecar agent" check once a second one lands, so this doesn't become a spot to forget.
  • 🟢 On the wheel-only artifacts[] branch (installer.py:267-275), wheels[0] after the sort will be an sdist if no .whl is present. That's a valid pip target, so behavior is correct — just noting it's intentional, not a .whl-guaranteed path.

No doc-sync obligation here: this is core installer internals, not a hub-agent package doc surface, and no docs/ page documents artifact selection.

@itomek
itomek disabled auto-merge July 15, 2026 00:55
@itomek

itomek commented Jul 15, 2026

Copy link
Copy Markdown
Collaborator Author

Linux evidence added — the fix is now verified on all three OS targets, plus the earlier Windows CI failure is resolved.

Ubuntu 24.04 (linux-x64), cold state — fresh ubuntu:24.04 container (linux/amd64), branch cloned at 924c8449, pip install -e ".[ui]", no prior ~/.gaia; drove the real endpoints:

POST /api/agents/install {"id":"email"}  →  {"id":"email","status":"queued"}
install-status → {"id":"email","status":"completed","phase":"completed","percent":100,"version":"0.4.0","error":null}
-rwx--x--x 1 root root 68354952 /root/.gaia/agents/email/email-agent
sha256: 7015be1ea212fddf967cbb631fe4a848dedda8d7d51ad38b9a3fdc67ac256973   (exact match: manifest linux-x64 artifact)
GET /api/agents/email/health → {"id":"email","state":"healthy",...}
./email-agent --help → usage: email-agent [-h] [--host HOST] [--port PORT] [--print-openapi]

Ran in a container because both physical Linux test boxes were unavailable at the time (one reserved by a long eval run, the other is the dual-boot machine currently booted into Windows); the install path is hardware-independent and the installed binary executes.

Per-OS summary: Windows 11 (physical, Radeon — the reporter's environment class): before→after + locked-file + update-in-place, SHA-verified ✅ · macOS (darwin-arm64, physical): SHA + exec bit + health ✅ · Ubuntu 24.04 (linux-x64, container): SHA + exec bit + health ✅.

CI: the "Test GAIA CLI on Windows (Full Integration)" failure was a runner loss (job killed at 11m mid-step, log blob never uploaded; workflow green on the last 5 main commits) — rerun passed, all checks green.

@itomek-amd
itomek-amd added this pull request to the merge queue Jul 15, 2026
Merged via the queue into main with commit 2c791e7 Jul 15, 2026
49 of 50 checks passed
@itomek-amd
itomek-amd deleted the fix/2084-hub-email-artifact-selection branch July 15, 2026 13:27
pull Bot pushed a commit to bhardwajRahul/gaia that referenced this pull request Jul 16, 2026
…holder lock (amd#2100)

Installing the email agent from the Agent Hub currently produces a
binary the UI refuses to run: amd#2086 puts a checksum-verified
`email-agent` exactly where `EmailSidecarManager` looks, but the
user-mode spawn path gates on `binaries.lock.json`'s placeholder SHAs
*before* checking what's installed, so the first email chat dies with a
"placeholder sha256" error. Now `fetch_binary` short-circuits to the
Hub-installed binary first — re-verified against the SHA-256 its install
sentinel recorded from the hub manifest — so Hub install → email chat
just works in user mode, no env overrides. With no installed binary, the
placeholder lock still fails loudly naming the lock, and a sentinel/file
mismatch raises `IntegrityError` with a reinstall remedy instead of
silently falling back.

Fixes amd#2095

## Test plan

- [ ] `python -m pytest tests/unit/test_email_sidecar_fetch.py
tests/unit/test_email_sidecar_manager.py
tests/unit/test_hub_installer.py -q` — includes new tests pinning the
ordering (installed-binary short-circuit before the lock is consulted),
the loud placeholder failure without an install, the tampered-install
`IntegrityError`, and the manager-level spawn with the placeholder lock
- [ ] `python util/lint.py --all`
- [ ] Manual golden path: fresh `~/.gaia/agents/email`, install the
email agent from the Agent Hub panel, open an email chat — the sidecar
spawns the installed binary (log: `using hub-installed verified binary`)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

tests Test changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Email Triage Download Error

2 participants