How to run Apex-Agents?

[bleonard]

Overall, it seems like I need access to mercor-mcp-shared to even build uv and definitely to run it.

Following Quick Start Option A on a clean clone, docker compose up -d --build fails. The documents, spreadsheets, mail, and calendar MCP servers depend on git+https://github.com/Mercor-Intelligence/mercor-mcp-shared.git, which is private.

[bleonard]

Maybe it's the same with huggingface too?

➜  hugging_face_task git:(main) ✗ ./run.sh
============================================================
HUGGING FACE TASK
============================================================
Example dir:     /Users/brian/benchmarks/archipelago/examples/hugging_face_task
Archipelago dir: /Users/brian/benchmarks/archipelago
============================================================
Installing agent dependencies...
Resolved 216 packages in 22ms
Checked 208 packages in 42ms
Installing grading dependencies...
Resolved 138 packages in 2ms
Checked 121 packages in 20ms
[11:29:58] Downloading task data from HuggingFace...
Traceback (most recent call last):
  File "/Users/brian/benchmarks/archipelago/agents/.venv/lib/python3.13/site-packages/huggingface_hub/utils/_http.py", line 553, in hf_raise_for_status
    response.raise_for_status()
    ~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/Users/brian/benchmarks/archipelago/agents/.venv/lib/python3.13/site-packages/httpx/_models.py", line 829, in raise_for_status
    raise HTTPStatusError(message, request=request, response=self)
httpx.HTTPStatusError: Client error '401 Unauthorized' for url 'https://huggingface.co/datasets/mercor/apex-agents/resolve/main/tasks_and_rubrics.json'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/401

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/brian/benchmarks/archipelago/examples/hugging_face_task/main.py", line 414, in <module>
    main()
    ~~~~^^
  File "/Users/brian/benchmarks/archipelago/examples/hugging_face_task/main.py", line 155, in main
    tasks_path = hf_hub_download(
        HF_DATASET, "tasks_and_rubrics.json", repo_type="dataset"
    )
  File "/Users/brian/benchmarks/archipelago/agents/.venv/lib/python3.13/site-packages/huggingface_hub/utils/_validators.py", line 89, in _inner_fn
    return fn(*args, **kwargs)
  File "/Users/brian/benchmarks/archipelago/agents/.venv/lib/python3.13/site-packages/huggingface_hub/file_download.py", line 1007, in hf_hub_download
    return _hf_hub_download_to_cache_dir(
        # Destination
    ...<15 lines>...
        dry_run=dry_run,
    )
  File "/Users/brian/benchmarks/archipelago/agents/.venv/lib/python3.13/site-packages/huggingface_hub/file_download.py", line 1135, in _hf_hub_download_to_cache_dir
    _raise_on_head_call_error(head_call_error, force_download, local_files_only)
    ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/brian/benchmarks/archipelago/agents/.venv/lib/python3.13/site-packages/huggingface_hub/file_download.py", line 1731, in _raise_on_head_call_error
    raise head_call_error
  File "/Users/brian/benchmarks/archipelago/agents/.venv/lib/python3.13/site-packages/huggingface_hub/file_download.py", line 1623, in _get_metadata_or_catch_error
    metadata = get_hf_file_metadata(
        url=url, timeout=etag_timeout, headers=headers, token=token, endpoint=endpoint
    )
  File "/Users/brian/benchmarks/archipelago/agents/.venv/lib/python3.13/site-packages/huggingface_hub/utils/_validators.py", line 89, in _inner_fn
    return fn(*args, **kwargs)
  File "/Users/brian/benchmarks/archipelago/agents/.venv/lib/python3.13/site-packages/huggingface_hub/file_download.py", line 1549, in get_hf_file_metadata
    response = _httpx_follow_relative_redirects(method="HEAD", url=url, headers=hf_headers, timeout=timeout)
  File "/Users/brian/benchmarks/archipelago/agents/.venv/lib/python3.13/site-packages/huggingface_hub/file_download.py", line 292, in _httpx_follow_relative_redirects
    hf_raise_for_status(response)
    ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^
  File "/Users/brian/benchmarks/archipelago/agents/.venv/lib/python3.13/site-packages/huggingface_hub/utils/_http.py", line 573, in hf_raise_for_status
    raise _format(GatedRepoError, message, response) from e
huggingface_hub.errors.GatedRepoError: 401 Client Error. (Request ID: Root=1-6a1f2126-271bf98b456b5d3f060400a2;ace97abf-5349-43c5-8827-e45ed5a8c99c)

Cannot access gated repo for url https://huggingface.co/datasets/mercor/apex-agents/resolve/main/tasks_and_rubrics.json.
Access to dataset mercor/apex-agents is restricted. You must have access to it and be authenticated to access it. Please log in.

I do see this is accessible: https://huggingface.co/datasets/mercor/apex-agents
I am invsitgating if I need to have a huggingface account and an env variable or something

[bleonard]

OK, I set HF_TOKEN and HUGGING_FACE_HUB_TOKEN and agreed to something on this page: https://huggingface.co/datasets/mercor/apex-agents

That got me further.

But then it fails at the same spot

  0.715   × Failed to download and build `mercor-mcp-shared @
  0.715   │ git+https://github.com/Mercor-Intelligence/mercor-mcp-shared.git@bb79278f61504c4d30d604cca73e304404999984`
  0.715   ├─▶ Git operation failed
  0.715   ├─▶ failed to clone into: /root/.cache/uv/git-v0/db/46377426aaabaa66
  0.715   ├─▶ failed to fetch commit `bb79278f61504c4d30d604cca73e304404999984`
  0.715   ╰─▶ process didn't exit successfully:
  0.715       `/usr/bin/git fetch --force --update-head-ok
  0.715       'https://github.com/Mercor-Intelligence/mercor-mcp-shared.git'
  0.715       '+bb79278f61504c4d30d604cca73e304404999984:refs/commit/bb79278f61504c4d30d604cca73e304404999984'`
  0.715       (exit status: 128)
  0.715       --- stderr
  0.715       fatal: could not read Username for 'https://github.com': terminal
  0.715       prompts disabled
  0.715
  0.715
  0.715 hint: `mercor-mcp-shared` was included because `mercor-rls-docs` (v0.1.0) depends on `mercor-mcp-shared`

It looks like I need to set UV_GITHUB_TOKEN (and have access to that repo, of course).

[liaohch3]

I am tracking the same issue in #82 as well.

As of the latest public main I tested (9d9afa46298732824feaa9f07b77607b1dc7a19b), the official examples are still not reproducible from a clean public clone. The run still eventually depends on the private Mercor-Intelligence/mercor-mcp-shared repository, so the MCP server dependencies cannot be installed without private GitHub access.

I do not see an official fix available yet, so I am also waiting for the upstream repo to make this runnable without private dependencies or document the required credentials clearly.