Skip to content

Releases: huggingface/OpenEnv

Release list

v0.4.1

v0.4.1 Latest
Latest

Choose a tag to compare

View all tags
@github-actions github-actions released this 03 Jul 11:37
v0.4.1
65c506e
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Core feature highlights

  • External environment imports: added openenv import support for ORS/OpenReward and Verifiers environments, including extracted wrapper templates, safer vendoring behavior, task API integration, and an imported-environment example.
  • QED Math environment: added a new QED Math env with MCP tools, LLM-judge grading, math verifier service support, reward/rubric metrics, docs, and an inference example.
  • Current client/runtime docs: modernized tutorial notebooks and docs around the async EnvClient, WebSocket transport, .sync() wrapper usage, and stale Echo imports.
  • Release polish and compatibility: fixed openenv serve exit semantics, refreshed FinQA's minimum OpenEnv version, refactored inspect eval kwargs, and unpinned Hugging Face Hub in HF sandbox examples now that SandboxPool ships in 1.22.0.

What's Changed

New Contributors

Full Changelog: v0.4.0...v0.4.1

Contributors

sergiopaniego, burtenshaw, and 2 other contributors
Assets 4
Loading

v0.4.0

v0.4.0

Choose a tag to compare

View all tags
@github-actions github-actions released this 02 Jul 11:31
v0.4.0
275a864
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Core feature highlights

  • Runtime providers: added Azure Container Apps, Modal, and Hugging Face sandbox providers; provider startup is now owned by the provider interface, with fixes for uv project cloning, context timeouts, and localhost proxy handling.
  • Client and MCP runtime: auto-synced generated environment clients, surfaced StepResult metadata in typed clients, and tightened action deserialization plus MCP tool parsing, listing, and HTTP operation helpers.
  • Packaging and distribution: simplified install semantics around the openenv package, removed redundant extras from env packages, moved Docker/base-image references to the Hugging Face org, and published the project under BSD 3-Clause.
  • New environments and recipes: added ECHO env-token world modeling, Sophistry Bench Sprint, Reasoning Core community environments, and AWM expert-in-the-loop benchmark assets.
  • Reliability and examples: fixed BrowserGym reset behavior, JSON imports in the coding env, DIPG/OpenEnv tutorial notebooks, Wordle sync-client usage, OpenSpiel base-image references, and several provider/runtime docs paths.

What's Changed

  • chore: bump dev version by @burtenshaw in #738
  • chore: update repo and docs URLs after migration to huggingface org by @sergiopaniego in #736
  • chore: migrate docs infrastructure to HF doc-builder by @sergiopaniego in #739
  • chore: update GitHub org references from meta-pytorch to huggingface by @sergiopaniego in #740
  • fix(opencode_env): remove non-existent tests package from setuptools config by @sergiopaniego in #741
  • tweak docs by closing some menu sections by @burtenshaw in #750
  • Chore/readme cleanup by @sergiopaniego in #754
  • chore(deps): roll up core dependabot updates by @burtenshaw in #769
  • chore(deps): roll up env dependabot updates by @burtenshaw in #770
  • docs: fix rendering issues post HF doc-builder migration by @sergiopaniego in #743
  • chore: update Hub example slugs in auto/ docstrings to openenv org by @sergiopaniego in #752
  • chore: remove legacy meta-pytorch git install check from _validation.py by @sergiopaniego in #753
  • docs: drop release-notes page and update technical committee list by @sergiopaniego in #755
  • docs: update technical committee list to include Nvidia, Mercor, and Fleet AI by @sergiopaniego in #777
  • fix: use backtick URL in AnthropicClient.endpoint docstring to prevent encoding artifact by @sergiopaniego in #776
  • chore: update Docker base image registry from meta-pytorch to huggingface by @sergiopaniego in #751
  • chore(deps): roll up core Dependabot updates by @burtenshaw in #788
  • fix(auto): use proper URL parsing in _is_local_url by @u7k4rs6 in #742
  • chore(deps): bump pypdf from 6.10.2 to 6.12.0 in /envs/repl_env by @dependabot[bot] in #790
  • chore(deps): roll up env Dependabot updates by @burtenshaw in #789
  • Add Reasoning Core to community environments by @sileod in #781
  • chore(deps): consolidate env Dependabot updates by @burtenshaw in #804
  • Fix observation metadata serialization and Echo Docker docs port by @burtenshaw in #805
  • docs: align partner and committee listings with official blog by @sergiopaniego in #813
  • docs: correct StepResult field list in concepts guide by @sergiopaniego in #811
  • fix: pin fastapi>=0.116.0 to prevent starlette 0.46+ incompatibility by @sergiopaniego in #810
  • fix: surface top-level metadata on StepResult in typed env clients by @sergiopaniego in #812
  • chore(deps): bump pypdf from 6.12.0 to 6.13.0 in /envs/repl_env by @dependabot[bot] in #814
  • [codex] Group env Dependabot updates by @burtenshaw in #815
  • fix(pr-review-bot): only review PRs I was mentioned/review-requested on by @Darktex in #820
  • feat: ECHO env-token world modeling (RFC 010 + two runnable examples) by @thegovind in #819
  • feat: add Azure Container Apps cloud sandbox provider by @thegovind in #793
  • chore(deps): aggregate envs dependabot updates by @burtenshaw in #822
  • chore(deps): bump pypdf from 6.13.0 to 6.13.3 in /envs/repl_env by @dependabot[bot] in #825
  • chore(deps): aggregate envs dependabot updates by @burtenshaw in #826
  • [codex] Add typing_extensions to core deps by @burtenshaw in #828
  • Add AWM expert-in-the-loop benchmark recipe and honest eval assets by @thegovind in #833
  • Add sophistry_bench_sprint_env: single-agent advocacy reward-hacking environment by @acharyaanusha in #787
  • [codex] refactor env client connection helpers by @burtenshaw in #829
  • Fix Wordle tutorial imports and sync client usage by @burtenshaw in #836
  • [codex] refactor mcp tool parsing by @burtenshaw in #834
  • [codex] refactor push auth helpers by @burtenshaw in #835
  • [refactor] refactor validation main guard check by @burtenshaw in #827
  • docs(echo): link echo-rl, Prime Intellect, and TerminalBench citations by @sergiopaniego in #842
  • docs: list all RFCs in README and rfcs index (add RFC 000 and 010) by @sergiopaniego in #843
  • [codex] refactor build context detection by @burtenshaw in #850
  • [codex] refactor get endpoint registration by @burtenshaw in #851
  • chore(deps): aggregate envs dependabot updates by @burtenshaw in #848
  • chore(deps): aggregate core dependabot updates by @burtenshaw in #849
  • fix(uv-provider): honor context_timeout_s and stop busy-spinning the health poll by @RubenHaisma in #845
  • docs(echo): add TRL backend note for ECHO by @sergiopaniego in #844
  • chore: update deploy-hf-env.yml by @hf-security-analysis[bot] in #852
  • fix(env-client): disable localhost proxy per-connection instead of racing on NO_PROXY by @RubenHaisma in #846
  • [codex] fix opencode example mcp path by @burtenshaw in #857
  • [codex] refactor action deserialization routing by @burtenshaw in #855
  • [codex] refactor skills agent target selection by @burtenshaw in #856
  • add runnable TRL backend demo for ECHO by @sergiopaniego in #858
  • docs: remove stale PyTorch logo from OpenEnv headers by @sergiopaniego in #859
  • fix(examples): await Atari async client by @burtenshaw in #863
  • docs: add RadixArk to committee, SGLang and Miles to supporters by @unseenmars in #872
  • infra: point openspiel base image to ghcr.io/huggingface/openenv-openspiel-base by @sergiopaniego in #871
  • Fix OpenEnv tutorial notebook JSON by @burtenshaw in #866
  • docs: document optional dependency extras (inspect, all) by @sergiopaniego in #869
  • Add ModalProvider(ContainerProvider) by @joyliu-q in #821
  • fix: address modal provider review feedback by @burtenshaw in #878
  • [codex] refactor MCP list tools helpers by @burtenshaw in #873
  • fix(uv-provider): clone git+ project paths before uv run --project by @acharyaanusha in #854
  • Fix BrowserGym server reset by @burtenshaw in #840
  • docs: complete and reconcile runtime-provider documentation by @sergiopaniego in #868
  • ci: use ubuntu-latest for openspiel base build (8-core-ubuntu runner unknown) by @sergiopaniego in #881
  • chore(deps): aggregate envs dependabot updates by @burtenshaw in #884
  • Fix DIPG RL notebook JSON by @burten...
Read more

Contributors

Darktex, hinogi, and 10 other contributors
Loading

v0.3.1

v0.3.1

Choose a tag to compare

View all tags
@github-actions github-actions released this 02 Jun 08:47
v0.3.1
7449c5d
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

What's Changed

New Contributors

Full Changelog: v0.3.0...v0.3.1

Contributors

kashif, sergiopaniego, and 6 other contributors
Loading

Release v0.3.0

Release v0.3.0

Choose a tag to compare

View all tags
@burtenshaw burtenshaw released this 11 May 11:37
v0.3.0
83711aa
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

OpenEnv 0.3.0

Core harness runtime

  • Add the new session harness runtime and collection pipeline for OpenEnv environments.
  • Add harness adapters and examples for BrowserGym, OpenSpiel, and Reasoning Gym workflows.
  • Add the openenv collect CLI path, shared collection helpers, and tests for rollout collection.

New environments

  • Add Agent World Model (AWM-1K), a synthetic agentic environment suite with MCP tools, SQLite-backed state, verification logic, and concurrent isolated sessions.
  • Add E2B-backed coding environments: jupyter_env, terminus_env, and coding_tools_env.
  • Add Gradio interfaces and lifecycle controls for the new coding environments.

Deployment and CLI

  • Extend openenv push with runtime variables and secrets via -e/--env-var, --secret, and optional variables: entries in openenv.yaml.
  • Add --hardware/-H to choose Hugging Face Space hardware at deploy time.
  • Add --count/-n to deploy multiple Space replicas with suffixed repo IDs.
  • Accept bare repository names in openenv push --repo-id.
  • Improve validation for default openenv init projects.

Web UI and environment fixes

  • Let environments control their Gradio tab layout.
  • Improve REPL environment onboarding and fix stale llm_model caching across resets.
  • Improve concurrent session behavior across BrowserGym, Echo, OpenSpiel, Reasoning Gym, and TextArena.
  • Fix CARLA done propagation and Docker environment-variable leakage.
  • Fix TBench2 reward evaluation behavior.
  • Standardize and harden Hugging Face canonical Gradio deployment tooling.

Documentation and examples

  • Restructure the docs navigation after the Sphinx migration and remove legacy MkDocs artifacts.
  • Add tutorials for BrowserGym harness usage, SFT warmup, MCP environments, Rubrics, Inspect AI evaluation, and end-to-end workflows.
  • Add deployment and environment-builder guide updates for the current openenv push workflow.
  • Add sync_env_docs.py checks to keep environment documentation aligned.

Maintenance

  • Update CI workflow actions and dependency groups.
  • Consolidate Dependabot behavior around root-level updates.
  • Refresh dependencies across environment lockfiles, including FastMCP, Python multipart, Authlib, Cryptography, Pillow, Pytest, and related environment-specific packages.

Full changelog: v0.2.3...v0.3.0

Loading

Release v0.2.3

Release v0.2.3

Choose a tag to compare

View all tags
@burtenshaw burtenshaw released this 28 Mar 18:56
v0.2.3
2faae32

OpenEnv 0.2.3

Core web fixes

  • redirect GET / and GET /web to /web/ for Gradio-backed Spaces
  • return 409 from GET /web/state before reset instead of surfacing an internal error
  • accept optional reset kwargs on POST /web/reset

Hub rollout tooling

  • add the shared gradio_web probe and expand repl_web for root-path validation
  • default canonical collection discovery to first-party unsuffixed spaces
  • keep repo-id override / reconcile support for canonical promotion and collection cleanup

REPL UI

  • keep the REPL-specific Gradio control panel and related server compatibility fixes for Hub deployment
Loading

OpenEnv v0.2.2 Release

OpenEnv v0.2.2 Release

Choose a tag to compare

View all tags
@burtenshaw burtenshaw released this 20 Mar 17:52
v0.2.2
7fdccfa
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Core Features

  • Async-first environment clients with persistent WebSocket sessions, plus a sync wrapper for non-async code. The core loop is standardized around reset(), step(), and state().
  • Rubric/evaluation support for RL and agentic evaluation, including delayed rewards, EvalHarness, Inspect/LightEval-style harnesses, and LLM-as-a-judge scoring via LLMJudge.
  • Auto-discovery API via AutoEnv and AutoAction, so callers can load envs by short name, package-name variant, or Hub repo ID instead of hardcoding imports.
  • MCP-native environments via MCPEnvironment, including tool discovery, tool calls through the env API, FastMCP 2.x/3.x compatibility, reserved-name validation, persistent MCP sessions, production vs simulation modes, and code mode support.
  • Built-in web UI at /web for interactive env inspection, with dynamic forms, action history, and live state updates.
  • Creator CLI bundled in openenv-core: openenv init, build, validate, push, fork, serve, and skills.
  • One-command Hugging Face publishing flow for env authors, plus fork/duplicate support for existing Spaces and custom registry pushes.

More Features

  • Generic client support via GenericEnvClient / GenericAction for raw-dict access when you do not want to install env-specific code or trust remote code locally.
  • Typed environment contracts built on Pydantic models for Action, Observation, State, and StepResult, including parameterized reset/step support.
  • Multiple connection modes for the same env client: connect to a live URL, launch from a Docker image, or run a Hugging Face Space locally with uv.
  • Runtime/provider abstraction for local execution: LocalDockerProvider, UVProvider, DockerSwarmProvider, Daytona support, and a Kubernetes-shaped extension point.
  • Multi-mode deployment validation in the CLI, including local structure checks and live endpoint/runtime checks for deployed envs.
  • One-command Hugging Face publishing flow for env authors, plus fork/duplicate support for existing Spaces and custom registry pushes.
  • Release/deploy infrastructure for env fleets, including versioned Hub tags, collection management, per-env deployment scripts, and endpoint verification tooling.
  • The package is now a real distributable core runtime, openenv-core, with the CLI included rather than a loose repo-only developer tool.
  • The framework is explicitly agent-facing, with Hub-hosted envs, MCP tools, sync/async client ergonomics, evaluation hooks, and RL-platform integration docs for TRL, Torchforge, Oumi, ART, and others.

Example Snippets

Async env usage plus sync wrapper

import asyncio
from echo_env import EchoAction, EchoEnv

async def main():
    async with EchoEnv(base_url="https://openenv-echo-env.hf.space") as env:
        await env.reset()
        result = await env.step(EchoAction(message="Hello, OpenEnv!"))
        print(result.observation.echoed_message)

asyncio.run(main())

with EchoEnv(base_url="https://openenv-echo-env.hf.space").sync() as env:
    env.reset()
    print(env.step(EchoAction(message="sync call")).observation.echoed_message)

Auto-discovery instead of manual imports

from openenv import AutoEnv, AutoAction

env = AutoEnv.from_env("coding-env")
CodeAction = AutoAction.from_env("coding-env")

with env.sync() as client:
    client.reset()
    result = client.step(CodeAction(code="print('Hello from AutoEnv')"))
    print(result.observation)

Run from Hub Docker or locally with uv

from echo_env import EchoEnv

env = await EchoEnv.from_env("openenv/echo_env")  # pull HF Docker image
local = await EchoEnv.from_env(
    "openenv/echo_env",
    use_docker=False,
    project_path="/path/to/local/echo_env",
)

CLI for creation, validation, publishing, and forking

openenv init my_env
cd my_env
openenv validate --verbose
openenv push --repo-id my-org/my-env --private
openenv fork openenv/echo_env --repo-id myuser/echo-env

MCP-style tool environments

from openenv.core.env_server import ListToolsAction, CallToolAction

obs = env.step(ListToolsAction())
print([tool.name for tool in obs.tools])

obs = env.step(CallToolAction(tool_name="echo", arguments={"message": "hi"}))
print(obs.result)

LLM-based rubric scoring

from openenv.core.rubrics import LLMJudge

judge = LLMJudge(
    prompt_template="Rate this action:\n{action}\n\nGiven observation:\n{observation}\nScore 0-1:",
    client=client,
)
score = await judge(action, observation)

Built-in web UI for local envs

from openenv.core.env_server import create_web_interface_app

app = create_web_interface_app(env, YourAction, YourObservation)
# browse http://localhost:8000/web

What's Changed

Read more

Contributors

zkwentz, kashif, and 56 other contributors
Loading

OpenEnv v0.2.1 Release

OpenEnv v0.2.1 Release

Choose a tag to compare

View all tags
@burtenshaw burtenshaw released this 04 Feb 10:25
v0.2.1
94a3e0d

OpenEnv v0.2.1 Release Notes

Release Commit: c2997e8f78588a6f0657950457c57725d1750fe8
Date: January 31, 2026

Highlights

This release introduces MCP (Model Context Protocol) support and significant improvements to the development workflow with TDD tooling.

New Features

MCP Support (Phase 1)

  • Added Model Context Protocol support for agent-environment communication (#224)
  • Refactored EchoEnv to use inline FastMCP tool pattern (#321)
  • Addressed review feedback for MCP implementation (#312)

RFC 004: Rubrics & Delayed Rewards

  • Added RFC 004 defining the Rubrics specification (#237)
  • Implemented delayed rewards support for trajectory-based scoring (#337)

TDD Workflow & Developer Tooling

  • Added comprehensive TDD workflow with git hooks, agents, and skills (#311)
  • Added .worktrees/ to .gitignore for isolated development (#330)

Bug Fixes & Improvements

  • Fixed reward metadata handling in coding environment (#305)
  • Fixed openenv-core dependency issues in build process
  • Fixed deprecation warnings during package builds
  • Fixed flat copy of TOML file in build process
  • Updated EchoEnv hub reference in quickstart guide (#317)

Documentation

  • Updated T2Bench documentation and deployment instructions

Dependencies

  • Bumped urllib3 to 2.6.3 in snake_env and textarena_env
  • Bumped filelock to 3.20.3 in textarena_env

Repository Maintenance

  • Unified line endings to LF across the repository (#350)
  • Added Scaler AI Labs to supporter list in README

Upgrading

pip install --upgrade openenv-core==0.2.1

What's Changed

New Contributors

Full Changelog: https://github.com/meta-pytorch/OpenEnv/commits/v0.2.1

Contributors

kashif, albertodepaola, and 22 other contributors
Loading