Documentation Index
Fetch the complete documentation index at: /llms.txt
Use this file to discover all available pages before exploring further.
NovitaSandboxProvider to build and start the Tbench2 OpenEnv server in a Novita Sandbox. It then connects to the server with Tbench2Env, resets a task, executes commands, and releases the sandbox when the run finishes.
The example is based on examples/novita_tbench2_simple.py in the OpenEnv repository.
Set NOVITA_API_KEY and run the example from the root of a local OpenEnv checkout:
export NOVITA_API_KEY="<YOUR_NOVITA_API_KEY>"
PYTHONPATH=src:envs uv run python examples/novita_tbench2_simple.py
import asyncio
import os
from openenv.core.containers.runtime.novita_provider import NovitaSandboxProvider
from tbench2_env import Tbench2Action, Tbench2Env
async def main() -> int:
tasks_dir = os.environ.get("TB2_TASKS_DIR")
if not tasks_dir:
print("TB2_TASKS_DIR not set. TB2 repo will be downloaded.")
task_id = os.environ.get("TB2_TASK_ID", "headless-terminal")
image = NovitaSandboxProvider.image_from_dockerfile(
"envs/tbench2_env/server/Dockerfile",
)
provider = NovitaSandboxProvider()
# The first start builds a Novita template; later starts reuse the cache.
base_url = provider.start_container(image=image)
provider.wait_for_ready(base_url, timeout_s=300)
try:
async with Tbench2Env(base_url=base_url, provider=provider) as env:
result = await env.reset(task_id=task_id)
print("Instruction head:")
print(result.observation.instruction[:200])
result = await env.step(
Tbench2Action(action_type="exec", command="ls -la")
)
print("Command output:")
print(result.observation.output)
result = await env.step(
Tbench2Action(
action_type="exec", command="curl -v http://127.0.0.1:8000"
)
)
print("Command output:")
print(result.observation.output)
finally:
provider.stop_container()
return 0
if __name__ == "__main__":
raise SystemExit(asyncio.run(main()))
examples/novita_tbench2_e2e_eval.py:
export NOVITA_API_KEY="<YOUR_NOVITA_API_KEY>"
export API_KEY="<YOUR_MODEL_API_KEY>"
export API_BASE_URL="<YOUR_MODEL_BASE_URL>"
PYTHONPATH=src:envs uv run python examples/novita_tbench2_e2e_eval.py \
--task regex-log \
--model "<YOUR_MODEL_NAME>" \
--max-steps 20 \
--verbose