feat(env): add Cua Cloud environment for desktop VMs by ddupont808 · Pull Request #1957 · harbor-framework/harbor

ddupont808 · 2026-06-17T03:12:46Z

Adds --env cua-cloud backed by Cua Cloud desktop VMs (Ubuntu, Windows, macOS).

Platforms

ubuntu — Ubuntu VM, lightweight, fast iteration
ubuntu-osworld — Ubuntu VM, full OSWorld golden image (Chrome, LibreOffice, GIMP, Thunderbird, VS Code, VLC)
windows — Windows Server VM
macos — macOS VM

Auth

export CUA_CLIENT_ID=ukey-...
export CUA_CLIENT_SECRET=...

Install: pip install 'harbor[cua]'

How it works

Each trial claims a VM from the Cua Cloud API, runs, then releases it back. Control plane (claim lifecycle) and data plane (exec/upload/screenshot) both go through run.cua.ai. The task's environment/ folder is intentionally empty — the VM image ships with the agent runtime pre-installed.

Tested on

hello-cua-cloud smoke task — oracle reward 1.0, 41s
OSWorld-Verified tasks from [Ready for Review] Adapter: OsWorld #1836 (datasets/osworld-verified, platform: ubuntu-osworld) — 4/7 × 1.0, 3/7 × 0.0 (no oracle solve script), 0 exceptions, 3m 51s

Files

src/harbor/environments/cua_cloud.py
examples/tasks/hello-cua-cloud/
examples/configs/job-cua-cloud.yaml

Adds a Harbor environment backed by Cua Cloud warm pools of Ubuntu, Windows, and macOS desktop VMs. **Usage** pip install 'harbor[cua]' export CUA_CLIENT_ID=ukey-... # from your Cua Cloud account export CUA_CLIENT_SECRET=... harbor run examples/configs/job-cua-cloud.yaml --agent oracle **Platforms** - `ubuntu` — lightweight Ubuntu pool for fast iteration - `ubuntu-osworld` — full OSWorld golden image (Chrome, LibreOffice, GIMP, …) - `windows` — Windows Server pool - `macos` — macOS pool **How it works** Each trial claims a pre-warmed VM from a pool, runs, then releases it. Both the control plane (VM claim lifecycle) and data plane (exec/upload/ screenshot) go through `run.cua.ai` using a single `CUA_CLIENT_ID` / `CUA_CLIENT_SECRET` credential pair. No kubeconfig required. **Files** - `src/harbor/environments/cua_cloud.py` — CuaCloudEnvironment - `src/harbor/environments/factory.py` — register cua-cloud type - `src/harbor/models/environment_type.py` — add CUA_CLOUD variant - `tests/unit/environments/test_cua_cloud.py` — 20 unit tests - `examples/tasks/hello-cua-cloud/` — smoke task (oracle reward 1.0) - `examples/configs/job-cua-cloud.yaml` — canonical job config

…(datasets/osworld-verified)

vercel · 2026-06-17T03:12:50Z

@ddupont808 is attempting to deploy a commit to the Harbor Framework Team on Vercel.

A member of the Team first needs to authorize it.

github-actions · 2026-06-17T03:12:56Z

Enjoy a better diff viewing experience by clicking one of these URLs:

…p_extra

alexgshaw · 2026-06-17T04:05:05Z

Would love to have this. Can you iterate against Devin comments / CI checks until they're passing and then we can merge?

… refresh interface

…_request)

…onse binding, stdout.strip, echo %errorlevel% handle-redirect

…_request

kobe0938

We trust your judgement on of the cua cloud env implementation,

and are ready to merge after these two small things:

the module docstring documents old CUA_CLOUD_KUBECONFIG / CUA_CLOUD_API_KEY auth, while the implementation requires CUA_CLIENT_ID / CUA_CLIENT_SECRET in preflight and TrainClient.from_key. This will send users to the wrong setup path.
have you tried to run terminal bench 2.1 using the oracle agent and see if it reaches a pass rate of over 95%.

Replaces stale CUA_CLOUD_KUBECONFIG / CUA_CLOUD_API_KEY references with the actual CUA_CLIENT_ID / CUA_CLIENT_SECRET env vars required by TrainClient.from_key() and preflight().

When a task sets [environment].docker_image, the cua-cloud environment now pulls that image inside the Ubuntu VM and runs the task's solve.sh / test.sh via docker exec. This enables terminal-bench and other Docker-based task sets to run on cua-cloud pools. Changes: - _docker_container field tracks the running container per trial - _start_docker_env(): installs docker.io if absent, pulls the task image, starts a long-running container with /solution, /tests, /logs, /agent bind-mounts matching harbor's EnvironmentPaths layout - exec(): routes through _compose_docker_exec when a container is running, so agent and verifier commands run inside the container - upload_dir/upload_file: strip CRLF from .sh files before uploading (Windows checkout produces CRLF that breaks bash inside Linux containers) - stop(): docker rm -f the container before deleting the claim

…ker container Two generalizable fixes applied once after container start: - add-apt-repository universe: covers packages like stockfish and many other tools that live in universe but not main/restricted - apt install adduser passwd sudo procps: minimal ubuntu images drop these; sysadmin tasks (create users, sudo, ps) need them

Pulls pre-built task docker_image inside the VM and routes all exec() through docker exec, mirroring how Daytona/E2B use task images. Key fixes: --network host eliminates MTU/DNS issues causing uv downloads to hang; /tmp/harbor-pkg-cache:/root/.cache mount prevents "no space left" errors for large packages (torch, numpy) on the container overlay. UV_BREAK_SYSTEM_PACKAGES=1 injected into every docker exec so uv pip install --system works in Debian/Ubuntu containers. Result: 74% pass rate on terminal-bench@2.0 oracle (66/89 tasks).

- Replace assert with explicit if/raise (AGENTS.md: runtime guards must not disappear under -O optimized execution) - Clarify _upload_environment_dir_after_start is a no-op for docker_image tasks with Dockerfile (pre-built image already has those files) - Merge main: add adapter extra, harbor[ec2] in cloud bundle; uv.lock regenerated with cua-train v0.1.1

- Add /harbor bind-mount so skill injection (post-start upload to /harbor/skills/) is visible inside the Docker container - logger.info → logger.debug for docker install/pull/container-start (AGENTS.md: prefer debug for operational details) - /tmp/harbor-pkg-cache and --network host already present; add comment explaining /harbor rationale

ddupont808 · 2026-06-23T22:39:46Z

We trust your judgement on of the cua cloud env implementation,

and are ready to merge after these two small things:

the module docstring documents old CUA_CLOUD_KUBECONFIG / CUA_CLOUD_API_KEY auth, while the implementation requires CUA_CLIENT_ID / CUA_CLIENT_SECRET in preflight and TrainClient.from_key. This will send users to the wrong setup path.

have you tried to run terminal bench 2.1 using the oracle agent and see if it reaches a pass rate of over 95%.

Both addressed, auth docs corrected and terminal-bench 2.0 oracle runs across all 89 tasks without environment issues

338b7d0 Add per-agent concurrency limits and agent-end hooks (harbor-framework#2014) a264cef Add Blaxel network policy and DinD support (harbor-framework#1980) d0e0869 feat(agents): propagate AgentConfig.env to all agent load paths (harbor-framework#1967) 4b09d97 feat: mvp for run launcher directly in viewer (harbor-framework#2017) e266675 feat: implement git repo skills (harbor-framework#1909) 31b38dd Skip uv download if already installed for mini swe agent (harbor-framework#1952) c6f4898 Refactor Harbor task handling and viewer UI (harbor-framework#2023) 19585c3 feat(atif): bring installed-agent trajectory converters to full ATIF v1.7 fidelity (harbor-framework#1888) 6485949 Add EC2 sandbox environment (harbor-framework#2031) c004b05 [CI] Post adapter validation as a fresh timestamped comment (harbor-framework#2040) 6671d38 Handle API usage limit errors (harbor-framework#2037) 34e545b add `harbor run --install-only` flag (harbor-framework#1983) c2e11d6 ci: update to windows server 2025 (harbor-framework#2033) 20160fc fix(swe-agent): don't let testbed-conda.sh abort under set -u (harbor-framework#2039) 7657d45 feat: combine --agent-import-path into --agent (harbor-framework#1988) 496293b test(pass-at-k): add unit tests for pass@k metric utility (harbor-framework#2046) 472bf7e fix(docker): fall back to tar upload when compose cp fails (harbor-framework#1944) 1cd8438 feat(cli): add agent/verifier log-filter flags to `harbor trial start` (harbor-framework#2047) 8cf5fdf fix(openhands-sdk): use uv to pin Python >=3.12 for SDK venv (harbor-framework#2041) 84b2a96 Api key auth (harbor-framework#2062) 52a31c3 Network mode support for docker environment (harbor-framework#1799) 3751070 Preview task definition image files (harbor-framework#2055) 0e8ae01 Add bash action, start_url, and configurable IO dir to computer-1 (harbor-framework#2042) 87c3c47 Minor updates. f3e7488 Merge branch 'main' of github.com:harbor-framework/harbor c0cecbb fix(resume): tolerate empty/truncated result.json on resume (harbor-framework#2053) dcaaebe default internet (harbor-framework#2066) 629f8ae Fix Gemini CLI JSONL session handling (harbor-framework#2069) 86d3bc0 fix(llms): cap Anthropic cache_control breakpoints at 4 (harbor-framework#2072) 4e6dfd6 Harden Kimi CLI runtime configuration (harbor-framework#2070) 0bc7e18 deps: pin transformers>=5.3.0 to fix CVE-2026-4372 (harbor-framework#2004) 6e2d9f5 Fix mini-swe-agent ATIF steps for responses API (harbor-framework#2073) f7110f1 feat(launcher): multi-agent/multi-dataset run launcher w/ configs (harbor-framework#2057) 053c05c feat(env): add Cua Cloud environment for desktop VMs (harbor-framework#1957) 5afda90 Add Codex web search CLI flag (harbor-framework#2032) c390779 Reorganize example configs by category (harbor-framework#2083) 0cdf5cf Add max_tokens support to MiniSweAgent (harbor-framework#2082) bd5005c Refine trial viewer typography and step detail interactions (harbor-framework#2077) a7667a0 fix(langgraph): forward task-environment MCP servers to the graph (harbor-framework#2085) 7ae9ee0 feat(agents): skip install for pre-built images (auto-detect + guarded mkdir) (harbor-framework#1970) 6fca45a feat: reload past run configs (harbor-framework#2076) 3fb4a92 Link observation results to source tool calls (harbor-framework#2087) e90f959 feat(viewer): save launcher config as YAML (harbor-framework#2089) 575a102 Remove job plugins from config files (harbor-framework#2074) 4340b0c Move verifier job options into a dedicated help panel (harbor-framework#2019) 701c90a feat(modal): add dynamic network policy switching (harbor-framework#2003) 703747d Toggle modal dynamic network configuration based on phase_network_policies (harbor-framework#2092) ff7e1a5 feat(rewardkit): opt-in cross-dimension aggregation for reward.json (harbor-framework#2010) 6712637 feat: launcher browse dataset path (harbor-framework#2093) 0de1cd4 Fix mswea reasoning. Co-authored-by: Cursor <cursoragent@cursor.com>

ddupont808 added 2 commits June 16, 2026 19:55

fix(cua-cloud): align osworld dataset path with harbor-framework#1836 …

f101b73

…(datasets/osworld-verified)

github-actions Bot added area:environments area:tests area:core area:package area:docs labels Jun 17, 2026

Merge branch 'main' into feature/cua-cloud

d408791

This comment was marked as resolved.

Sign in to view

fix(cua-cloud): token refresh on 401/302, assert→if/raise, factory pi…

5d9bb9c

…p_extra

This comment was marked as resolved.

Sign in to view

alexgshaw requested a review from kobe0938 June 17, 2026 04:05

ddupont808 added 2 commits June 17, 2026 00:00

fix(hello-cua-cloud): convert shell scripts from CRLF to LF

7168299

fix(hello-cua-cloud): enforce LF line endings for shell scripts

f47f480

This comment was marked as resolved.

Sign in to view

fix(cua-cloud): BOM in shell scripts, JSON escaping, 302 raise, token…

1ff8c60

… refresh interface

This comment was marked as resolved.

Sign in to view

ddupont808 and others added 3 commits June 17, 2026 13:20

fix(cua-cloud): raise on persistent 3xx in _k8s_request (mirrors _svc…

8071282

…_request)

Merge branch 'main' into feature/cua-cloud

72e8157

fix(cua-cloud): ruff format + update uv.lock with cua-train 0.1.1

40cd200

alexgshaw assigned kobe0938 Jun 17, 2026

This comment was marked as resolved.

Sign in to view

ddupont808 and others added 2 commits June 17, 2026 14:44

fix(cua-cloud): @OverRide decorators, dict generics, Task[None], resp…

460675d

…onse binding, stdout.strip, echo %errorlevel% handle-redirect

Merge branch 'main' into feature/cua-cloud

1f63155

This comment was marked as resolved.

Sign in to view

ddupont808 added 2 commits June 17, 2026 14:57

fix(cua-cloud): replace assert with if/raise in _k8s_request and _svc…

89aa200

…_request

chore: merge main + regenerate uv.lock

96e60e3

ddupont808 and others added 3 commits June 17, 2026 22:06

Merge remote-tracking branch 'origin/main' into feature/cua-cloud

0bd2068

fix: resolve uv.lock conflict markers, regenerate with cua-train

cf99cd5

Merge branch 'main' into feature/cua-cloud

e156f15

kobe0938 reviewed Jun 19, 2026

View reviewed changes

ddupont808 added 2 commits June 22, 2026 10:39

fix(cua-cloud): correct module docstring auth vars

ca4628e

Replaces stale CUA_CLOUD_KUBECONFIG / CUA_CLOUD_API_KEY references with the actual CUA_CLIENT_ID / CUA_CLIENT_SECRET env vars required by TrainClient.from_key() and preflight().

This comment was marked as resolved.

Sign in to view

ddupont808 added 2 commits June 23, 2026 11:27

This comment was marked as resolved.

Sign in to view

ddupont808 and others added 2 commits June 23, 2026 15:10

Merge branch 'main' into feature/cua-cloud

0ca3eba

kobe0938 approved these changes Jun 24, 2026

View reviewed changes

kobe0938 merged commit 053c05c into harbor-framework:main Jun 24, 2026
7 checks passed

Uh oh!

Conversation

ddupont808 commented Jun 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vercel Bot commented Jun 17, 2026

Uh oh!

github-actions Bot commented Jun 17, 2026

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

alexgshaw commented Jun 17, 2026

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

kobe0938 left a comment

Choose a reason for hiding this comment

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

ddupont808 commented Jun 23, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ddupont808 commented Jun 17, 2026 •

edited

Loading