From 237ef3a64d2a7f44c04ee40ce80a826164281bbe Mon Sep 17 00:00:00 2001 From: Peter Steinberger Date: Mon, 4 May 2026 21:17:49 +0100 Subject: [PATCH] docs: sync provider and vnc docs (#26) --- README.md | 18 +- docs/README.md | 7 +- docs/architecture.md | 10 +- docs/cli.md | 47 ++++- docs/commands/admin.md | 18 +- docs/commands/cleanup.md | 6 + docs/commands/config.md | 2 +- docs/commands/doctor.md | 3 + docs/commands/image.md | 3 + docs/commands/inspect.md | 3 + docs/commands/list.md | 3 + docs/commands/run.md | 7 +- docs/commands/screenshot.md | 7 + docs/commands/ssh.md | 3 + docs/commands/status.md | 3 + docs/commands/stop.md | 3 + docs/commands/vnc.md | 7 +- docs/commands/warmup.md | 2 +- docs/commands/webvnc.md | 11 ++ docs/features/README.md | 9 +- docs/features/actions-hydration.md | 6 +- docs/features/aws.md | 117 ++++++++++++ docs/features/blacksmith-testbox.md | 15 ++ docs/features/hetzner.md | 78 ++++++++ docs/features/interactive-desktop-vnc.md | 234 +++++++++++++---------- docs/features/providers.md | 16 +- docs/features/vnc-linux.md | 87 +++++++++ docs/features/vnc-macos.md | 88 +++++++++ docs/features/vnc-windows.md | 111 +++++++++++ docs/how-it-works.md | 6 +- docs/infrastructure.md | 54 ++++-- docs/operations.md | 20 ++ docs/source-map.md | 12 +- 33 files changed, 844 insertions(+), 172 deletions(-) create mode 100644 docs/features/aws.md create mode 100644 docs/features/hetzner.md create mode 100644 docs/features/vnc-linux.md create mode 100644 docs/features/vnc-macos.md create mode 100644 docs/features/vnc-windows.md diff --git a/README.md b/README.md index c5d6965..41d9d51 100644 --- a/README.md +++ b/README.md @@ -6,13 +6,13 @@ **Warm a box, sync the diff, run the suite.** -Crabbox is an open-source remote testbox runner for maintainers and AI agents. Lease a fast Linux machine on owned cloud capacity, or point at an existing macOS/Windows SSH host, sync your dirty checkout, run a command remotely, stream output, and release. Local edit-save-run loop, cloud-grade compute. +Crabbox is an open-source remote testbox runner for maintainers and AI agents. Lease fast managed cloud capacity, or point at an existing SSH host, sync your dirty checkout, run a command remotely, stream output, and release. Local edit-save-run loop, cloud-grade compute. ```sh crabbox run -- pnpm test ``` -Behind that single command: a Go CLI on your laptop, a Cloudflare Worker broker that owns provider credentials and lease state, and a vanilla Ubuntu runner on Hetzner Cloud or AWS EC2 Spot. Crabbox can also wrap Blacksmith Testboxes when you choose `provider: blacksmith-testbox`, or use `provider: ssh` for existing macOS and Windows targets. +Behind that single command: a Go CLI on your laptop, a Cloudflare Worker broker that owns provider credentials and lease state, and a managed runner on Hetzner Cloud or AWS EC2. Crabbox can also wrap Blacksmith Testboxes when you choose `provider: blacksmith-testbox`, or use `provider: ssh` for existing macOS and Windows targets. --- @@ -55,7 +55,7 @@ Every lease has a stable `cbx_...` ID and a friendly crustacean slug (`blue-lobs ```text your laptop Cloudflare Worker cloud provider ------------- ------------------ -------------- -crabbox CLI -- HTTPS --> Fleet Durable Object --> Hetzner / AWS Spot +crabbox CLI -- HTTPS --> Fleet Durable Object --> Hetzner / AWS EC2 | lease + cost state | | | +------------ SSH + rsync to leased runner <--------------+ @@ -75,7 +75,7 @@ For the full mental model, see [How Crabbox Works](docs/how-it-works.md). For th - **Run observability.** Every coordinator-backed run gets an early `run_...` handle. Use `crabbox attach ` while it is active, `crabbox events --after --limit ` for durable lifecycle/output events, and `crabbox logs ` for retained output after completion. - **Stable timing records.** `--timing-json` on `run`, `warmup`, and `actions hydrate` gives scripts one machine-readable sync/command/total timing schema across AWS, Hetzner, and Blacksmith Testboxes. - **Local-first sync.** No clean-checkout requirement. Tracked + nonignored files only, fingerprint skip on no-op runs, sanity checks against suspicious mass deletions, optional shallow base-ref hydration for changed-test workflows. -- **Brokered cloud.** Maintainers and agents share infra without sharing provider tokens. Hetzner and AWS EC2 Spot are first-class Linux providers; AWS also owns managed Windows and EC2 Mac targets. Providers fall back across compatible instance families when capacity or quota rejects a request. +- **Brokered cloud.** Maintainers and agents share infra without sharing provider tokens. Hetzner and AWS EC2 are first-class managed providers; AWS also owns managed Windows and EC2 Mac targets. Linux defaults to Spot unless capacity config says otherwise. Providers fall back across compatible instance families when capacity or quota rejects a request. - **macOS and Windows static hosts.** `provider: ssh` reuses existing machines; it does not create macOS or Windows Crabbox boxes. macOS and Windows WSL2 use the POSIX rsync path; native Windows uses PowerShell plus tar archive sync. - **Blacksmith Testbox wrapper.** Set `provider: blacksmith-testbox` to delegate warmup/run/list/status/stop to the Blacksmith CLI while Crabbox keeps local slugs, repo claims, timing summaries, and config conventions. - **Trusted AWS images.** Operators can create AMIs from active brokered AWS leases and promote a known-good image as the coordinator default. @@ -96,20 +96,22 @@ Hetzner standard ccx33, cpx62, cx53 large ccx53, ccx43, cpx62, cx53 beast ccx63, ccx53, ccx43, cpx62, cx53 -AWS Spot standard c7a/c7i/m7a/m7i.8xlarge family +AWS Linux standard c7a/c7i/m7a/m7i.8xlarge family fast …16xlarge family large …24xlarge family beast …48xlarge family, falling back to 32x/24x/16x AWS Win standard m7i.large, m7a.large, t3.large - fast m7i.2xlarge, m7a.2xlarge, m7i.xlarge - large m7i.4xlarge, m7a.4xlarge, m7i.2xlarge + fast m7i.xlarge, m7a.xlarge, t3.xlarge + large m7i.2xlarge, m7a.2xlarge, t3.2xlarge beast m7i.4xlarge, m7a.4xlarge, m7i.2xlarge AWS WSL2 standard m8i.large, m8i-flex.large, c8i.large, r8i.large fast m8i.xlarge, m8i-flex.xlarge, c8i.xlarge, r8i.xlarge large m8i.2xlarge, m8i-flex.2xlarge, c8i.2xlarge, r8i.2xlarge - beast m8i.4xlarge, m8i-flex.4xlarge, c8i.4xlarge, r8i.4xlarge + beast m8i.4xlarge, m8i-flex.4xlarge, c8i.4xlarge, r8i.4xlarge, m8i.2xlarge + +AWS macOS all mac2.metal unless --type is set ``` Override with `--type` or `CRABBOX_SERVER_TYPE` for a specific instance. diff --git a/docs/README.md b/docs/README.md index 66494f0..6a54521 100644 --- a/docs/README.md +++ b/docs/README.md @@ -6,20 +6,20 @@ Crabbox is a shared remote testbox system for OpenClaw maintainers and AI agents. The goal is to keep the local developer story unchanged - edit, save, run - while moving compute and tests onto owned cloud capacity. -A `crabbox run` command leases a brokered Linux machine or reuses a static macOS/Windows SSH host, syncs your tracked and nonignored local files, executes the command remotely, streams stdout and stderr back, and releases or unclaims the target. Behind the scenes a small Cloudflare-hosted broker owns cloud provider credentials, lease state, cleanup, usage, and cost guardrails so individual machines and CLIs never need to. +A `crabbox run` command leases a brokered cloud machine or reuses a static SSH host, syncs your tracked and nonignored local files, executes the command remotely, streams stdout and stderr back, and releases or unclaims the target. Behind the scenes a small Cloudflare-hosted broker owns cloud provider credentials, lease state, cleanup, usage, and cost guardrails so individual machines and CLIs never need to. ## How it fits together ```text your laptop Cloudflare Worker cloud provider ------------- ------------------ -------------- -crabbox CLI -- HTTPS --> Fleet Durable Object --> Hetzner / AWS Spot +crabbox CLI -- HTTPS --> Fleet Durable Object --> Hetzner / AWS EC2 | lease + cost state | | | +------------ SSH + rsync to leased runner <--------------+ ``` -The CLI is a Go binary. The broker is a Cloudflare Worker plus a single Durable Object. Brokered runners are vanilla Ubuntu boxes prepared by cloud-init with SSH, Git, rsync, curl, jq, and `/work/crabbox`. Static macOS and Windows targets are existing SSH hosts selected with `provider: ssh`. Project runtimes come from Actions hydration or repo-owned setup. Runners hold no broker credentials - they are leaf nodes. +The CLI is a Go binary. The broker is a Cloudflare Worker plus a single Durable Object. Brokered Linux runners are vanilla Ubuntu boxes prepared by cloud-init with SSH, Git, rsync, curl, jq, and `/work/crabbox`; AWS can also broker managed Windows and EC2 Mac desktop targets. Static hosts are existing SSH machines selected with `provider: ssh`. Project runtimes come from Actions hydration or repo-owned setup. Runners hold no broker credentials - they are leaf nodes. ## A run, end to end @@ -78,6 +78,7 @@ Pick whichever matches your intent: - **Get the mental model:** [How Crabbox Works](how-it-works.md), [Architecture](architecture.md), [Orchestrator](orchestrator.md). - **Use the CLI:** [CLI](cli.md), [Commands](commands/README.md), [Features](features/README.md), [Actions hydration](features/actions-hydration.md). +- **Pick a target:** [Providers](features/providers.md), [AWS](features/aws.md), [Hetzner](features/hetzner.md), [Blacksmith Testbox](features/blacksmith-testbox.md), [Interactive desktop and VNC](features/interactive-desktop-vnc.md). - **Operate it:** [Operations](operations.md), [Observability](observability.md), [Troubleshooting](troubleshooting.md), [Performance](performance.md). - **Set it up or audit it:** [Infrastructure](infrastructure.md), [Security](security.md), [Source Map](source-map.md), [MVP Plan](mvp-plan.md). diff --git a/docs/architecture.md b/docs/architecture.md index c642453..e57bfb1 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -6,7 +6,7 @@ Crabbox has three main parts: - CLI: local Go binary used by maintainers and agents. - Coordinator: Cloudflare Worker plus Durable Object state. -- Workers: Hetzner or SSH-accessible machines that run commands. +- Workers: managed cloud or SSH-accessible machines that run commands. The coordinator leases machines. The CLI executes work. Machines do not need to call back to the coordinator in the MVP. @@ -19,7 +19,7 @@ developer laptop Cloudflare Worker Durable Object lease state | - | Hetzner API or AWS EC2 Spot API + | Hetzner API or AWS EC2 API v cloud machines @@ -36,7 +36,7 @@ leased machine 2. CLI creates a per-lease SSH key. 3. CLI sends `POST /v1/leases` with lease ID, slug, profile, TTL, idle timeout, desired machine class, and SSH public key. 4. Coordinator validates identity and policy. -5. Durable Object chooses a provider from config and creates a Hetzner server or AWS EC2 Spot instance. +5. Durable Object chooses a provider from config and creates a Hetzner server or AWS EC2 instance. 6. Coordinator returns lease ID, slug, machine address, SSH user, workdir, and expiry. 7. CLI waits for `crabbox-ready`. 8. CLI seeds remote Git when possible, compares sync fingerprints, and syncs changed files with `rsync --delete`. @@ -101,7 +101,7 @@ Owned backends: - `hetzner-static`: pre-created warm machines. - `hetzner-ephemeral`: created per lease or overflow. -- `aws-spot`: one-time EC2 Spot instances for burst capacity. +- `aws`: one-time EC2 instances for burst capacity, managed Windows/WSL2, and EC2 Mac. - `ssh-static`: manually managed machines reachable by SSH. Brokered backends, later: @@ -109,7 +109,7 @@ Brokered backends, later: - `github-actions`: register or dispatch real Actions-backed runner work when workflow parity is required. - `external-runner`: adapter boundary for other hosted runner systems if needed. -The MVP implements `hetzner-ephemeral` and `aws-spot`, and leaves interfaces ready for `hetzner-static`. +The current broker implements `hetzner-ephemeral` and `aws`, and leaves interfaces ready for `hetzner-static`. ## Machine Bootstrap diff --git a/docs/cli.md b/docs/cli.md index eebb20c..d37e69c 100644 --- a/docs/cli.md +++ b/docs/cli.md @@ -71,7 +71,7 @@ One-shot run: crabbox run --profile project-check -- pnpm check:changed ``` -AWS EC2 Spot run: +AWS EC2 run: ```sh crabbox run --class beast -- pnpm check:changed @@ -122,7 +122,7 @@ crabbox run --provider ssh --target windows --windows-mode wsl2 --static-host wi Create managed AWS desktop boxes: ```sh -crabbox warmup --provider aws --target windows --desktop --market on-demand +crabbox warmup --provider aws --target windows --desktop CRABBOX_AWS_MAC_HOST_ID=h-... crabbox warmup --provider aws --target macos --desktop --market on-demand crabbox vnc --id blue-lobster crabbox screenshot --id blue-lobster --output desktop.png @@ -253,8 +253,16 @@ Flags: --profile profile to run on --class machine class override --type provider server or instance type override +--market spot|on-demand AWS capacity market override --ttl maximum lease lifetime, default 90m --idle-timeout idle expiry, default 30m +--desktop provision or require visible desktop capability +--browser provision or require browser capability +--tailscale join new managed Linux leases to the configured tailnet +--tailscale-tags Tailscale tags for new managed leases +--tailscale-hostname-template