Compare commits

...

11 Commits

Author SHA1 Message Date
Vincent Koc
6c0987eb2f
feat(metrics): record sync run metrics 2026-05-01 07:51:56 -07:00
Vincent Koc
e06fd0e2c2
feat(aws): promote cached runner images 2026-05-01 06:28:51 -07:00
Vincent Koc
f335fae6f5
fix(actions): reuse registered runners 2026-05-01 05:38:08 -07:00
Vincent Koc
eae1976eb3
Revert "feat: add OpenClaw plugin"
This reverts commit 6c30a7669e.
2026-05-01 04:24:22 -07:00
Vincent Koc
460ac0ec2a
docs: update changelog for aws images 2026-05-01 04:19:41 -07:00
Vincent Koc
8ea7d83ead
Merge remote-tracking branch 'origin/main' into feat/openclaw-plugin
* origin/main:
  fix(cli): require real ssh readiness
  fix(aws): forward ssh source cidrs
  Update CHANGELOG.md
  fix: clean up blacksmith local lease state

# Conflicts:
#	internal/cli/coordinator_test.go
2026-05-01 04:12:26 -07:00
Vincent Koc
e3831e0c08
feat(aws): promote runner AMIs 2026-05-01 04:10:09 -07:00
Vincent Koc
3216e9db70
feat(aws): manage runner AMIs 2026-05-01 04:04:01 -07:00
Vincent Koc
1b6ff40b76
Merge remote-tracking branch 'origin/main' into feat/openclaw-plugin
* origin/main:
  feat: add blacksmith testbox workflow flags
  feat: add blacksmith provider and harden broker auth
  chore: start 0.2.0 development
  feat: add github browser login
  docs: add crabbox skill
  docs: update install tagline
  docs: back crabbox documentation with source map
  chore: update dependencies and enforce go coverage
  ci: tolerate missing homebrew tap token
  fix: stabilize slug idle leases
  feat: add OpenClaw plugin

# Conflicts:
#	CHANGELOG.md
#	README.md
#	package.json
2026-05-01 03:46:20 -07:00
Vincent Koc
5b345db868
feat(actions): support configured hydrate fields 2026-05-01 03:39:57 -07:00
Vincent Koc
77356ac3f3
feat: add OpenClaw plugin 2026-05-01 00:40:42 -07:00
33 changed files with 1325 additions and 661 deletions

View File

@ -10,6 +10,9 @@
- Added GitHub org membership enforcement before minting browser-login tokens.
- Added the canonical coordinator endpoint configured for OAuth callback generation.
- Added Blacksmith Testbox workflow flags for `crabbox warmup` and `crabbox run`, enabling one-command Testbox runs without repo YAML or environment variables.
- Added AWS runner image management with `crabbox image current`, `image list`, `image create`, and `image promote` so trusted operators can bake and select warmed AMIs.
- Added configured GitHub Actions hydrate fields through `actions.fields`, with CLI `-f key=value` overrides for repo-specific workflow inputs.
- Added structured sync-size metrics to coordinator run history for latency and throughput triage.
### Changed
@ -21,14 +24,16 @@
### Fixed
- Cleaned up Blacksmith Testbox local lease claims and per-lease SSH keys after failed warmups, explicit stops, and one-shot runs.
- Fixed GitHub Actions runner reuse by avoiding the unsupported `config.sh remove --unattended` flag on current runner releases.
- Restricted Worker admin routes to shared-token admin auth so GitHub browser-login users cannot call admin endpoints.
- Fixed `whoami` reporting for GitHub browser-login tokens.
- Fixed exact `cbx_...` lookups bypassing owner-scoped slug authorization checks.
- Added cleanup and a pending-login cap for unauthenticated GitHub OAuth login starts.
- Rebounded coordinator-stored run logs to the latest 64 KiB tail even if a client bypasses the CLI cap.
## 0.1.0 - 2026-05-01
Crabbox 0.1.0 is the first public release: a Go CLI, Cloudflare Worker coordinator, and OpenClaw plugin for leasing fast remote Linux machines, syncing dirty worktrees, running commands, and releasing or reusing warm boxes safely.
Crabbox 0.1.0 is the first public release: a Go CLI and Cloudflare Worker coordinator for leasing fast remote Linux machines, syncing dirty worktrees, running commands, and releasing or reusing warm boxes safely.
### Highlights
@ -38,7 +43,6 @@ Crabbox 0.1.0 is the first public release: a Go CLI, Cloudflare Worker coordinat
- Keep warm boxes ergonomic without runaway cost: kept leases auto-release after an idle timeout, defaulting to `30m`, while `--ttl` remains a maximum wall-clock cap.
- Hydrate a leased box through a project-owned GitHub Actions workflow so repositories define their own runtimes, services, secrets, caches, and readiness.
- Keep runner bootstrap intentionally tiny: SSH, Git, rsync, curl, jq, `/work/crabbox`, and cache directories only. Go, Node, pnpm, Docker, databases, and services belong to the repo setup layer.
- Drive Crabbox from OpenClaw through native plugin tools for run, warmup, status, list, and stop.
- Install via Homebrew with `brew install openclaw/tap/crabbox`, or download GoReleaser archives for macOS, Linux, and Windows.
### CLI
@ -116,12 +120,6 @@ Crabbox 0.1.0 is the first public release: a Go CLI, Cloudflare Worker coordinat
- Added stop-marker writing so `crabbox stop` can ask the waiting Actions job to exit cleanly.
- Runner labels include `crabbox`, canonical lease labels, readable slug labels, and profile/class labels.
### OpenClaw Plugin
- Added a native OpenClaw plugin package at the repository root.
- Added `crabbox_run`, `crabbox_warmup`, `crabbox_status`, `crabbox_list`, and `crabbox_stop` tools.
- Added plugin tests that verify command construction and disabled-tool behavior.
### Results, Cache, And History
- Added JUnit XML parsing and summaries for remote test result files.

View File

@ -39,6 +39,22 @@ crabbox run -- pnpm test
crabbox warmup # prints cbx_... + a slug
crabbox run --id blue-lobster -- pnpm test:changed
crabbox ssh --id blue-lobster
crabbox inspect --id blue-lobster --json
```
Inspect usage and estimated cost:
```sh
crabbox usage
crabbox usage --scope org --org openclaw
crabbox usage --scope all --json
```
`crabbox usage` reads coordinator history, so it requires a configured broker. Cost is an estimate for compute leases, not a provider invoice: the coordinator prefers explicit `CRABBOX_COST_RATES_JSON` overrides, then provider pricing from AWS Spot history or Hetzner server-type prices, then built-in fallback rates. Full reference: [docs/commands/usage.md](docs/commands/usage.md).
Stop a kept server:
```sh
crabbox stop blue-lobster
```
@ -71,7 +87,7 @@ For the full mental model, see [How Crabbox Works](docs/how-it-works.md). For th
- **Blacksmith Testbox wrapper.** Set `provider: blacksmith-testbox` to delegate warmup/run/list/status/stop to the Blacksmith CLI while Crabbox keeps local slugs, repo claims, timing summaries, and config conventions.
- **Cost guardrails.** Per-lease and monthly spend caps. Live pricing from EC2 Spot history or Hetzner server-type prices, with static fallbacks. `crabbox usage` summarizes spend by user, org, provider, and type.
- **GitHub Actions hydration.** `crabbox actions hydrate` registers a leased box as an ephemeral Actions runner, so the repo's own workflow installs runtimes, services, and secrets. Crabbox does not parse Actions YAML.
- **OpenClaw plugin.** The repo root is a native OpenClaw plugin. Agents drive Crabbox through `crabbox_run`, `crabbox_warmup`, `crabbox_status`, `crabbox_list`, and `crabbox_stop` instead of shelling out.
- **AWS image cache.** `crabbox image current|list|create` lets trusted operators inspect the active AMI and capture scrubbed, warmed AWS runner images after hydration.
- **Operator surface.** `doctor`, `init`, `status`, `inspect`, `list`, `usage`, `history`, `logs`, `results`, `cache`, `admin`, `cleanup`, plus `--json` output where it matters.
## Machine classes
@ -132,14 +148,6 @@ blacksmith:
Forwarded environment is intentionally narrow: `NODE_OPTIONS` and `CI`. Do not pass secrets as command-line arguments. Full env-var reference and per-command flags are in [docs/cli.md](docs/cli.md) and [docs/commands/](docs/commands/README.md).
## OpenClaw plugin
The repo root is a native OpenClaw plugin package. Once installed, it exposes Crabbox as agent tools:
- `crabbox_run`, `crabbox_warmup`, `crabbox_status`, `crabbox_list`, `crabbox_stop`
The plugin shells out to the configured `crabbox` binary, so local config, broker login, repo claims, and sync behavior stay owned by the CLI. Set `plugins.entries.crabbox.config.binary` if `crabbox` is not on `PATH`.
## Development
```sh
@ -175,7 +183,7 @@ open dist/docs-site/index.html
## Status
Crabbox 0.1.0 (2026-05-01) is the first public release. Working today: brokered Hetzner + AWS Spot provisioning, warm-box reuse, GitHub Actions hydration, cost guardrails, run history, JUnit summaries, OpenClaw plugin, and the full CLI surface listed above. **Not yet:** untrusted multi-tenant isolation — Crabbox today assumes shared trust between operators of a single broker.
Crabbox 0.1.0 (2026-05-01) is the first public release. Working today: brokered Hetzner + AWS Spot provisioning, warm-box reuse, GitHub Actions hydration, cost guardrails, run history, JUnit summaries, and the full CLI surface listed above. **Not yet:** untrusted multi-tenant isolation — Crabbox today assumes shared trust between operators of a single broker.
## License

View File

@ -58,18 +58,6 @@ crabbox stop blue-lobster
`crabbox doctor` validates local config, network reachability, and SSH key availability before you commit to a long workflow. `crabbox usage` summarizes recent spend by user, org, provider, and server type.
## OpenClaw plugin
The repository root is also a native OpenClaw plugin package. Once installed in OpenClaw, it exposes Crabbox operations as agent tools:
- `crabbox_run`
- `crabbox_warmup`
- `crabbox_status`
- `crabbox_list`
- `crabbox_stop`
The plugin shells out to the configured `crabbox` binary with argv arrays, so local Crabbox config, broker login, repo claims, and sync behavior stay owned by the CLI. Configure `plugins.entries.crabbox.config.binary` if the binary is not on `PATH`.
## Where to read next
Pick whichever matches your intent:

View File

@ -42,6 +42,10 @@ crabbox results <run-id> [--json]
crabbox cache stats --id <lease-id-or-slug> [--json]
crabbox cache purge --id <lease-id-or-slug> --kind pnpm|npm|docker|git|all --force
crabbox cache warm --id <lease-id-or-slug> -- <command...>
crabbox image current [--json]
crabbox image list [--name <glob>] [--json]
crabbox image create --id <lease-id-or-slug> --name <ami-name> [--wait] [--no-reboot]
crabbox image promote <ami-id>
crabbox actions hydrate --id <lease-id-or-slug> [--workflow <file|name|id>] [--wait-timeout <duration>]
crabbox actions register --id <lease-id-or-slug> [--repo owner/name]
crabbox actions dispatch [--workflow <file|name|id>] [-f key=value]
@ -156,6 +160,14 @@ crabbox cache warm --id blue-lobster -- pnpm install --frozen-lockfile
crabbox cache purge --id blue-lobster --kind pnpm --force
```
Create a warm AWS image from a hydrated kept box:
```sh
crabbox image current
crabbox image create --id blue-lobster --name openclaw-crabbox-20260501 --wait
crabbox image promote ami-0123456789abcdef0
```
Trusted operator lease controls:
```sh
@ -292,6 +304,8 @@ class: beast
actions:
workflow: .github/workflows/crabbox.yml
ref: main
fields:
- crabbox_docker_cache=true
runnerLabels:
- crabbox
sync:

View File

@ -15,6 +15,7 @@ Command docs live here, one file per top-level command. Keep `docs/cli.md` as th
- [logs](logs.md)
- [results](results.md)
- [cache](cache.md)
- [image](image.md)
- [status](status.md)
- [list](list.md)
- [usage](usage.md)

View File

@ -40,6 +40,8 @@ actions:
workflow: .github/workflows/crabbox.yml
job: hydrate
ref: main
fields:
- crabbox_docker_cache=true
runnerLabels:
- crabbox
runnerVersion: latest
@ -48,6 +50,7 @@ actions:
Workflow jobs should target the dynamic label printed by registration, for example `crabbox-cbx-123`, plus any static labels configured for the project.
When `actions.job` is set and the workflow declares `crabbox_job`, Crabbox sends it and verifies that the ready marker came from that job. Older workflows can omit both.
Use `actions.fields` for repository-specific workflow inputs that should be sent on every hydration. CLI `-f key=value` values override matching configured fields for that dispatch.
## Hydration Flow

24
docs/commands/image.md Normal file
View File

@ -0,0 +1,24 @@
# image
`crabbox image` manages AWS AMIs for faster runner startup.
```sh
crabbox image current
crabbox image list
crabbox image list --name 'openclaw-crabbox-*'
crabbox image create --id blue-lobster --name openclaw-crabbox-20260501
crabbox image create --id cbx_abcdef123456 --name openclaw-hot --wait --json
crabbox image promote ami-0123456789abcdef0
```
The brokered path uses the coordinator's AWS credentials and requires admin auth. Direct-provider mode uses local AWS credentials. `image current` shows the AMI that AWS leases will use: configured `aws.ami` / `CRABBOX_AWS_AMI` when set, otherwise the latest Ubuntu 24.04 AMI Crabbox resolves in the configured region.
`image list` returns self-owned AMIs tagged by Crabbox image creation. Pass `--name` to search self-owned images by AMI name glob.
`image create` captures an AMI from an existing AWS lease. By default it runs a best-effort scrub over SSH before calling EC2 `CreateImage`: it removes common AWS/Docker credential stores, Actions env handoff files, shell history, cloud-init logs, and old journal entries. Use `--skip-scrub` only for deliberately disposable boxes that never received secrets.
By default AWS may reboot the instance to capture a more consistent image. `--no-reboot` is faster but can capture inconsistent filesystem state. Use `--wait` when scripts need the AMI to be available before exiting.
`image promote <ami-id>` writes `aws.ami` to the writable Crabbox config file. Prefer this over committing account-specific AMI IDs to repo-local `.crabbox.yaml`.
Do not bake long-lived credentials into AMIs. Use the AMI for tools, packages, Docker/buildx cache, and base layers; keep runtime secrets in GitHub Actions, instance profiles, SSM, or the coordinator.

View File

@ -22,6 +22,7 @@ Core features:
- [History and logs](history-logs.md): coordinator run records and retained remote output tails.
- [Test results](test-results.md): JUnit summaries attached to recorded runs.
- [Cache controls](cache.md): inspect, purge, and warm remote package/build caches.
- [AWS images](../commands/image.md): inspect and create scrubbed AMIs from warmed AWS leases.
- [Auth and admin](auth-admin.md): login/logout/whoami and trusted operator controls.
- [Lifecycle cleanup](lifecycle-cleanup.md): release, expiry, keep mode, and direct cleanup.
- [Repository onboarding](repository-onboarding.md): `crabbox init`, repo config, workflow stub, and agent skill.
@ -37,6 +38,7 @@ Command docs:
- [logs](../commands/logs.md)
- [results](../commands/results.md)
- [cache](../commands/cache.md)
- [image](../commands/image.md)
- [status](../commands/status.md)
- [list](../commands/list.md)
- [usage](../commands/usage.md)

View File

@ -32,6 +32,7 @@ AWS behavior:
- uses Spot placement score across configured regions in direct AWS mode;
- can fall back to On-Demand after Spot capacity/quota failures when configured;
- fetches Spot price history when cost estimates need provider pricing.
- can inspect the configured runner AMI, list Crabbox-created AMIs, and create a tagged AMI from an existing AWS lease after best-effort scrub.
Machine classes map to provider-specific types:
@ -51,6 +52,8 @@ beast c7a.48xlarge, c7i.48xlarge, m7a.48xlarge, m7i.48xlarge, r7a.48xlarge,
Direct provider mode still exists when no coordinator is configured. It uses local AWS credentials or `HCLOUD_TOKEN`/`HETZNER_TOKEN` and should stay secondary to the brokered path.
AWS images are an operator acceleration layer, not a secret store. Bake Docker, buildx, language runtimes, package caches, and heavy base layers. Keep runtime secrets in the coordinator, GitHub Actions, AWS instance profiles, SSM, or a secrets manager.
Crabbox can also wrap Blacksmith Testboxes with `provider: blacksmith-testbox`. That backend does not use the Crabbox broker or direct cloud credentials. It shells out to the authenticated Blacksmith CLI for `testbox warmup`, `run`, `status`, `list`, and `stop`, while Crabbox keeps local slugs, repo claims, config, and timing summaries. See [Blacksmith Testbox](blacksmith-testbox.md).
Related docs:

View File

@ -24,9 +24,9 @@ Bootstrap installs:
- jq;
- OpenSSH server.
Bootstrap intentionally does not install project language runtimes such as Go, Node, pnpm, Docker, databases, or service dependencies. Those belong in GitHub Actions hydration, devcontainers, Nix, mise/asdf, or repository setup scripts. A machine should not pass readiness until `crabbox-ready` succeeds over SSH.
Bootstrap intentionally does not install project language runtimes such as Go, Node, pnpm, Docker, databases, or service dependencies. Those belong in GitHub Actions hydration, devcontainers, Nix, mise/asdf, repository setup scripts, or a trusted AWS AMI selected with `aws.ami` / `CRABBOX_AWS_AMI`. A machine should not pass readiness until `crabbox-ready` succeeds over SSH.
The CLI prefers the configured SSH port and can fall back to port 22 during early bootstrap. Long term, snapshots or provider images can replace slow cloud-init once the bootstrap contract is stable.
The CLI prefers the configured SSH port and can fall back to port 22 during early bootstrap. `crabbox image create` can capture a scrubbed AWS AMI from a warmed lease when cloud-init plus hydration is too slow for repeated work.
Related docs:

View File

@ -47,7 +47,7 @@ Reports include lease count, active lease count, elapsed runtime, estimated elap
## Run History And Logs
Coordinator-backed `crabbox run` creates a run record before the remote command starts and finishes it with exit code, timing, and the latest retained output tail.
Coordinator-backed `crabbox run` creates a run record before the remote command starts and finishes it with exit code, timing, sync size metrics, and the latest retained output tail.
Use:
@ -59,7 +59,13 @@ bin/crabbox logs run_...
bin/crabbox results run_...
```
History is for command debugging, not unlimited log archival. Logs are bounded tails of remote stdout/stderr. Test results are stored as structured summaries when `--junit` or `results.junit` is configured.
History is for command debugging, not unlimited log archival. Logs are bounded tails of remote stdout/stderr on both the CLI and coordinator side. Test results are stored as structured summaries when `--junit` or `results.junit` is configured.
Useful JSON fields for slow-run triage:
- `syncMs`, `commandMs`, and `durationMs`;
- `syncFiles`, `syncBytes`, `syncDeleted`, `syncManifestBytes`, and `syncSkipped`;
- `logBytes` and `logTruncated`.
## Remote Debugging

View File

@ -95,3 +95,11 @@ Use wall-clock timing around the whole command, not just the remote test process
```
The useful number includes lease wait, SSH readiness, sync, Git hydration, command execution, and release. For warm leases, sync fingerprints and package caches should make repeated runs much faster than cold runs.
Coordinator-backed runs also retain structured metrics in `history --json`:
```sh
bin/crabbox history --lease cbx_... --json
```
Use `syncMs`, `commandMs`, `durationMs`, `syncFiles`, `syncBytes`, `syncDeleted`, `syncManifestBytes`, and `syncSkipped` to separate source sync overhead from the remote command itself. If memory looks high, compare those values against `free -h`, `ps aux --sort=-rss | head`, `docker system df`, and `bin/crabbox cache stats --id cbx_...` on the same lease before blaming the coordinator path.

423
index.js
View File

@ -1,423 +0,0 @@
import { spawn } from "node:child_process";
const PLUGIN_ID = "crabbox";
const DEFAULT_BINARY = "crabbox";
const DEFAULT_MAX_OUTPUT_BYTES = 60_000;
const DEFAULT_TIMEOUT_SECONDS = 30 * 60;
const commandArraySchema = {
type: "array",
minItems: 1,
items: {
type: "string",
minLength: 1,
},
};
const envSchema = {
type: "object",
additionalProperties: {
type: "string",
},
};
const providerSchema = {
type: "string",
enum: ["aws", "hetzner"],
};
function readConfig(api) {
const raw = api?.pluginConfig && typeof api.pluginConfig === "object" ? api.pluginConfig : {};
return {
binary: readString(raw, "binary") ?? DEFAULT_BINARY,
maxOutputBytes: readPositiveInteger(raw, "maxOutputBytes", DEFAULT_MAX_OUTPUT_BYTES),
timeoutSeconds: readPositiveInteger(raw, "timeoutSeconds", DEFAULT_TIMEOUT_SECONDS),
allowRun: readBoolean(raw, "allowRun", true),
allowWarmup: readBoolean(raw, "allowWarmup", true),
allowStop: readBoolean(raw, "allowStop", true),
};
}
function readString(source, key) {
const value = source?.[key];
return typeof value === "string" && value.trim() ? value.trim() : undefined;
}
function readBoolean(source, key, fallback) {
const value = source?.[key];
return typeof value === "boolean" ? value : fallback;
}
function readPositiveInteger(source, key, fallback) {
const value = source?.[key];
return typeof value === "number" && Number.isFinite(value) && value > 0
? Math.floor(value)
: fallback;
}
function readStringArray(source, key) {
const value = source?.[key];
if (!Array.isArray(value) || value.length === 0) {
throw new Error(`${key} must be a non-empty string array`);
}
const next = value.map((item) => {
if (typeof item !== "string" || !item.trim()) {
throw new Error(`${key} must contain only non-empty strings`);
}
return item;
});
return next;
}
function readEnv(source) {
const value = source?.env;
if (value === undefined) {
return {};
}
if (!value || typeof value !== "object" || Array.isArray(value)) {
throw new Error("env must be an object of string values");
}
const entries = Object.entries(value).map(([key, entry]) => {
if (typeof entry !== "string") {
throw new Error(`env.${key} must be a string`);
}
return [key, entry];
});
return Object.fromEntries(entries);
}
function maybePush(args, flag, value) {
if (value !== undefined) {
args.push(flag, value);
}
}
function maybePushBool(args, flag, value) {
if (value === true) {
args.push(flag);
}
}
function toolResult(text, details) {
return {
content: [{ type: "text", text }],
details,
};
}
function commandLine(binary, args) {
return [binary, ...args].map((part) => (/\s/.test(part) ? JSON.stringify(part) : part)).join(" ");
}
function appendChunk(current, chunk, maxBytes) {
if (!chunk) {
return current;
}
const next = current + chunk;
if (Buffer.byteLength(next, "utf8") <= maxBytes) {
return next;
}
return next.slice(0, maxBytes) + "\n[truncated]\n";
}
function runCrabbox(config, args, options = {}) {
const timeoutSeconds = options.timeoutSeconds ?? config.timeoutSeconds;
const maxOutputBytes = config.maxOutputBytes;
return new Promise((resolve, reject) => {
const child = spawn(config.binary, args, {
env: { ...process.env, ...options.env },
stdio: ["ignore", "pipe", "pipe"],
});
let stdout = "";
let stderr = "";
let didTimeout = false;
const timer = setTimeout(() => {
didTimeout = true;
child.kill("SIGTERM");
}, timeoutSeconds * 1000);
const abort = () => child.kill("SIGTERM");
options.signal?.addEventListener("abort", abort, { once: true });
child.stdout?.setEncoding("utf8");
child.stderr?.setEncoding("utf8");
child.stdout?.on("data", (chunk) => {
stdout = appendChunk(stdout, chunk, maxOutputBytes);
});
child.stderr?.on("data", (chunk) => {
stderr = appendChunk(stderr, chunk, maxOutputBytes);
});
child.on("error", (error) => {
clearTimeout(timer);
options.signal?.removeEventListener("abort", abort);
reject(error);
});
child.on("close", (code, signal) => {
clearTimeout(timer);
options.signal?.removeEventListener("abort", abort);
const result = {
ok: code === 0 && !didTimeout,
code,
signal,
timedOut: didTimeout,
stdout,
stderr,
command: commandLine(config.binary, args),
};
resolve(result);
});
});
}
function formatResult(result) {
const parts = [`$ ${result.command}`, `exit=${result.code ?? "signal"}${result.signal ? ` signal=${result.signal}` : ""}`];
if (result.timedOut) {
parts.push("timed out");
}
if (result.stdout.trim()) {
parts.push(`stdout:\n${result.stdout.trimEnd()}`);
}
if (result.stderr.trim()) {
parts.push(`stderr:\n${result.stderr.trimEnd()}`);
}
return parts.join("\n\n");
}
async function execute(config, args, params, signal) {
const timeoutSeconds = readPositiveInteger(params, "timeoutSeconds", config.timeoutSeconds);
const result = await runCrabbox(config, args, {
env: readEnv(params),
signal,
timeoutSeconds,
});
return toolResult(formatResult(result), result);
}
function registerRun(api, config) {
api.registerTool({
name: "crabbox_run",
description: "Run a command on an existing Crabbox lease after syncing the current repository.",
parameters: {
type: "object",
additionalProperties: false,
required: ["id", "command"],
properties: {
id: {
type: "string",
description: "Crabbox lease ID or friendly slug.",
},
command: commandArraySchema,
env: envSchema,
noSync: {
type: "boolean",
description: "Pass --no-sync.",
},
syncOnly: {
type: "boolean",
description: "Pass --sync-only.",
},
forceSyncLarge: {
type: "boolean",
description: "Pass --force-sync-large.",
},
checksum: {
type: "boolean",
description: "Pass --checksum.",
},
debug: {
type: "boolean",
description: "Pass --debug.",
},
reclaim: {
type: "boolean",
description: "Pass --reclaim.",
},
junit: {
type: "string",
description: "Comma-separated remote JUnit XML paths.",
},
timeoutSeconds: {
type: "number",
description: "Local wrapper timeout for this Crabbox CLI invocation.",
},
},
},
async execute(_toolCallId, params, signal) {
if (!config.allowRun) {
throw new Error("crabbox_run is disabled by plugin config");
}
const args = ["run", "--id", readString(params, "id")];
maybePushBool(args, "--no-sync", params?.noSync);
maybePushBool(args, "--sync-only", params?.syncOnly);
maybePushBool(args, "--force-sync-large", params?.forceSyncLarge);
maybePushBool(args, "--checksum", params?.checksum);
maybePushBool(args, "--debug", params?.debug);
maybePushBool(args, "--reclaim", params?.reclaim);
maybePush(args, "--junit", readString(params, "junit"));
args.push("--", ...readStringArray(params, "command"));
return execute(config, args, params, signal);
},
});
}
function registerWarmup(api, config) {
api.registerTool({
name: "crabbox_warmup",
description: "Provision or reuse a Crabbox lease and wait until it is ready.",
parameters: {
type: "object",
additionalProperties: false,
properties: {
provider: providerSchema,
profile: { type: "string" },
class: { type: "string" },
type: {
type: "string",
description: "Provider server or instance type.",
},
ttl: {
type: "string",
description: "Maximum lease lifetime, for example 90m.",
},
idleTimeout: {
type: "string",
description: "Idle timeout, for example 30m.",
},
keep: {
type: "boolean",
description: "Pass --keep.",
},
actionsRunner: {
type: "boolean",
description: "Pass --actions-runner.",
},
reclaim: {
type: "boolean",
description: "Pass --reclaim.",
},
timeoutSeconds: {
type: "number",
description: "Local wrapper timeout for this Crabbox CLI invocation.",
},
},
},
async execute(_toolCallId, params, signal) {
if (!config.allowWarmup) {
throw new Error("crabbox_warmup is disabled by plugin config");
}
const args = ["warmup"];
maybePush(args, "--provider", readString(params, "provider"));
maybePush(args, "--profile", readString(params, "profile"));
maybePush(args, "--class", readString(params, "class"));
maybePush(args, "--type", readString(params, "type"));
maybePush(args, "--ttl", readString(params, "ttl"));
maybePush(args, "--idle-timeout", readString(params, "idleTimeout"));
maybePushBool(args, "--keep", params?.keep);
maybePushBool(args, "--actions-runner", params?.actionsRunner);
maybePushBool(args, "--reclaim", params?.reclaim);
return execute(config, args, params, signal);
},
});
}
function registerStatus(api, config) {
api.registerTool({
name: "crabbox_status",
description: "Read the current state for a Crabbox lease.",
parameters: {
type: "object",
additionalProperties: false,
required: ["id"],
properties: {
id: { type: "string" },
provider: providerSchema,
wait: { type: "boolean" },
waitTimeout: {
type: "string",
description: "Maximum wait duration, for example 10m.",
},
json: { type: "boolean" },
timeoutSeconds: {
type: "number",
description: "Local wrapper timeout for this Crabbox CLI invocation.",
},
},
},
async execute(_toolCallId, params, signal) {
const args = ["status", "--id", readString(params, "id")];
maybePush(args, "--provider", readString(params, "provider"));
maybePushBool(args, "--wait", params?.wait);
maybePush(args, "--wait-timeout", readString(params, "waitTimeout"));
maybePushBool(args, "--json", params?.json);
return execute(config, args, params, signal);
},
});
}
function registerList(api, config) {
api.registerTool({
name: "crabbox_list",
description: "List current Crabbox machines.",
parameters: {
type: "object",
additionalProperties: false,
properties: {
provider: providerSchema,
json: { type: "boolean" },
timeoutSeconds: {
type: "number",
description: "Local wrapper timeout for this Crabbox CLI invocation.",
},
},
},
async execute(_toolCallId, params, signal) {
const args = ["list"];
maybePush(args, "--provider", readString(params, "provider"));
maybePushBool(args, "--json", params?.json);
return execute(config, args, params, signal);
},
});
}
function registerStop(api, config) {
api.registerTool({
name: "crabbox_stop",
description: "Stop a kept Crabbox lease by ID or friendly slug.",
parameters: {
type: "object",
additionalProperties: false,
required: ["id"],
properties: {
id: { type: "string" },
provider: providerSchema,
timeoutSeconds: {
type: "number",
description: "Local wrapper timeout for this Crabbox CLI invocation.",
},
},
},
async execute(_toolCallId, params, signal) {
if (!config.allowStop) {
throw new Error("crabbox_stop is disabled by plugin config");
}
const args = ["stop"];
maybePush(args, "--provider", readString(params, "provider"));
args.push(readString(params, "id"));
return execute(config, args, params, signal);
},
});
}
export default {
id: PLUGIN_ID,
name: "Crabbox",
description: "Run Crabbox remote testbox checks from OpenClaw.",
register(api) {
const config = readConfig(api);
registerRun(api, config);
registerWarmup(api, config);
registerStatus(api, config);
registerList(api, config);
registerStop(api, config);
api.logger?.info?.("Crabbox plugin registered");
},
};

View File

@ -1,101 +0,0 @@
import assert from "node:assert/strict";
import fs from "node:fs";
import os from "node:os";
import path from "node:path";
import test from "node:test";
import plugin from "./index.js";
function createFakeCrabbox() {
const dir = fs.mkdtempSync(path.join(os.tmpdir(), "crabbox-plugin-"));
const file = path.join(dir, "crabbox-fake.js");
fs.writeFileSync(
file,
`#!/usr/bin/env node
const payload = { argv: process.argv.slice(2), env: { CRABBOX_TEST_VALUE: process.env.CRABBOX_TEST_VALUE } };
process.stdout.write(JSON.stringify(payload));
if (process.env.CRABBOX_FAKE_EXIT) process.exit(Number(process.env.CRABBOX_FAKE_EXIT));
`,
"utf8",
);
fs.chmodSync(file, 0o755);
return { dir, file };
}
function registerWithConfig(pluginConfig) {
const tools = [];
plugin.register({
pluginConfig,
registerTool(tool) {
tools.push(tool);
},
logger: { info() {} },
});
return tools;
}
function getTool(tools, name) {
const tool = tools.find((entry) => entry.name === name);
assert.ok(tool, `expected ${name} to be registered`);
return tool;
}
test("registers the Crabbox tool surface", () => {
const tools = registerWithConfig({});
assert.deepEqual(
tools.map((tool) => tool.name).sort(),
["crabbox_list", "crabbox_run", "crabbox_status", "crabbox_stop", "crabbox_warmup"],
);
});
test("crabbox_run executes the CLI without shell wrapping", async () => {
const fake = createFakeCrabbox();
const tools = registerWithConfig({ binary: fake.file });
const result = await getTool(tools, "crabbox_run").execute("call-1", {
id: "blue-lobster",
command: ["go", "test", "./..."],
env: { CRABBOX_TEST_VALUE: "present" },
});
assert.equal(result.details.code, 0);
assert.deepEqual(JSON.parse(result.details.stdout).argv, [
"run",
"--id",
"blue-lobster",
"--",
"go",
"test",
"./...",
]);
assert.equal(JSON.parse(result.details.stdout).env.CRABBOX_TEST_VALUE, "present");
});
test("crabbox_status includes optional flags", async () => {
const fake = createFakeCrabbox();
const tools = registerWithConfig({ binary: fake.file });
const result = await getTool(tools, "crabbox_status").execute("call-1", {
id: "cbx_123",
wait: true,
waitTimeout: "10m",
json: true,
});
assert.deepEqual(JSON.parse(result.details.stdout).argv, [
"status",
"--id",
"cbx_123",
"--wait",
"--wait-timeout",
"10m",
"--json",
]);
});
test("disabled run tool fails before invoking crabbox", async () => {
const fake = createFakeCrabbox();
const tools = registerWithConfig({ binary: fake.file, allowRun: false });
await assert.rejects(
getTool(tools, "crabbox_run").execute("call-1", {
id: "blue-lobster",
command: ["go", "test", "./..."],
}),
/disabled/,
);
});

View File

@ -109,7 +109,8 @@ func (a App) actionsHydrate(ctx context.Context, args []string) error {
return err
}
ref := actionsRef(cfg, repo)
fields := actionsHydrateFields(leaseID, label, cfg.Actions.Job, *keepAliveMinutes, fieldFlags)
extraFields := mergeWorkflowInputFields(cfg.Actions.Fields, fieldFlags)
fields := actionsHydrateFields(leaseID, label, cfg.Actions.Job, *keepAliveMinutes, extraFields)
if inputs, ok, err := githubWorkflowDispatchInputs(ctx, repo.Root, ghRepo, cfg.Actions.Workflow, ref); err != nil {
fmt.Fprintf(a.Stderr, "warning: inspect workflow inputs failed: %v\n", err)
} else if ok {
@ -313,6 +314,28 @@ func actionsHydrateFields(leaseID, label, job string, keepAliveMinutes int, extr
return fields
}
func mergeWorkflowInputFields(base, override []string) []string {
fields := append([]string{}, base...)
index := map[string]int{}
for i, field := range fields {
if name := fieldName(field); name != "" {
index[name] = i
}
}
for _, field := range override {
name := fieldName(field)
if name != "" {
if existing, ok := index[name]; ok {
fields[existing] = field
continue
}
index[name] = len(fields)
}
fields = append(fields, field)
}
return fields
}
func githubWorkflowDispatchInputs(ctx context.Context, dir string, repo GitHubRepo, workflow, ref string) (map[string]bool, bool, error) {
workflow = strings.TrimPrefix(workflow, "/")
if !strings.HasPrefix(workflow, ".github/workflows/") {
@ -601,7 +624,7 @@ if [ ! -x ./config.sh ] || [ ! -f ".crabbox-runner-version-$version-$runner_arch
touch ".crabbox-runner-version-$version-$runner_arch"
fi
if [ -f .runner ]; then
./config.sh remove --unattended --token "$RUNNER_TOKEN" || true
./config.sh remove --token "$RUNNER_TOKEN" || true
fi
sudo ./bin/installdependencies.sh >/tmp/crabbox-actions-runner-deps.log 2>&1 || true
./config.sh --unattended --replace %s --url "https://github.com/${RUNNER_REPO}" --token "$RUNNER_TOKEN" --name "$RUNNER_NAME" --labels "$RUNNER_LABELS"

View File

@ -46,6 +46,17 @@ func TestActionsHydrateFieldsOmitsEmptyJobForOldWorkflows(t *testing.T) {
}
}
func TestMergeWorkflowInputFieldsLetsFlagsOverrideConfig(t *testing.T) {
got := mergeWorkflowInputFields(
[]string{"crabbox_docker_cache=false", "crabbox_prepare_images=1"},
[]string{"crabbox_docker_cache=true", "custom=value"},
)
want := []string{"crabbox_docker_cache=true", "crabbox_prepare_images=1", "custom=value"}
if strings.Join(got, "\n") != strings.Join(want, "\n") {
t.Fatalf("fields=%#v want %#v", got, want)
}
}
func TestFilterWorkflowInputsDropsUndeclaredOptionalInputs(t *testing.T) {
fields := actionsHydrateFields("cbx_123", "crabbox-cbx-123", "hydrate", 90, []string{"custom=value"})
filtered, dropped := filterWorkflowInputs(fields, map[string]bool{
@ -115,6 +126,7 @@ func TestGitHubActionsRunnerInstallScriptUsesOfficialRunner(t *testing.T) {
for _, want := range []string{
"https://api.github.com/repos/actions/runner/releases/latest",
"https://github.com/actions/runner/releases/download/",
"./config.sh remove --token",
"./config.sh --unattended --replace --ephemeral",
"crabbox-actions-runner.service",
} {
@ -122,6 +134,9 @@ func TestGitHubActionsRunnerInstallScriptUsesOfficialRunner(t *testing.T) {
t.Fatalf("install script missing %q", want)
}
}
if strings.Contains(got, "remove --unattended") {
t.Fatalf("install script should not pass --unattended to config.sh remove")
}
}
func TestParseActionsHydrationState(t *testing.T) {

View File

@ -58,6 +58,8 @@ func (a App) Run(ctx context.Context, args []string) error {
return a.results(ctx, args[1:])
case "cache":
return a.cache(ctx, args[1:])
case "image":
return a.image(ctx, args[1:])
case "config":
return a.config(ctx, args[1:])
case "init":
@ -125,6 +127,7 @@ Commands:
logs Print recorded run logs
results Show recorded test result summaries
cache Inspect, purge, or warm remote caches
image Inspect and create AWS runner AMIs
status Show lease state; add --wait to block until ready
list List Crabbox machines
usage Show cost and usage estimates by user, org, or fleet
@ -147,6 +150,10 @@ Common Flows:
crabbox logs run_123
crabbox results run_123
crabbox cache stats --id blue-lobster
crabbox image current
crabbox image list
crabbox image create --id blue-lobster --name openclaw-crabbox-20260501
crabbox image promote ami-0123456789abcdef0
crabbox usage --scope org
crabbox admin leases --state active
crabbox warmup --actions-runner

View File

@ -304,9 +304,121 @@ func (c *AWSClient) SetTags(ctx context.Context, id string, labels map[string]st
return err
}
type AWSImage struct {
ID string `json:"id"`
Name string `json:"name,omitempty"`
Description string `json:"description,omitempty"`
State string `json:"state,omitempty"`
OwnerID string `json:"ownerId,omitempty"`
CreationDate string `json:"creationDate,omitempty"`
Public bool `json:"public,omitempty"`
Source string `json:"source,omitempty"`
Region string `json:"region,omitempty"`
Tags map[string]string `json:"tags,omitempty"`
}
func (c *AWSClient) CurrentImage(ctx context.Context, cfg Config) (AWSImage, error) {
imageID, source, err := c.currentImageID(ctx, cfg)
if err != nil {
return AWSImage{}, err
}
image, err := c.describeImage(ctx, imageID)
if err != nil {
return AWSImage{}, err
}
image.Source = source
image.Region = cfg.AWSRegion
return image, nil
}
func (c *AWSClient) ListImages(ctx context.Context, cfg Config, name string) ([]AWSImage, error) {
filters := []types.Filter{
{Name: aws.String("state"), Values: []string{"available", "pending"}},
}
if name != "" {
filters = append(filters, types.Filter{Name: aws.String("name"), Values: []string{name}})
} else {
filters = append(filters,
types.Filter{Name: aws.String("tag:crabbox"), Values: []string{"true"}},
types.Filter{Name: aws.String("tag:crabbox_image"), Values: []string{"true"}},
)
}
out, err := c.ec2.DescribeImages(ctx, &ec2.DescribeImagesInput{
Owners: []string{"self"},
Filters: filters,
})
if err != nil {
return nil, err
}
images := make([]AWSImage, 0, len(out.Images))
for _, image := range out.Images {
view := awsImageToView(image)
view.Region = cfg.AWSRegion
images = append(images, view)
}
sort.Slice(images, func(i, j int) bool {
return images[i].CreationDate > images[j].CreationDate
})
return images, nil
}
func (c *AWSClient) CreateImage(ctx context.Context, cfg Config, instanceID, name, description, leaseID, slug string, noReboot, wait bool) (AWSImage, error) {
if name == "" {
return AWSImage{}, exit(2, "--name is required")
}
now := time.Now().UTC()
tags := map[string]string{
"Name": name,
"crabbox": "true",
"crabbox_image": "true",
"created_by": "crabbox",
"provider": "aws",
"region": cfg.AWSRegion,
"created_at": now.Format(time.RFC3339),
}
if leaseID != "" {
tags["source_lease"] = leaseID
}
if slug != "" {
tags["source_slug"] = slug
}
out, err := c.ec2.CreateImage(ctx, &ec2.CreateImageInput{
Description: aws.String(description),
InstanceId: aws.String(instanceID),
Name: aws.String(name),
NoReboot: aws.Bool(noReboot),
TagSpecifications: []types.TagSpecification{
{ResourceType: types.ResourceTypeImage, Tags: awsTags(tags)},
{ResourceType: types.ResourceTypeSnapshot, Tags: awsTags(tags)},
},
})
if err != nil {
return AWSImage{}, err
}
imageID := aws.ToString(out.ImageId)
if wait {
waiter := ec2.NewImageAvailableWaiter(c.ec2)
if err := waiter.Wait(ctx, &ec2.DescribeImagesInput{ImageIds: []string{imageID}}, 45*time.Minute); err != nil {
return AWSImage{}, err
}
}
image, err := c.describeImage(ctx, imageID)
if err != nil {
return AWSImage{ID: imageID, Name: name, State: "pending", Source: "created", Region: cfg.AWSRegion, Tags: tags}, nil
}
image.Source = "created"
image.Region = cfg.AWSRegion
return image, nil
}
func (c *AWSClient) resolveAMI(ctx context.Context, cfg Config) (string, error) {
imageID, _, err := c.currentImageID(ctx, cfg)
return imageID, err
}
func (c *AWSClient) currentImageID(ctx context.Context, cfg Config) (string, string, error) {
if cfg.AWSAMI != "" {
return cfg.AWSAMI, nil
return cfg.AWSAMI, "config", nil
}
out, err := c.ec2.DescribeImages(ctx, &ec2.DescribeImagesInput{
Owners: []string{awsUbuntuOwner},
@ -318,15 +430,45 @@ func (c *AWSClient) resolveAMI(ctx context.Context, cfg Config) (string, error)
},
})
if err != nil {
return "", err
return "", "", err
}
if len(out.Images) == 0 {
return "", exit(3, "no Ubuntu 24.04 x86_64 AMI found in %s; set CRABBOX_AWS_AMI", cfg.AWSRegion)
return "", "", exit(3, "no Ubuntu 24.04 x86_64 AMI found in %s; set CRABBOX_AWS_AMI", cfg.AWSRegion)
}
sort.Slice(out.Images, func(i, j int) bool {
return aws.ToString(out.Images[i].CreationDate) > aws.ToString(out.Images[j].CreationDate)
})
return aws.ToString(out.Images[0].ImageId), nil
return aws.ToString(out.Images[0].ImageId), "ubuntu-default", nil
}
func (c *AWSClient) describeImage(ctx context.Context, imageID string) (AWSImage, error) {
out, err := c.ec2.DescribeImages(ctx, &ec2.DescribeImagesInput{
ImageIds: []string{imageID},
})
if err != nil {
return AWSImage{}, err
}
if len(out.Images) == 0 {
return AWSImage{}, exit(4, "aws image not found: %s", imageID)
}
return awsImageToView(out.Images[0]), nil
}
func awsImageToView(image types.Image) AWSImage {
tags := make(map[string]string)
for _, tag := range image.Tags {
tags[aws.ToString(tag.Key)] = aws.ToString(tag.Value)
}
return AWSImage{
ID: aws.ToString(image.ImageId),
Name: aws.ToString(image.Name),
Description: aws.ToString(image.Description),
State: string(image.State),
OwnerID: aws.ToString(image.OwnerId),
CreationDate: aws.ToString(image.CreationDate),
Public: aws.ToBool(image.Public),
Tags: tags,
}
}
func (c *AWSClient) ensureSecurityGroup(ctx context.Context, cfg Config) (string, error) {

View File

@ -71,6 +71,7 @@ type ActionsConfig struct {
Workflow string
Job string
Ref string
Fields []string
RunnerLabels []string
RunnerVersion string
Ephemeral bool
@ -261,6 +262,7 @@ type fileActionsConfig struct {
Workflow string `yaml:"workflow,omitempty"`
Job string `yaml:"job,omitempty"`
Ref string `yaml:"ref,omitempty"`
Fields []string `yaml:"fields,omitempty"`
RunnerLabels []string `yaml:"runnerLabels,omitempty"`
RunnerVersion string `yaml:"runnerVersion,omitempty"`
Ephemeral *bool `yaml:"ephemeral,omitempty"`
@ -525,6 +527,9 @@ func applyFileConfig(cfg *Config, file fileConfig) {
if file.Actions.Ref != "" {
cfg.Actions.Ref = file.Actions.Ref
}
if len(file.Actions.Fields) > 0 {
cfg.Actions.Fields = appendUniqueStrings(nil, file.Actions.Fields...)
}
if len(file.Actions.RunnerLabels) > 0 {
cfg.Actions.RunnerLabels = appendUniqueStrings(nil, file.Actions.RunnerLabels...)
}

View File

@ -60,6 +60,9 @@ actions:
workflow: .github/workflows/crabbox.yml
job: hydrate
ref: main
fields:
- crabbox_docker_cache=true
- crabbox_prepare_images=1
runnerLabels:
- crabbox
- linux-large
@ -131,6 +134,9 @@ ssh:
if cfg.Actions.Repo != "openclaw/crabbox" || cfg.Actions.Workflow != ".github/workflows/crabbox.yml" || cfg.Actions.Job != "hydrate" || cfg.Actions.Ref != "main" {
t.Fatalf("actions config not loaded: %#v", cfg.Actions)
}
if len(cfg.Actions.Fields) != 2 || cfg.Actions.Fields[0] != "crabbox_docker_cache=true" || cfg.Actions.Fields[1] != "crabbox_prepare_images=1" {
t.Fatalf("actions fields config not loaded: %#v", cfg.Actions.Fields)
}
if cfg.Actions.Ephemeral || len(cfg.Actions.RunnerLabels) != 2 || cfg.Actions.RunnerLabels[1] != "linux-large" {
t.Fatalf("actions runner config not loaded: %#v", cfg.Actions)
}

View File

@ -71,6 +71,14 @@ type CoordinatorWhoami struct {
Auth string `json:"auth"`
}
type CoordinatorImageResponse struct {
Image AWSImage `json:"image"`
}
type CoordinatorImagesResponse struct {
Images []AWSImage `json:"images"`
}
type CoordinatorGitHubLoginStart struct {
LoginID string `json:"loginID"`
URL string `json:"url"`
@ -110,6 +118,11 @@ type CoordinatorRun struct {
SyncMs int64 `json:"syncMs,omitempty"`
CommandMs int64 `json:"commandMs,omitempty"`
DurationMs int64 `json:"durationMs,omitempty"`
SyncFiles int `json:"syncFiles,omitempty"`
SyncBytes int64 `json:"syncBytes,omitempty"`
SyncDeleted int `json:"syncDeleted,omitempty"`
SyncManifest int64 `json:"syncManifestBytes,omitempty"`
SyncSkipped bool `json:"syncSkipped,omitempty"`
LogBytes int64 `json:"logBytes"`
LogTruncated bool `json:"logTruncated"`
Results *TestResultSummary `json:"results,omitempty"`
@ -343,6 +356,61 @@ func (c *CoordinatorClient) Whoami(ctx context.Context) (CoordinatorWhoami, erro
return res, err
}
func (c *CoordinatorClient) CurrentAWSImage(ctx context.Context, region string) (AWSImage, error) {
var res CoordinatorImageResponse
values := url.Values{}
values.Set("provider", "aws")
if region != "" {
values.Set("region", region)
}
path := "/v1/images/current?" + values.Encode()
err := c.do(ctx, http.MethodGet, path, nil, &res)
return res.Image, err
}
func (c *CoordinatorClient) AWSImages(ctx context.Context, region, name string) ([]AWSImage, error) {
var res CoordinatorImagesResponse
values := url.Values{}
values.Set("provider", "aws")
if region != "" {
values.Set("region", region)
}
if name != "" {
values.Set("name", name)
}
path := "/v1/images?" + values.Encode()
err := c.do(ctx, http.MethodGet, path, nil, &res)
return res.Images, err
}
func (c *CoordinatorClient) CreateAWSImage(ctx context.Context, leaseID, name, description string, noReboot, wait bool) (AWSImage, error) {
var res CoordinatorImageResponse
err := c.do(ctx, http.MethodPost, "/v1/images", map[string]any{
"provider": "aws",
"leaseID": leaseID,
"name": name,
"description": description,
"noReboot": noReboot,
"wait": wait,
}, &res)
return res.Image, err
}
func (c *CoordinatorClient) PromoteAWSImage(ctx context.Context, region, imageID string) (AWSImage, error) {
var res CoordinatorImageResponse
values := url.Values{}
values.Set("provider", "aws")
if region != "" {
values.Set("region", region)
}
path := "/v1/images/promote?" + values.Encode()
err := c.do(ctx, http.MethodPost, path, map[string]any{
"provider": "aws",
"imageID": imageID,
}, &res)
return res.Image, err
}
func (c *CoordinatorClient) StartGitHubLogin(ctx context.Context, pollSecretHash, provider string) (CoordinatorGitHubLoginStart, error) {
var res CoordinatorGitHubLoginStart
body := map[string]any{"pollSecretHash": pollSecretHash}
@ -415,15 +483,42 @@ func (c *CoordinatorClient) CreateRun(ctx context.Context, leaseID string, cfg C
return res.Run, err
}
func (c *CoordinatorClient) FinishRun(ctx context.Context, runID string, exitCode int, sync, command time.Duration, log string, truncated bool, results *TestResultSummary) (CoordinatorRun, error) {
type RunFinishMetrics struct {
Sync time.Duration
Command time.Duration
SyncFiles int
SyncBytes int64
SyncDeleted int
SyncManifest int64
SyncSkipped bool
}
func runFinishMetricsFromTimings(timings runTimings) RunFinishMetrics {
return RunFinishMetrics{
Sync: timings.sync,
Command: timings.command,
SyncFiles: timings.syncStats.files,
SyncBytes: timings.syncStats.bytes,
SyncDeleted: timings.syncStats.deleted,
SyncManifest: timings.syncStats.manifestBytes,
SyncSkipped: timings.syncSkipped,
}
}
func (c *CoordinatorClient) FinishRun(ctx context.Context, runID string, exitCode int, metrics RunFinishMetrics, log string, truncated bool, results *TestResultSummary) (CoordinatorRun, error) {
var res CoordinatorRunResponse
err := c.do(ctx, http.MethodPost, "/v1/runs/"+url.PathEscape(runID)+"/finish", map[string]any{
"exitCode": exitCode,
"syncMs": sync.Milliseconds(),
"commandMs": command.Milliseconds(),
"log": log,
"logTruncated": truncated,
"results": results,
"exitCode": exitCode,
"syncMs": metrics.Sync.Milliseconds(),
"commandMs": metrics.Command.Milliseconds(),
"syncFiles": metrics.SyncFiles,
"syncBytes": metrics.SyncBytes,
"syncDeleted": metrics.SyncDeleted,
"syncManifestBytes": metrics.SyncManifest,
"syncSkipped": metrics.SyncSkipped,
"log": log,
"logTruncated": truncated,
"results": results,
}, &res)
return res.Run, err
}

View File

@ -115,6 +115,58 @@ func TestCoordinatorTouchAndUpdateHeartbeatBodies(t *testing.T) {
}
}
func TestCoordinatorImageMethods(t *testing.T) {
var gotMethod string
var gotPath string
var gotBody string
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
gotMethod = r.Method
gotPath = r.URL.String()
data, _ := io.ReadAll(r.Body)
gotBody = string(data)
w.Header().Set("Content-Type", "application/json")
switch {
case r.Method == http.MethodGet && strings.HasPrefix(r.URL.String(), "/v1/images/current?"):
_, _ = w.Write([]byte(`{"image":{"id":"ami-123","name":"current","source":"config","region":"eu-west-1"}}`))
case r.Method == http.MethodGet && strings.HasPrefix(r.URL.String(), "/v1/images?"):
_, _ = w.Write([]byte(`{"images":[{"id":"ami-123","name":"cached"}]}`))
case r.Method == http.MethodPost && r.URL.Path == "/v1/images":
_, _ = w.Write([]byte(`{"image":{"id":"ami-456","name":"created","source":"created"}}`))
case r.Method == http.MethodPost && strings.HasPrefix(r.URL.String(), "/v1/images/promote?"):
_, _ = w.Write([]byte(`{"image":{"id":"ami-789","name":"promoted","source":"promoted"}}`))
default:
t.Fatalf("unexpected request %s %s", r.Method, r.URL.String())
}
}))
defer server.Close()
client := CoordinatorClient{BaseURL: server.URL, Client: server.Client()}
image, err := client.CurrentAWSImage(context.Background(), "eu-west-1")
if err != nil || image.ID != "ami-123" || !strings.Contains(gotPath, "provider=aws") {
t.Fatalf("current image=%#v path=%s err=%v", image, gotPath, err)
}
images, err := client.AWSImages(context.Background(), "eu-west-1", "openclaw-*")
if err != nil || len(images) != 1 || images[0].ID != "ami-123" || !strings.Contains(gotPath, "name=openclaw-%2A") {
t.Fatalf("images=%#v path=%s err=%v", images, gotPath, err)
}
created, err := client.CreateAWSImage(context.Background(), "cbx_123", "openclaw-cache", "warm", true, false)
if err != nil || created.ID != "ami-456" || gotMethod != http.MethodPost {
t.Fatalf("created=%#v method=%s err=%v", created, gotMethod, err)
}
for _, want := range []string{`"leaseID":"cbx_123"`, `"name":"openclaw-cache"`, `"noReboot":true`} {
if !strings.Contains(gotBody, want) {
t.Fatalf("body missing %q: %s", want, gotBody)
}
}
promoted, err := client.PromoteAWSImage(context.Background(), "eu-west-1", "ami-789")
if err != nil || promoted.ID != "ami-789" || gotMethod != http.MethodPost || !strings.Contains(gotPath, "/v1/images/promote?") {
t.Fatalf("promoted=%#v method=%s path=%s err=%v", promoted, gotMethod, gotPath, err)
}
if !strings.Contains(gotBody, `"imageID":"ami-789"`) {
t.Fatalf("promote body missing imageID: %s", gotBody)
}
}
func TestCoordinatorCreateLeaseSendsAWSSSHCIDRs(t *testing.T) {
var body struct {
AWSSSHCIDRs []string `json:"awsSSHCIDRs"`

241
internal/cli/image.go Normal file
View File

@ -0,0 +1,241 @@
package cli
import (
"context"
"encoding/json"
"fmt"
"strings"
)
func (a App) image(ctx context.Context, args []string) error {
if len(args) == 0 {
return exit(2, "usage: crabbox image current|list|create|promote")
}
switch args[0] {
case "current":
return a.imageCurrent(ctx, args[1:])
case "list":
return a.imageList(ctx, args[1:])
case "create":
return a.imageCreate(ctx, args[1:])
case "promote":
return a.imagePromote(ctx, args[1:])
default:
return exit(2, "unknown image command %q", args[0])
}
}
func (a App) imageCurrent(ctx context.Context, args []string) error {
fs := newFlagSet("image current", a.Stderr)
provider := fs.String("provider", "aws", "provider: aws")
jsonOut := fs.Bool("json", false, "print JSON")
if err := parseFlags(fs, args); err != nil {
return err
}
cfg, err := imageAWSConfig(*provider)
if err != nil {
return err
}
image, err := currentAWSImage(ctx, cfg)
if err != nil {
return err
}
return printAWSImage(a.Stdout, image, *jsonOut)
}
func (a App) imageList(ctx context.Context, args []string) error {
fs := newFlagSet("image list", a.Stderr)
provider := fs.String("provider", "aws", "provider: aws")
name := fs.String("name", "", "AMI name glob; defaults to Crabbox-tagged images")
jsonOut := fs.Bool("json", false, "print JSON")
if err := parseFlags(fs, args); err != nil {
return err
}
cfg, err := imageAWSConfig(*provider)
if err != nil {
return err
}
images, err := listAWSImages(ctx, cfg, *name)
if err != nil {
return err
}
if *jsonOut {
return json.NewEncoder(a.Stdout).Encode(images)
}
for _, image := range images {
fmt.Fprintf(a.Stdout, "%-20s %-10s %-20s %s\n", image.ID, blank(image.State, "-"), blank(image.CreationDate, "-"), image.Name)
}
return nil
}
func (a App) imageCreate(ctx context.Context, args []string) error {
fs := newFlagSet("image create", a.Stderr)
provider := fs.String("provider", "aws", "provider: aws")
id := fs.String("id", "", "source lease id or slug")
name := fs.String("name", "", "AMI name")
description := fs.String("description", "", "AMI description")
noReboot := fs.Bool("no-reboot", false, "avoid rebooting before image capture; faster but less consistent")
wait := fs.Bool("wait", false, "wait until the AMI becomes available")
skipScrub := fs.Bool("skip-scrub", false, "skip best-effort secret scrub on the source runner")
jsonOut := fs.Bool("json", false, "print JSON")
if err := parseFlags(fs, args); err != nil {
return err
}
if *id == "" && fs.NArg() > 0 {
*id = fs.Arg(0)
}
if *id == "" || *name == "" {
return exit(2, "usage: crabbox image create --id <lease-id-or-slug> --name <ami-name>")
}
cfg, err := imageAWSConfig(*provider)
if err != nil {
return err
}
server, target, leaseID, err := a.resolveLeaseTarget(ctx, cfg, *id)
if err != nil {
return err
}
if server.Provider != "" && server.Provider != "aws" {
return exit(2, "image create only supports AWS leases, got provider=%s", server.Provider)
}
if server.CloudID == "" || !strings.HasPrefix(server.CloudID, "i-") {
return exit(2, "source lease does not have an AWS instance id")
}
if !*skipScrub {
fmt.Fprintf(a.Stderr, "scrubbing source runner before AMI capture lease=%s instance=%s\n", leaseID, server.CloudID)
if err := runSSHQuiet(ctx, target, remoteImageScrub()); err != nil {
return exit(7, "image source scrub failed; rerun with --skip-scrub only if the box contains no secrets: %v", err)
}
}
if coord, ok, err := newCoordinatorClient(cfg); err != nil {
return err
} else if ok {
image, err := coord.CreateAWSImage(ctx, leaseID, *name, *description, *noReboot, *wait)
if err != nil {
return err
}
return printAWSImage(a.Stdout, image, *jsonOut)
}
client, err := newAWSClient(ctx, cfg)
if err != nil {
return err
}
image, err := client.CreateImage(ctx, cfg, server.CloudID, *name, *description, leaseID, serverSlug(server), *noReboot, *wait)
if err != nil {
return err
}
return printAWSImage(a.Stdout, image, *jsonOut)
}
func (a App) imagePromote(ctx context.Context, args []string) error {
fs := newFlagSet("image promote", a.Stderr)
provider := fs.String("provider", "aws", "provider: aws")
jsonOut := fs.Bool("json", false, "print JSON")
if err := parseFlags(fs, args); err != nil {
return err
}
if *provider != "aws" {
return exit(2, "image promote only supports provider=aws")
}
if fs.NArg() != 1 {
return exit(2, "usage: crabbox image promote <ami-id>")
}
ami := fs.Arg(0)
if !strings.HasPrefix(ami, "ami-") {
return exit(2, "invalid AWS AMI id: %s", ami)
}
cfg, err := imageAWSConfig(*provider)
if err != nil {
return err
}
if coord, ok, err := newCoordinatorClient(cfg); err != nil {
return err
} else if ok {
image, err := coord.PromoteAWSImage(ctx, cfg.AWSRegion, ami)
if err != nil {
return err
}
return printAWSImage(a.Stdout, image, *jsonOut)
}
path := writableConfigPath()
if path == "" {
return exit(2, "user config directory is unavailable")
}
file, err := readFileConfig(path)
if err != nil {
return err
}
if file.AWS == nil {
file.AWS = &fileAWSConfig{}
}
file.AWS.AMI = ami
written, err := writeUserFileConfig(file)
if err != nil {
return err
}
if *jsonOut {
return json.NewEncoder(a.Stdout).Encode(map[string]string{"config": written, "provider": "aws", "ami": ami})
}
fmt.Fprintf(a.Stdout, "wrote %s aws.ami=%s\n", written, ami)
return nil
}
func imageAWSConfig(provider string) (Config, error) {
cfg, err := loadConfig()
if err != nil {
return Config{}, err
}
cfg.Provider = provider
if cfg.Provider != "aws" {
return Config{}, exit(2, "crabbox image only supports provider=aws")
}
return cfg, nil
}
func currentAWSImage(ctx context.Context, cfg Config) (AWSImage, error) {
if coord, ok, err := newCoordinatorClient(cfg); err != nil {
return AWSImage{}, err
} else if ok {
return coord.CurrentAWSImage(ctx, cfg.AWSRegion)
}
client, err := newAWSClient(ctx, cfg)
if err != nil {
return AWSImage{}, err
}
return client.CurrentImage(ctx, cfg)
}
func listAWSImages(ctx context.Context, cfg Config, name string) ([]AWSImage, error) {
if coord, ok, err := newCoordinatorClient(cfg); err != nil {
return nil, err
} else if ok {
return coord.AWSImages(ctx, cfg.AWSRegion, name)
}
client, err := newAWSClient(ctx, cfg)
if err != nil {
return nil, err
}
return client.ListImages(ctx, cfg, name)
}
func printAWSImage(out interface{ Write([]byte) (int, error) }, image AWSImage, jsonOut bool) error {
if jsonOut {
return json.NewEncoder(out).Encode(image)
}
fmt.Fprintf(out, "id=%s\nname=%s\nstate=%s\nsource=%s\nregion=%s\ncreated=%s\n", image.ID, image.Name, image.State, blank(image.Source, "-"), blank(image.Region, "-"), blank(image.CreationDate, "-"))
return nil
}
func remoteImageScrub() string {
return `set -eu
sudo systemctl stop actions.runner.* 2>/dev/null || true
sudo rm -rf /root/.aws /root/.docker /home/*/.aws /home/*/.docker 2>/dev/null || true
sudo find /work/crabbox -path '*/.crabbox/actions/*.env.sh' -type f -delete 2>/dev/null || true
sudo find /work/crabbox -name '.env' -type f -delete 2>/dev/null || true
history -c 2>/dev/null || true
sudo rm -f /root/.*history /home/*/.*history 2>/dev/null || true
sudo cloud-init clean --logs 2>/dev/null || true
sudo journalctl --rotate 2>/dev/null || true
sudo journalctl --vacuum-time=1s 2>/dev/null || true
sync`
}

View File

@ -0,0 +1,49 @@
package cli
import (
"bytes"
"os"
"path/filepath"
"strings"
"testing"
)
func TestRemoteImageScrubRemovesCommonSecretStores(t *testing.T) {
script := remoteImageScrub()
for _, want := range []string{
"/root/.aws",
"/home/*/.aws",
"/root/.docker",
"/.crabbox/actions/*.env.sh",
"cloud-init clean --logs",
"journalctl --vacuum-time=1s",
} {
if !strings.Contains(script, want) {
t.Fatalf("scrub script missing %q:\n%s", want, script)
}
}
}
func TestImagePromoteWritesAWSAMIToConfig(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "config.yaml")
t.Setenv("CRABBOX_CONFIG", path)
if err := os.WriteFile(path, []byte("provider: aws\naws:\n region: eu-west-1\n"), 0o600); err != nil {
t.Fatal(err)
}
var stdout bytes.Buffer
app := App{Stdout: &stdout, Stderr: &bytes.Buffer{}}
if err := app.imagePromote(t.Context(), []string{"ami-0123456789abcdef0"}); err != nil {
t.Fatal(err)
}
cfg, err := readFileConfig(path)
if err != nil {
t.Fatal(err)
}
if cfg.AWS == nil || cfg.AWS.AMI != "ami-0123456789abcdef0" {
t.Fatalf("aws ami not written: %#v", cfg.AWS)
}
if !strings.Contains(stdout.String(), "aws.ami=ami-0123456789abcdef0") {
t.Fatalf("stdout=%q", stdout.String())
}
}

View File

@ -281,6 +281,11 @@ func (a App) runCommand(ctx context.Context, args []string) error {
if err := checkSyncPreflight(manifest, cfg, *forceSyncLarge, a.Stderr); err != nil {
return err
}
timings.syncStats = syncStats{
files: len(manifest.Files),
bytes: manifest.Bytes,
deleted: len(manifest.Deleted),
}
timings.syncSteps.preflight = time.Since(stepStart)
fingerprint := ""
if cfg.Sync.Fingerprint {
@ -309,11 +314,13 @@ func (a App) runCommand(ctx context.Context, args []string) error {
timings.syncSteps.gitSeed = time.Since(stepStart)
}
manifestData := manifest.NUL()
deletedManifestData := manifest.DeletedNUL()
timings.syncStats.manifestBytes = int64(len(manifestData) + len(deletedManifestData))
stepStart = time.Now()
if err := runSSHInputQuiet(ctx, target, remoteWriteSyncManifestNew(workdir), string(manifestData)); err != nil {
if err := runSSHInputQuietBytes(ctx, target, remoteWriteSyncManifestNew(workdir), manifestData); err != nil {
return exit(7, "write sync manifest: %v", err)
}
if err := runSSHInputQuiet(ctx, target, remoteWriteSyncDeletedNew(workdir), string(manifest.DeletedNUL())); err != nil {
if err := runSSHInputQuietBytes(ctx, target, remoteWriteSyncDeletedNew(workdir), deletedManifestData); err != nil {
return exit(7, "write sync delete manifest: %v", err)
}
timings.syncSteps.manifestWrite = time.Since(stepStart)
@ -409,7 +416,7 @@ afterSync:
}
}
if runID != "" {
if _, err := coord.FinishRun(context.Background(), runID, code, timings.sync, timings.command, logBuffer.String(), logBuffer.Truncated(), results); err != nil {
if _, err := coord.FinishRun(context.Background(), runID, code, runFinishMetricsFromTimings(timings), logBuffer.String(), logBuffer.Truncated(), results); err != nil {
fmt.Fprintf(a.Stderr, "warning: run history finish failed for %s: %v\n", runID, err)
}
}
@ -427,9 +434,17 @@ type runTimings struct {
sync time.Duration
command time.Duration
syncSteps syncStepTimings
syncStats syncStats
syncSkipped bool
}
type syncStats struct {
files int
bytes int64
deleted int
manifestBytes int64
}
type syncStepTimings struct {
sshReady time.Duration
mkdir time.Duration
@ -458,9 +473,29 @@ func formatRunSummary(timings runTimings, total time.Duration, exitCode int) str
if breakdown := formatSyncStepTimings(timings.syncSteps); breakdown != "" {
summary += " sync_steps=" + breakdown
}
if stats := formatSyncStats(timings.syncStats); stats != "" {
summary += " " + stats
}
return summary
}
func formatSyncStats(stats syncStats) string {
if stats.files == 0 && stats.bytes == 0 && stats.deleted == 0 && stats.manifestBytes == 0 {
return ""
}
parts := []string{
fmt.Sprintf("sync_files=%d", stats.files),
fmt.Sprintf("sync_bytes=%s", humanBytes(stats.bytes)),
}
if stats.deleted > 0 {
parts = append(parts, fmt.Sprintf("sync_deleted=%d", stats.deleted))
}
if stats.manifestBytes > 0 {
parts = append(parts, fmt.Sprintf("sync_manifest=%s", humanBytes(stats.manifestBytes)))
}
return strings.Join(parts, " ")
}
func formatSyncStepTimings(steps syncStepTimings) string {
parts := make([]string, 0, 14)
appendStep := func(name string, duration time.Duration) {

View File

@ -14,6 +14,12 @@ func TestFormatRunSummary(t *testing.T) {
manifest: 20 * time.Millisecond,
rsync: 900 * time.Millisecond,
},
syncStats: syncStats{
files: 10,
bytes: 1536,
deleted: 1,
manifestBytes: 120,
},
syncSkipped: true,
}, 5*time.Second, 7)
for _, want := range []string{
@ -24,6 +30,10 @@ func TestFormatRunSummary(t *testing.T) {
"sync_skipped=true",
"exit=7",
"sync_steps=manifest:20ms,rsync:900ms",
"sync_files=10",
"sync_bytes=1.5 KiB",
"sync_deleted=1",
"sync_manifest=120 B",
} {
if !strings.Contains(got, want) {
t.Fatalf("summary missing %q in %q", want, got)

View File

@ -125,8 +125,16 @@ func runSSHCombinedOutput(ctx context.Context, target SSHTarget, remote string)
}
func runSSHInputQuiet(ctx context.Context, target SSHTarget, remote, input string) error {
return runSSHInputQuietReader(ctx, target, remote, strings.NewReader(input))
}
func runSSHInputQuietBytes(ctx context.Context, target SSHTarget, remote string, input []byte) error {
return runSSHInputQuietReader(ctx, target, remote, bytes.NewReader(input))
}
func runSSHInputQuietReader(ctx context.Context, target SSHTarget, remote string, input io.Reader) error {
cmd := exec.CommandContext(ctx, "ssh", sshArgs(target, remote)...)
cmd.Stdin = strings.NewReader(input)
cmd.Stdin = input
cmd.Stdout = io.Discard
cmd.Stderr = io.Discard
return cmd.Run()

View File

@ -1,47 +0,0 @@
{
"id": "crabbox",
"name": "Crabbox",
"description": "Run Crabbox remote testbox checks from OpenClaw.",
"activation": {
"onStartup": true
},
"contracts": {
"tools": ["crabbox_run", "crabbox_warmup", "crabbox_status", "crabbox_list", "crabbox_stop"]
},
"configSchema": {
"type": "object",
"additionalProperties": false,
"properties": {
"binary": {
"type": "string",
"description": "Crabbox executable path.",
"default": "crabbox"
},
"maxOutputBytes": {
"type": "number",
"description": "Maximum captured stdout/stderr bytes returned to the model.",
"default": 60000
},
"timeoutSeconds": {
"type": "number",
"description": "Default local wrapper timeout for Crabbox CLI invocations.",
"default": 1800
},
"allowRun": {
"type": "boolean",
"description": "Allow the crabbox_run tool.",
"default": true
},
"allowWarmup": {
"type": "boolean",
"description": "Allow the crabbox_warmup tool.",
"default": true
},
"allowStop": {
"type": "boolean",
"description": "Allow the crabbox_stop tool.",
"default": true
}
}
}
}

View File

@ -1,29 +0,0 @@
{
"name": "@openclaw/crabbox-plugin",
"version": "0.2.0",
"description": "OpenClaw plugin for running Crabbox remote testbox workflows",
"license": "MIT",
"type": "module",
"main": "index.js",
"openclaw": {
"extensions": [
"./index.js"
],
"compat": {
"pluginApi": ">=2026.4.25"
}
},
"scripts": {
"check": "node --check index.js && node --test index.test.js",
"test": "node --test index.test.js"
},
"files": [
"index.js",
"openclaw.plugin.json",
"README.md",
"LICENSE"
],
"engines": {
"node": ">=22.0.0"
}
}

View File

@ -5,7 +5,7 @@ import { cloudInit } from "./bootstrap";
import { awsInstanceTypeCandidatesForClass, validCIDRs, type LeaseConfig } from "./config";
import { leaseProviderLabels } from "./provider-labels";
import { leaseProviderName } from "./slug";
import type { Env, ProviderMachine } from "./types";
import type { AWSImageView, Env, ProviderMachine } from "./types";
const awsUbuntuOwner = "099720109477";
const ec2Version = "2016-11-15";
@ -163,6 +163,84 @@ export class EC2SpotClient {
});
}
async currentImage(config: LeaseConfig): Promise<AWSImageView> {
const { imageID, source } = await this.currentImageID(config);
const image = await this.describeImage(imageID);
return { ...image, source, region: this.region };
}
async listImages(name: string): Promise<AWSImageView[]> {
const params: Record<string, string> = {
"Owner.1": "self",
"Filter.1.Name": "state",
"Filter.1.Value.1": "available",
"Filter.1.Value.2": "pending",
};
if (name) {
params["Filter.2.Name"] = "name";
params["Filter.2.Value.1"] = name;
} else {
params["Filter.2.Name"] = "tag:crabbox";
params["Filter.2.Value.1"] = "true";
params["Filter.3.Name"] = "tag:crabbox_image";
params["Filter.3.Value.1"] = "true";
}
const root = await this.ec2("DescribeImages", params);
return items(record(root["imagesSet"])["item"])
.map((item) => ({ ...imageToView(item), region: this.region }))
.toSorted((left, right) => (right.creationDate ?? "").localeCompare(left.creationDate ?? ""));
}
async createImage(
instanceID: string,
name: string,
description: string,
leaseID: string,
slug: string,
noReboot: boolean,
wait: boolean,
): Promise<AWSImageView> {
const now = new Date().toISOString();
const tags: Record<string, string> = {
Name: name,
crabbox: "true",
crabbox_image: "true",
created_by: "crabbox",
provider: "aws",
region: this.region,
created_at: now,
source_lease: leaseID,
};
if (slug) {
tags["source_slug"] = slug;
}
const params: Record<string, string> = {
InstanceId: instanceID,
Name: name,
Description: description,
NoReboot: noReboot ? "true" : "false",
"TagSpecification.1.ResourceType": "image",
"TagSpecification.2.ResourceType": "snapshot",
};
addTags(params, "TagSpecification.1.Tag", tags);
addTags(params, "TagSpecification.2.Tag", tags);
const root = await this.ec2("CreateImage", params);
const imageID = asString(root["imageId"]);
if (wait) {
await this.waitForImage(imageID);
}
return this.describeImage(imageID)
.then((image) => ({ ...image, source: "created" }))
.catch(() => ({
id: imageID,
name,
state: "pending",
source: "created",
region: this.region,
tags,
}));
}
async setTags(instanceID: string, labels: Record<string, string>): Promise<void> {
const params: Record<string, string> = { "ResourceId.1": instanceID };
addTags(params, "Tag", labels);
@ -252,8 +330,13 @@ export class EC2SpotClient {
}
private async resolveAMI(config: LeaseConfig): Promise<string> {
const { imageID } = await this.currentImageID(config);
return imageID;
}
private async currentImageID(config: LeaseConfig): Promise<{ imageID: string; source: string }> {
if (config.awsAMI || this.env.CRABBOX_AWS_AMI) {
return config.awsAMI || this.env.CRABBOX_AWS_AMI || "";
return { imageID: config.awsAMI || this.env.CRABBOX_AWS_AMI || "", source: "config" };
}
const root = await this.ec2("DescribeImages", {
"Owner.1": awsUbuntuOwner,
@ -273,7 +356,34 @@ export class EC2SpotClient {
if (!imageID) {
throw new Error(`no Ubuntu 24.04 x86_64 AMI found in ${this.region}`);
}
return imageID;
return { imageID, source: "ubuntu-default" };
}
private async describeImage(imageID: string): Promise<AWSImageView> {
const root = await this.ec2("DescribeImages", { "ImageId.1": imageID });
const image = items(record(root["imagesSet"])["item"])[0];
const view = imageToView(image);
if (!view.id) {
throw new Error(`aws image not found: ${imageID}`);
}
return { ...view, region: this.region };
}
private async waitForImage(imageID: string): Promise<void> {
const deadline = Date.now() + 2_700_000;
while (Date.now() < deadline) {
// oxlint-disable-next-line eslint/no-await-in-loop -- polling waits for AMI state.
const image = await this.describeImage(imageID);
if (image.state === "available") {
return;
}
if (image.state === "failed") {
throw new Error(`aws image failed: ${imageID}`);
}
// oxlint-disable-next-line eslint/no-await-in-loop -- this delay is the polling interval.
await sleep(15_000);
}
throw new Error(`timed out waiting for AWS image: ${imageID}`);
}
private async ensureSecurityGroup(config: LeaseConfig): Promise<string> {
@ -433,6 +543,20 @@ function instanceToMachine(input: unknown): ProviderMachine {
};
}
function imageToView(input: unknown): AWSImageView {
const image = record(input);
return {
id: asString(image["imageId"]),
name: asString(image["name"]),
description: asString(image["description"]),
state: asString(image["imageState"]),
ownerId: asString(image["imageOwnerId"] || image["ownerId"]),
creationDate: asString(image["creationDate"]),
public: asString(image["isPublic"]) === "true",
tags: tagMap(image["tagSet"]),
};
}
function tagMap(input: unknown): Record<string, string> {
const out: Record<string, string> = {};
for (const item of items(record(input)["item"])) {

View File

@ -6,6 +6,9 @@ import { errorMessage, json, pathParts, readJson, requestOwner } from "./http";
import { githubAuthRoute } from "./oauth";
import { leaseSlugFromID, normalizeLeaseSlug, slugWithCollisionSuffix } from "./slug";
import type {
AWSImageCreateRequest,
AWSImagePromoteRequest,
AWSImageView,
Env,
LeaseRecord,
LeaseRequest,
@ -21,6 +24,10 @@ import { costLimits, enforceCostLimits, leaseCost, requestOrg, usageSummary } fr
const fleetID = "default";
function activeAWSImageKey(region: string): string {
return `image:aws:${region}:active`;
}
export class FleetDurableObject implements DurableObject {
constructor(
private readonly state: DurableObjectState,
@ -50,6 +57,12 @@ export class FleetDurableObject implements DurableObject {
if (method === "GET" && parts.join("/") === "v1/whoami") {
return this.whoami(request);
}
if (parts[0] === "v1" && parts[1] === "images") {
if (!isAdminRequest(request)) {
return json({ error: "forbidden", message: "admin token required" }, { status: 403 });
}
return await this.imageRoute(request, parts[2]);
}
if (method === "GET" && parts.join("/") === "v1/admin/leases") {
if (!isAdminRequest(request)) {
return json({ error: "forbidden", message: "admin token required" }, { status: 403 });
@ -99,6 +112,10 @@ export class FleetDurableObject implements DurableObject {
if (config.provider === "aws" && config.awsSSHCIDRs.length === 0) {
config.awsSSHCIDRs = requestSourceCIDRs(request);
}
if (config.provider === "aws" && !config.awsAMI && !this.env.CRABBOX_AWS_AMI) {
config.awsAMI =
(await this.state.storage.get<string>(activeAWSImageKey(config.awsRegion))) ?? "";
}
const leaseID = validLeaseID(input.leaseID) ? input.leaseID : newLeaseID();
const leases = await this.leaseRecords();
const slug = allocateLeaseSlug(
@ -266,6 +283,101 @@ export class FleetDurableObject implements DurableObject {
return json({ machines });
}
private async imageRoute(request: Request, action?: string): Promise<Response> {
const method = request.method.toUpperCase();
const url = new URL(request.url);
const provider = url.searchParams.get("provider") || "aws";
const region = url.searchParams.get("region") || this.env.CRABBOX_AWS_REGION || "eu-west-1";
if (provider !== "aws") {
return json(
{ error: "bad_request", message: "image routes only support provider=aws" },
{ status: 400 },
);
}
if (method === "GET" && action === "current") {
const activeImageID = await this.state.storage.get<string>(activeAWSImageKey(region));
const config = leaseConfig({
provider: "aws",
awsRegion: region,
...(activeImageID ? { awsAMI: activeImageID } : {}),
sshPublicKey: "ssh-ed25519 crabbox-image-inspect",
});
const image = await this.provider("aws", region).currentImage?.(config);
if (image && activeImageID) {
image.source = "promoted";
}
return json({ image });
}
if (method === "GET" && !action) {
const images = await this.provider("aws", region).listAWSImages?.(
url.searchParams.get("name") || "",
);
return json({ images: images ?? [] });
}
if (method === "POST" && !action) {
const input = await readJson<AWSImageCreateRequest>(request);
if (input.provider && input.provider !== "aws") {
return json(
{ error: "bad_request", message: "image create only supports provider=aws" },
{ status: 400 },
);
}
if (!input.leaseID || !input.name) {
return json(
{ error: "bad_request", message: "leaseID and name are required" },
{ status: 400 },
);
}
const lease = await this.resolveLease(input.leaseID, request, true);
if (!lease) {
return notFound();
}
if (lease.provider !== "aws") {
return json(
{ error: "bad_request", message: `lease provider is ${lease.provider}` },
{ status: 400 },
);
}
const image = await this.provider("aws", lease.region).createAWSImage?.(
lease,
input.name,
input.description || "",
Boolean(input.noReboot),
Boolean(input.wait),
);
return json({ image });
}
if (method === "POST" && action === "promote") {
const input = await readJson<AWSImagePromoteRequest>(request);
if (input.provider && input.provider !== "aws") {
return json(
{ error: "bad_request", message: "image promote only supports provider=aws" },
{ status: 400 },
);
}
const imageID = input.imageID?.trim() ?? "";
if (!imageID.startsWith("ami-")) {
return json(
{ error: "bad_request", message: "valid imageID is required" },
{ status: 400 },
);
}
const config = leaseConfig({
provider: "aws",
awsRegion: region,
awsAMI: imageID,
sshPublicKey: "ssh-ed25519 crabbox-image-promote",
});
const image = await this.provider("aws", region).currentImage?.(config);
await this.state.storage.put(activeAWSImageKey(region), imageID);
if (image) {
image.source = "promoted";
}
return json({ image });
}
return json({ error: "not_found" }, { status: 404 });
}
private async listLeases(request: Request): Promise<Response> {
const leases = isAdminRequest(request)
? this.filterLeases(await this.leaseRecords(), request)
@ -398,18 +510,35 @@ export class FleetDurableObject implements DurableObject {
if (commandMs !== undefined) {
run.commandMs = commandMs;
}
const syncFiles = finiteNumber(input.syncFiles);
if (syncFiles !== undefined) {
run.syncFiles = syncFiles;
}
const syncBytes = finiteNumber(input.syncBytes);
if (syncBytes !== undefined) {
run.syncBytes = syncBytes;
}
const syncDeleted = finiteNumber(input.syncDeleted);
if (syncDeleted !== undefined) {
run.syncDeleted = syncDeleted;
}
const syncManifestBytes = finiteNumber(input.syncManifestBytes);
if (syncManifestBytes !== undefined) {
run.syncManifestBytes = syncManifestBytes;
}
run.syncSkipped = Boolean(input.syncSkipped);
if (Number.isFinite(started)) {
run.durationMs = now.getTime() - started;
}
run.state = run.exitCode === 0 ? "succeeded" : "failed";
run.endedAt = now.toISOString();
const log = input.log ?? "";
run.logBytes = new TextEncoder().encode(log).byteLength;
run.logTruncated = Boolean(input.logTruncated);
const boundedLog = boundedRunLog(input.log ?? "");
run.logBytes = boundedLog.bytes;
run.logTruncated = Boolean(input.logTruncated) || boundedLog.truncated;
if (input.results) {
run.results = boundedTestResults(input.results);
}
await this.state.storage.put(runLogKey(runID), log);
await this.state.storage.put(runLogKey(runID), boundedLog.log);
await this.putRun(run);
return json({ run });
}
@ -653,6 +782,21 @@ function finiteNumber(value: number | undefined): number | undefined {
const MAX_RESULT_FILES = 50;
const MAX_RESULT_FAILURES = 100;
const MAX_RESULT_STRING_BYTES = 4096;
const MAX_RUN_LOG_BYTES = 64 * 1024;
function boundedRunLog(log: string): { log: string; bytes: number; truncated: boolean } {
const encoder = new TextEncoder();
const bytes = encoder.encode(log);
if (bytes.byteLength <= MAX_RUN_LOG_BYTES) {
return { log, bytes: bytes.byteLength, truncated: false };
}
const decoder = new TextDecoder();
let out = decoder.decode(bytes.slice(bytes.byteLength - MAX_RUN_LOG_BYTES));
while (encoder.encode(out).byteLength > MAX_RUN_LOG_BYTES) {
out = out.slice(1);
}
return { log: out, bytes: encoder.encode(out).byteLength, truncated: true };
}
function boundedTestResults(results: TestResultSummary): TestResultSummary {
return {
@ -811,6 +955,15 @@ interface CloudProvider {
serverType: string,
config: ReturnType<typeof leaseConfig>,
): Promise<number | undefined>;
currentImage?(config: ReturnType<typeof leaseConfig>): Promise<AWSImageView>;
listAWSImages?(name: string): Promise<AWSImageView[]>;
createAWSImage?(
lease: LeaseRecord,
name: string,
description: string,
noReboot: boolean,
wait: boolean,
): Promise<AWSImageView>;
}
class HetznerProvider implements CloudProvider {
@ -893,4 +1046,30 @@ class AWSProvider implements CloudProvider {
hourlyPriceUSD(serverType: string): Promise<number | undefined> {
return this.client.hourlySpotPriceUSD(serverType);
}
currentImage(config: ReturnType<typeof leaseConfig>): Promise<AWSImageView> {
return this.client.currentImage(config);
}
listAWSImages(name: string): Promise<AWSImageView[]> {
return this.client.listImages(name);
}
createAWSImage(
lease: LeaseRecord,
name: string,
description: string,
noReboot: boolean,
wait: boolean,
): Promise<AWSImageView> {
return this.client.createImage(
lease.cloudID,
name,
description,
lease.id,
lease.slug || "",
noReboot,
wait,
);
}
}

View File

@ -112,6 +112,11 @@ export interface RunRecord {
syncMs?: number;
commandMs?: number;
durationMs?: number;
syncFiles?: number;
syncBytes?: number;
syncDeleted?: number;
syncManifestBytes?: number;
syncSkipped?: boolean;
logBytes: number;
logTruncated: boolean;
results?: TestResultSummary;
@ -131,6 +136,11 @@ export interface RunFinishRequest {
exitCode: number;
syncMs?: number;
commandMs?: number;
syncFiles?: number;
syncBytes?: number;
syncDeleted?: number;
syncManifestBytes?: number;
syncSkipped?: boolean;
log?: string;
logTruncated?: boolean;
results?: TestResultSummary;
@ -201,3 +211,30 @@ export interface ProviderMachine {
host: string;
labels: Record<string, string>;
}
export interface AWSImageView {
id: string;
name?: string;
description?: string;
state?: string;
ownerId?: string;
creationDate?: string;
public?: boolean;
source?: string;
region?: string;
tags?: Record<string, string>;
}
export interface AWSImageCreateRequest {
provider?: Provider;
leaseID?: string;
name?: string;
description?: string;
noReboot?: boolean;
wait?: boolean;
}
export interface AWSImagePromoteRequest {
provider?: Provider;
imageID?: string;
}

View File

@ -292,6 +292,134 @@ describe("fleet lease identity and idle", () => {
);
expect(allowed.status).toBe(200);
});
it("keeps AWS image routes admin-only and creates, promotes, and reuses images", async () => {
const storage = new MemoryStorage();
const calls: string[] = [];
let createdAMI = "";
const fleet = testFleet(storage, {
aws: {
...fakeProvider(),
async currentImage(config: { awsAMI: string }) {
calls.push(`current:${config.awsAMI || "default"}`);
return {
id: config.awsAMI || "ami-current",
name: "current",
source: "config",
region: "eu-west-1",
};
},
async listAWSImages(name: string) {
calls.push(`list:${name}`);
return [{ id: "ami-cached", name: "openclaw-crabbox", region: "eu-west-1" }];
},
async createAWSImage(lease: LeaseRecord, name: string) {
calls.push(`create:${lease.id}:${name}`);
createdAMI = "ami-created";
return { id: createdAMI, name, source: "created", region: "eu-west-1" };
},
},
});
storage.seed(
"lease:cbx_000000000001",
testLease({
id: "cbx_000000000001",
slug: "blue-lobster",
provider: "aws",
cloudID: "i-123",
region: "eu-west-1",
}),
);
const denied = await fleet.fetch(
request("GET", "/v1/images", {
headers: {
"cf-access-authenticated-user-email": "friend@example.com",
"x-crabbox-org": "openclaw",
},
}),
);
expect(denied.status).toBe(403);
const current = await fleet.fetch(
request("GET", "/v1/images/current?provider=aws", {
headers: { "x-crabbox-admin": "true" },
}),
);
expect(current.status).toBe(200);
await expect(current.json()).resolves.toMatchObject({ image: { id: "ami-current" } });
const listed = await fleet.fetch(
request("GET", "/v1/images?provider=aws&name=openclaw-*", {
headers: { "x-crabbox-admin": "true" },
}),
);
expect(listed.status).toBe(200);
await expect(listed.json()).resolves.toMatchObject({ images: [{ id: "ami-cached" }] });
const created = await fleet.fetch(
request("POST", "/v1/images", {
headers: { "x-crabbox-admin": "true" },
body: { provider: "aws", leaseID: "cbx_000000000001", name: "openclaw-cache" },
}),
);
expect(created.status).toBe(200);
await expect(created.json()).resolves.toMatchObject({ image: { id: "ami-created" } });
const promoted = await fleet.fetch(
request("POST", "/v1/images/promote?provider=aws", {
headers: { "x-crabbox-admin": "true" },
body: { provider: "aws", imageID: createdAMI },
}),
);
expect(promoted.status).toBe(200);
await expect(promoted.json()).resolves.toMatchObject({
image: { id: "ami-created", source: "promoted" },
});
expect(storage.value("image:aws:eu-west-1:active")).toBe("ami-created");
const promotedCurrent = await fleet.fetch(
request("GET", "/v1/images/current?provider=aws", {
headers: { "x-crabbox-admin": "true" },
}),
);
expect(promotedCurrent.status).toBe(200);
await expect(promotedCurrent.json()).resolves.toMatchObject({
image: { id: "ami-created", source: "promoted" },
});
expect(calls).toEqual([
"current:default",
"list:openclaw-*",
"create:cbx_000000000001:openclaw-cache",
"current:ami-created",
"current:ami-created",
]);
});
it("uses the promoted AWS image for future leases", async () => {
const storage = new MemoryStorage();
let usedAMI = "";
storage.seed("image:aws:eu-west-1:active", "ami-promoted");
const fleet = testFleet(storage, {
aws: fakeProvider((config) => {
usedAMI = config.awsAMI ?? "";
}),
});
const created = await fleet.fetch(
request("POST", "/v1/leases", {
headers: { "cf-access-authenticated-user-email": "peter@example.com" },
body: {
provider: "aws",
sshPublicKey: "ssh-ed25519 test",
},
}),
);
expect(created.status).toBe(201);
expect(usedAMI).toBe("ami-promoted");
});
});
describe("fleet run history", () => {
@ -323,6 +451,11 @@ describe("fleet run history", () => {
exitCode: 0,
syncMs: 12,
commandMs: 34,
syncFiles: 123,
syncBytes: 4567,
syncDeleted: 2,
syncManifestBytes: 789,
syncSkipped: true,
log: "ok\n",
results: {
format: "junit",
@ -340,10 +473,24 @@ describe("fleet run history", () => {
);
expect(finish.status).toBe(200);
const finished = (await finish.json()) as {
run: { state: string; logBytes: number; results?: { tests: number } };
run: {
state: string;
logBytes: number;
syncFiles?: number;
syncBytes?: number;
syncDeleted?: number;
syncManifestBytes?: number;
syncSkipped?: boolean;
results?: { tests: number };
};
};
expect(finished.run.state).toBe("succeeded");
expect(finished.run.logBytes).toBe(3);
expect(finished.run.syncFiles).toBe(123);
expect(finished.run.syncBytes).toBe(4567);
expect(finished.run.syncDeleted).toBe(2);
expect(finished.run.syncManifestBytes).toBe(789);
expect(finished.run.syncSkipped).toBe(true);
expect(finished.run.results?.tests).toBe(2);
const listed = await fleet.fetch(
@ -411,6 +558,40 @@ describe("fleet run history", () => {
expect(storage.value<string>("runlog:run_000000000001")).toBe("secret log\n");
});
it("keeps only the bounded log tail", async () => {
const storage = new MemoryStorage();
const fleet = testFleet(storage);
const create = await fleet.fetch(
request("POST", "/v1/runs", {
body: {
leaseID: "cbx_000000000001",
provider: "aws",
class: "beast",
serverType: "c7a.48xlarge",
command: ["go", "test", "./..."],
},
}),
);
const { run } = (await create.json()) as { run: { id: string } };
const log = `${"a".repeat(70 * 1024)}tail`;
const finish = await fleet.fetch(
request("POST", `/v1/runs/${run.id}/finish`, {
body: { exitCode: 0, log },
}),
);
expect(finish.status).toBe(200);
const finished = (await finish.json()) as {
run: { logBytes: number; logTruncated: boolean };
};
expect(finished.run.logBytes).toBe(64 * 1024);
expect(finished.run.logTruncated).toBe(true);
const stored = storage.value<string>(`runlog:${run.id}`) ?? "";
expect(new TextEncoder().encode(stored).byteLength).toBe(64 * 1024);
expect(stored.endsWith("tail")).toBe(true);
});
it("bounds stored result summaries", async () => {
const fleet = testFleet();
const create = await fleet.fetch(
@ -713,13 +894,13 @@ function testFleet(storage = new MemoryStorage(), providers = {}): FleetDurableO
);
}
function fakeProvider(onCreate?: (config: { awsSSHCIDRs: string[] }) => void) {
function fakeProvider(onCreate?: (config: { awsSSHCIDRs: string[]; awsAMI?: string }) => void) {
return {
async listCrabboxServers() {
return [];
},
async createServerWithFallback(
config: { awsSSHCIDRs: string[] },
config: { awsSSHCIDRs: string[]; awsAMI?: string },
_leaseID: string,
slug: string,
) {