docs: complete crabbox documentation audit
This commit is contained in:
Peter Steinberger
parent
d5e8a9c288
commit
8a335c6263
@ -15,14 +15,15 @@ local checkout.
|
||||
- Prefer local targeted tests for tight edit loops.
|
||||
- Check repo-local `crabbox.yaml` or `.crabbox.yaml` before adding flags.
|
||||
- Install with `brew install openclaw/tap/crabbox`.
|
||||
- Auth is required for brokered operation:
|
||||
`printf '%s' "$CRABBOX_COORDINATOR_TOKEN" | crabbox login --url https://crabbox-coordinator.steipete.workers.dev --provider aws --token-stdin`.
|
||||
- Auth is required for brokered operation. Normal users run `crabbox login`.
|
||||
- Trusted operator automation can store the shared token with:
|
||||
`printf '%s' "$CRABBOX_COORDINATOR_TOKEN" | crabbox login --url https://crabbox.openclaw.ai --provider aws --token-stdin`.
|
||||
- User config lives at `~/Library/Application Support/crabbox/config.yaml` on
|
||||
macOS or the platform user config dir elsewhere. It should contain:
|
||||
|
||||
```yaml
|
||||
broker:
|
||||
url: https://crabbox-coordinator.steipete.workers.dev
|
||||
url: https://crabbox.openclaw.ai
|
||||
token: <token>
|
||||
provider: aws
|
||||
```
|
||||
@ -67,6 +68,7 @@ crabbox results <run_id>
|
||||
crabbox cache stats --id <id-or-slug>
|
||||
crabbox ssh --id <id-or-slug>
|
||||
crabbox usage --scope org
|
||||
CRABBOX_LIVE=1 CRABBOX_LIVE_REPO=/path/to/openclaw scripts/live-smoke.sh
|
||||
```
|
||||
|
||||
Use `--debug` on `run` when measuring sync timing.
|
||||
|
||||
17
.github/workflows/ci.yml
vendored
17
.github/workflows/ci.yml
vendored
@ -94,6 +94,23 @@ jobs:
|
||||
run: npm run build
|
||||
working-directory: worker
|
||||
|
||||
docs:
|
||||
name: Docs
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 5
|
||||
|
||||
steps:
|
||||
- name: Check out
|
||||
uses: actions/checkout@v6
|
||||
|
||||
- name: Set up Node
|
||||
uses: actions/setup-node@v6
|
||||
with:
|
||||
node-version: 24
|
||||
|
||||
- name: Check docs
|
||||
run: npm run docs:check
|
||||
|
||||
release-check:
|
||||
name: Release Check
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
@ -6,12 +6,14 @@ Crabbox 0.3.0 adds the first trusted-operator image lifecycle for AWS runners: o
|
||||
|
||||
### Added
|
||||
|
||||
- Added the Access-protected coordinator route `https://crabbox-access.openclaw.ai` for service-token proof and hardened automation.
|
||||
- Added `crabbox image create --id <cbx_id> --name <ami-name> [--wait]` for trusted operators to create AWS AMIs from active brokered AWS leases.
|
||||
- Added `crabbox image promote <ami-id>` for trusted operators to promote an available AMI as the coordinator default for future brokered AWS leases.
|
||||
- Added JSON output and wait polling for image creation, including `--wait-timeout` and `--no-reboot` controls.
|
||||
- Added coordinator image routes for admin-token callers: `POST /v1/images`, `GET /v1/images/{ami-id}`, and `POST /v1/images/{ami-id}/promote`.
|
||||
- Added AWS provider support for `CreateImage` and `DescribeImages`, with Crabbox-owned AMI tags.
|
||||
- Added `docs/commands/image.md` and linked the image command from the CLI docs, command index, docs site, and source map.
|
||||
- Added `npm run docs:check` with internal Markdown link validation plus docs-site generation, and wired it into CI.
|
||||
- Added `scripts/live-smoke.sh` for opt-in AWS, Hetzner, and Blacksmith Testbox live smoke coverage from a real repository checkout.
|
||||
|
||||
### Changed
|
||||
|
||||
12
README.md
12
README.md
@ -26,7 +26,7 @@ Prerequisites on the laptop: `git`, `ssh`, `ssh-keygen`, `rsync`, `curl`.
|
||||
## Quick start
|
||||
|
||||
```sh
|
||||
# log in once per machine (stores a bearer token in the OS keychain)
|
||||
# log in once per machine (stores a broker token in user config)
|
||||
crabbox login
|
||||
|
||||
# verify local prerequisites and broker reachability
|
||||
@ -155,9 +155,15 @@ scripts/check-go-coverage.sh 85.0
|
||||
npm ci --prefix worker
|
||||
npm test --prefix worker
|
||||
npm run build --prefix worker
|
||||
|
||||
# Docs
|
||||
npm run docs:check
|
||||
|
||||
# Optional live smoke, when broker/provider credentials are available
|
||||
CRABBOX_LIVE=1 CRABBOX_LIVE_REPO=/path/to/openclaw scripts/live-smoke.sh
|
||||
```
|
||||
|
||||
CI runs the full gate (gofmt, vet, race tests, coverage threshold, GoReleaser snapshot, Worker lint/typecheck/tests/build) on every push and PR. Tagged pushes matching `v*` publish Go archives via GoReleaser and bump the Homebrew formula at [openclaw/homebrew-tap](https://github.com/openclaw/homebrew-tap).
|
||||
CI runs the full gate (gofmt, vet, race tests, coverage threshold, docs link/build check, GoReleaser snapshot, Worker lint/typecheck/tests/build) on every push and PR. Tagged pushes matching `v*` publish Go archives via GoReleaser and bump the Homebrew formula at [openclaw/homebrew-tap](https://github.com/openclaw/homebrew-tap).
|
||||
|
||||
Worker deployment, required secrets, and DNS routing live in [docs/infrastructure.md](docs/infrastructure.md).
|
||||
|
||||
@ -172,7 +178,7 @@ Worker deployment, required secrets, and DNS routing live in [docs/infrastructur
|
||||
The GitHub Pages site at <https://openclaw.github.io/crabbox/> is generated from the `docs/` Markdown:
|
||||
|
||||
```sh
|
||||
node scripts/build-docs-site.mjs
|
||||
npm run docs:check
|
||||
open dist/docs-site/index.html
|
||||
```
|
||||
|
||||
|
||||
@ -43,7 +43,7 @@ Verify with `crabbox --version`.
|
||||
## Quick start
|
||||
|
||||
```sh
|
||||
# log in once per machine - stores a bearer token in the OS keychain
|
||||
# log in once per machine - stores a broker token in user config
|
||||
crabbox login
|
||||
|
||||
# one-shot run on a fresh leased box
|
||||
@ -86,6 +86,6 @@ Markdown in this directory is the user-facing documentation source. Implementati
|
||||
Build the docs site locally:
|
||||
|
||||
```sh
|
||||
node scripts/build-docs-site.mjs
|
||||
npm run docs:check
|
||||
open dist/docs-site/index.html
|
||||
```
|
||||
|
||||
@ -14,7 +14,7 @@ The coordinator leases machines. The CLI executes work. Machines do not need to
|
||||
developer laptop
|
||||
crabbox CLI
|
||||
|
|
||||
| HTTPS JSON API, Cloudflare Access
|
||||
| HTTPS JSON API, Crabbox auth
|
||||
v
|
||||
Cloudflare Worker
|
||||
Durable Object lease state
|
||||
@ -32,7 +32,7 @@ leased machine
|
||||
|
||||
## Lease Flow
|
||||
|
||||
1. CLI loads config and authenticates to Cloudflare Access.
|
||||
1. CLI loads config and authenticates with a signed GitHub login token or shared operator token.
|
||||
2. CLI creates a per-lease SSH key.
|
||||
3. CLI sends `POST /v1/leases` with lease ID, slug, profile, TTL, idle timeout, desired machine class, and SSH public key.
|
||||
4. Coordinator validates identity and policy.
|
||||
|
||||
@ -28,6 +28,32 @@ Flags:
|
||||
The source lease must still be active in the coordinator. The Worker calls AWS
|
||||
`CreateImage` from the backing instance ID and tags the image as Crabbox-owned.
|
||||
|
||||
Recommended bake flow:
|
||||
|
||||
```sh
|
||||
crabbox warmup --provider aws --class standard --ttl 2h --idle-timeout 30m
|
||||
crabbox run --id <slug> --shell -- 'command -v ssh git rsync curl jq && test -d /work/crabbox'
|
||||
crabbox image create --id <cbx_id> --name openclaw-crabbox-YYYYMMDD-HHMM --wait
|
||||
```
|
||||
|
||||
Use a fresh, intentionally warmed lease as the source. Do not bake personal
|
||||
workspace state, local secrets, repository checkouts, or one-off debugging
|
||||
artifacts into the image.
|
||||
|
||||
Failure handling:
|
||||
|
||||
- If `--wait` times out, run `crabbox image create ... --json` or inspect the
|
||||
AWS AMI state before retrying. AWS image creation can continue after the CLI
|
||||
stops polling.
|
||||
- If the AMI enters a failed state, leave the current promoted image in place
|
||||
and create a new image from a fresh lease.
|
||||
- If the source lease disappears, create a new warm lease and restart the bake;
|
||||
image creation requires the backing AWS instance ID.
|
||||
- If the baked image boots but never reaches `crabbox-ready`, do not promote it.
|
||||
Keep the previous promoted AMI and debug bootstrap on a normal lease first.
|
||||
- Cleanup of stale candidate AMIs is an AWS operator task. Promotion does not
|
||||
delete old images or snapshots.
|
||||
|
||||
## promote
|
||||
|
||||
Promote an available AMI as the coordinator's default AWS image:
|
||||
@ -40,6 +66,20 @@ Future brokered AWS leases use the promoted image when the request does not set
|
||||
an explicit `awsAMI` or `CRABBOX_AWS_AMI` override. Promotion stores coordinator
|
||||
metadata only; it does not copy or modify the AMI.
|
||||
|
||||
Promotion and rollback:
|
||||
|
||||
```sh
|
||||
crabbox image promote ami-new
|
||||
crabbox warmup --provider aws --class standard --ttl 20m --idle-timeout 6m
|
||||
crabbox run --id <slug> --shell -- 'echo image-smoke-ok && uname -srm && test -d /work/crabbox'
|
||||
crabbox stop <slug>
|
||||
```
|
||||
|
||||
If the smoke fails, promote the previous known-good AMI again. The coordinator
|
||||
stores only the selected AMI ID, so rollback is another `image promote` call.
|
||||
Keep the previous AMI available until at least one brokered AWS smoke succeeds
|
||||
on the new image.
|
||||
|
||||
Related docs:
|
||||
|
||||
- [Infrastructure](../infrastructure.md)
|
||||
|
||||
@ -11,7 +11,7 @@ crabbox list --json
|
||||
|
||||
`crabbox pool list` remains as a compatibility alias.
|
||||
|
||||
In `blacksmith-testbox` mode this forwards to `blacksmith testbox list`. JSON output is not supported because the Blacksmith CLI owns the list formatting.
|
||||
In `blacksmith-testbox` mode this forwards to `blacksmith testbox list`. Human output preserves the Blacksmith table; `--json` emits Crabbox-parsed rows with id, status, repo, workflow, job, ref, and created time when the upstream table exposes those columns.
|
||||
|
||||
Flags:
|
||||
|
||||
|
||||
@ -13,7 +13,7 @@ crabbox usage --scope all --json
|
||||
|
||||
Usage requires a configured coordinator. Direct-provider mode has no central history to query.
|
||||
|
||||
Lease ownership comes from Cloudflare Access when available. In bearer-token mode, the CLI sends `CRABBOX_OWNER`, Git email env, or local `git config user.email`; set `CRABBOX_ORG` to group leases under an org.
|
||||
Lease ownership comes from the signed GitHub login token for normal users. In shared bearer-token mode, the CLI sends `CRABBOX_OWNER`, Git email env, or local `git config user.email`; set `CRABBOX_ORG` to group leases under an org. Access-protected fallback routes can override shared-token owner headers when they forward a Cloudflare Access email.
|
||||
|
||||
GitHub browser-login users see their own owner/org usage regardless of requested `--scope`, `--user`, or `--org`. Fleet-wide `--scope org` and `--scope all` views require shared-token admin auth.
|
||||
|
||||
|
||||
@ -13,7 +13,7 @@ Human output:
|
||||
user=steipete@gmail.com org=openclaw auth=github broker=https://crabbox.openclaw.ai
|
||||
```
|
||||
|
||||
Identity comes from Cloudflare Access email when present, then signed GitHub login tokens, then bearer-token headers. In shared bearer-token mode, the CLI sends `X-Crabbox-Owner` from `CRABBOX_OWNER`, Git email env, or `git config user.email`, and `X-Crabbox-Org` from `CRABBOX_ORG`.
|
||||
Identity normally comes from the signed GitHub login token. Shared bearer-token automation reports owner/org from `X-Crabbox-Owner` and `X-Crabbox-Org`; the CLI fills those from `CRABBOX_OWNER`, Git email env, `git config user.email`, and `CRABBOX_ORG`. If a fallback route forwards Cloudflare Access identity, that Access email wins over shared-token owner headers. JSON output also reports the forwarded auth mode, such as `github` or `bearer`.
|
||||
|
||||
Related docs:
|
||||
|
||||
|
||||
@ -11,7 +11,7 @@ Read when:
|
||||
Core features:
|
||||
|
||||
- [Coordinator](coordinator.md): brokered leases through Cloudflare Workers and Durable Objects.
|
||||
- [Broker auth and routing](broker-auth-routing.md): bearer tokens, Cloudflare Access identity, and Worker routes.
|
||||
- [Broker auth and routing](broker-auth-routing.md): GitHub login, shared bearer tokens, optional Cloudflare Access, and Worker routes.
|
||||
- [Providers](providers.md): Hetzner, AWS EC2 Spot, Blacksmith Testbox selection, classes, and fallback.
|
||||
- [Blacksmith Testbox](blacksmith-testbox.md): wrapper mode that delegates machines and sync to the Blacksmith CLI.
|
||||
- [Runner bootstrap](runner-bootstrap.md): cloud-init, installed tools, SSH port, and readiness.
|
||||
|
||||
@ -11,10 +11,10 @@ Crabbox supports GitHub browser login for normal users and shared bearer-token l
|
||||
Identity sent to the coordinator:
|
||||
|
||||
```text
|
||||
Cloudflare Access email, when present
|
||||
signed GitHub login token from browser auth
|
||||
X-Crabbox-Owner from CRABBOX_OWNER, Git email env, or git config user.email
|
||||
X-Crabbox-Org from CRABBOX_ORG
|
||||
Cloudflare Access email, when forwarded by a protected fallback route
|
||||
CRABBOX_DEFAULT_ORG fallback in the Worker
|
||||
```
|
||||
|
||||
|
||||
@ -82,6 +82,11 @@ blacksmith testbox stop --id <tbx_id>
|
||||
|
||||
The wrapper is deliberately thin. If Blacksmith adds behavior to those commands, Crabbox should prefer forwarding rather than reimplementing it.
|
||||
|
||||
`crabbox list --provider blacksmith-testbox --json` parses the Blacksmith table
|
||||
output into JSON rows with the fields Crabbox can see. That parser is a
|
||||
compatibility layer, not a Blacksmith API contract. If the Blacksmith CLI adds
|
||||
native JSON output, Crabbox should switch to that and drop table parsing.
|
||||
|
||||
## Auth
|
||||
|
||||
Auth stays with Blacksmith. Run `blacksmith auth login` before using this provider. Crabbox does not call the Crabbox login broker, does not send work to the Cloudflare coordinator, and does not hold Blacksmith credentials.
|
||||
|
||||
@ -10,7 +10,8 @@ The broker is exposed through Cloudflare Workers routes:
|
||||
|
||||
```text
|
||||
https://crabbox.openclaw.ai
|
||||
https://crabbox-coordinator.steipete.workers.dev
|
||||
https://crabbox-access.openclaw.ai
|
||||
https://crabbox-coordinator.services-91b.workers.dev
|
||||
crabbox.clawd.bot/*
|
||||
```
|
||||
|
||||
@ -57,6 +58,8 @@ X-Crabbox-Org: <org>
|
||||
|
||||
If the coordinator route is also protected by Cloudflare Access, the CLI can send Access credentials before the Worker receives the request. Configure `CRABBOX_ACCESS_CLIENT_ID` and `CRABBOX_ACCESS_CLIENT_SECRET` for a Cloudflare Access service token, or `CRABBOX_ACCESS_TOKEN` to forward an already minted Access JWT as `cf-access-token`. These Access credentials only satisfy Cloudflare Access; the Worker still requires the Crabbox bearer token or a signed Crabbox user token.
|
||||
|
||||
The live Access-protected route is `https://crabbox-access.openclaw.ai`. Its Access app is service-token-only (`non_identity`) and currently allows the local Crabbox CLI service token, so automated clients can prove both layers independently: first Cloudflare Access, then the Worker bearer or signed user token.
|
||||
|
||||
Owner selection for bearer-token requests:
|
||||
|
||||
```text
|
||||
@ -66,9 +69,9 @@ GIT_COMMITTER_EMAIL
|
||||
git config user.email
|
||||
```
|
||||
|
||||
`CRABBOX_ORG` sets the org header. When Cloudflare Access identity is present, Access email wins over the CLI-provided owner.
|
||||
`CRABBOX_ORG` sets the org header. When a request comes through Cloudflare Access and Access identity is forwarded, that Access email wins over the CLI-provided owner. Normal `crabbox login` requests use the signed GitHub token identity.
|
||||
|
||||
GitHub user tokens are signed by the Worker and are not admin tokens. Admin routes require the shared operator token. The `crabbox.openclaw.ai/*` route is the canonical CLI and browser-login endpoint. The worker.dev and `crabbox.clawd.bot/*` routes are fallbacks.
|
||||
GitHub user tokens are signed by the Worker and are not admin tokens. Admin routes require the shared operator token. The `crabbox.openclaw.ai/*` route is the canonical CLI and browser-login endpoint. `crabbox-access.openclaw.ai/*` is the service-token-protected endpoint. `https://crabbox-coordinator.services-91b.workers.dev` and `crabbox.clawd.bot/*` are fallbacks.
|
||||
|
||||
Related docs:
|
||||
|
||||
|
||||
@ -10,7 +10,7 @@ The coordinator is the Cloudflare Worker plus Fleet Durable Object. Normal Crabb
|
||||
|
||||
Responsibilities:
|
||||
|
||||
- authenticate broker requests with the shared token and Cloudflare Access context when present;
|
||||
- authenticate broker requests with signed GitHub user tokens or the shared operator token, with optional Cloudflare Access context on protected fallback routes;
|
||||
- serialize fleet state in one Durable Object;
|
||||
- create, heartbeat, release, expire, and look up leases;
|
||||
- own provider credentials;
|
||||
|
||||
@ -34,10 +34,11 @@ CRABBOX_DEFAULT_ORG
|
||||
|
||||
Identity for usage:
|
||||
|
||||
- Cloudflare Access email wins when present;
|
||||
- bearer-token CLI requests send `X-Crabbox-Owner`;
|
||||
- signed GitHub login tokens carry owner/org identity;
|
||||
- shared bearer-token CLI requests send `X-Crabbox-Owner`;
|
||||
- `X-Crabbox-Owner` comes from `CRABBOX_OWNER`, Git email env, or `git config user.email`;
|
||||
- `CRABBOX_ORG` sends `X-Crabbox-Org`.
|
||||
- Access-protected fallback routes can override shared-token owner headers with the forwarded Cloudflare Access email.
|
||||
|
||||
`estimatedUSD` is elapsed runtime cost. `reservedUSD` is TTL worst-case cost reserved before provisioning. Provider extras such as static IP charges, egress, snapshots, taxes, credits, and discounts are not fully modeled.
|
||||
|
||||
|
||||
@ -51,6 +51,21 @@ beast c7a.48xlarge, c7i.48xlarge, m7a.48xlarge, m7i.48xlarge, r7a.48xlarge,
|
||||
|
||||
Direct provider mode still exists when no coordinator is configured. It uses local AWS credentials or `HCLOUD_TOKEN`/`HETZNER_TOKEN` and should stay secondary to the brokered path.
|
||||
|
||||
Direct smoke shape:
|
||||
|
||||
```sh
|
||||
tmp="$(mktemp)"
|
||||
printf 'provider: hetzner\n' > "$tmp"
|
||||
CRABBOX_CONFIG="$tmp" CRABBOX_COORDINATOR= crabbox warmup --provider hetzner --class standard --ttl 15m --idle-timeout 4m
|
||||
CRABBOX_CONFIG="$tmp" CRABBOX_COORDINATOR= crabbox run --provider hetzner --id <slug> --no-sync -- echo direct-hetzner-ok
|
||||
CRABBOX_CONFIG="$tmp" CRABBOX_COORDINATOR= crabbox stop --provider hetzner <slug>
|
||||
rm -f "$tmp"
|
||||
```
|
||||
|
||||
Use `--provider aws` with AWS SDK credentials for direct AWS smoke. Direct mode
|
||||
has no Durable Object alarm; cleanup is best-effort through provider labels and
|
||||
manual `crabbox cleanup`.
|
||||
|
||||
Crabbox can also wrap Blacksmith Testboxes with `provider: blacksmith-testbox`. That backend does not use the Crabbox broker or direct cloud credentials. It shells out to the authenticated Blacksmith CLI for `testbox warmup`, `run`, `status`, `list`, and `stop`, while Crabbox keeps local slugs, repo claims, config, and timing summaries. See [Blacksmith Testbox](blacksmith-testbox.md).
|
||||
|
||||
Related docs:
|
||||
|
||||
@ -11,10 +11,12 @@ Crabbox creates a fresh SSH key per lease by default. This avoids sharing a long
|
||||
Local key storage is under the Crabbox user config directory, outside the repository:
|
||||
|
||||
```text
|
||||
macOS: ~/Library/Application Support/crabbox/keys/<lease>/
|
||||
Linux: ~/.config/crabbox/keys/<lease>/
|
||||
macOS: ~/Library/Application Support/crabbox/testboxes/<lease>/id_ed25519
|
||||
Linux: ~/.config/crabbox/testboxes/<lease>/id_ed25519
|
||||
```
|
||||
|
||||
A per-lease `known_hosts` file lives beside the key. SSH ControlMaster sockets are also scoped to the key path, so reused provider IPs do not poison the user's global `~/.ssh/known_hosts` and do not cross streams between leases.
|
||||
|
||||
The CLI sends only the public key to the coordinator. The Worker imports or reuses that public key in the provider:
|
||||
|
||||
- Hetzner SSH key;
|
||||
|
||||
@ -94,7 +94,7 @@ Direct mode needs local provider credentials (AWS SDK chain or `HCLOUD_TOKEN`).
|
||||
|
||||
## Auth And Identity
|
||||
|
||||
The broker accepts bearer-token automation and can also use Cloudflare Access identity when present. Bearer-token CLI requests send:
|
||||
The broker accepts signed GitHub login tokens for normal users and shared bearer tokens for trusted automation. Fallback routes can also sit behind Cloudflare Access before the Worker sees the request. Bearer-token CLI requests send:
|
||||
|
||||
```text
|
||||
Authorization: Bearer <token>
|
||||
@ -102,7 +102,7 @@ X-Crabbox-Owner: <email>
|
||||
X-Crabbox-Org: <org>
|
||||
```
|
||||
|
||||
Owner is resolved from `CRABBOX_OWNER`, the Git email env, or `git config user.email`. `CRABBOX_ORG` sets the org. Cloudflare Access email wins when both are present.
|
||||
Owner is resolved from the signed GitHub token for `crabbox login` users. In shared-token mode, owner comes from `CRABBOX_OWNER`, the Git email env, or `git config user.email`; `CRABBOX_ORG` sets the org. If a fallback route forwards Cloudflare Access identity, that Access email wins over shared-token owner headers.
|
||||
|
||||
## Sync Model
|
||||
|
||||
|
||||
@ -8,6 +8,12 @@ Canonical Worker endpoint:
|
||||
https://crabbox.openclaw.ai
|
||||
```
|
||||
|
||||
Access-protected Worker endpoint:
|
||||
|
||||
```text
|
||||
https://crabbox-access.openclaw.ai
|
||||
```
|
||||
|
||||
Legacy fallback route:
|
||||
|
||||
```text
|
||||
@ -17,10 +23,10 @@ https://crabbox.clawd.bot
|
||||
Workers.dev fallback endpoint:
|
||||
|
||||
```text
|
||||
https://crabbox-coordinator.steipete.workers.dev
|
||||
https://crabbox-coordinator.services-91b.workers.dev
|
||||
```
|
||||
|
||||
The `crabbox.openclaw.ai/*` Worker route is the stable automation and browser-login endpoint. `crabbox.clawd.bot/*` and the workers.dev URL remain fallback routes.
|
||||
The `crabbox.openclaw.ai/*` Worker route is the stable automation and browser-login endpoint. `crabbox-access.openclaw.ai/*` is the Cloudflare Access-protected route for service-token proof and hardened automation. `crabbox.clawd.bot/*` and the workers.dev URL remain fallback routes.
|
||||
|
||||
## Cloudflare
|
||||
|
||||
@ -34,13 +40,13 @@ Use Cloudflare for:
|
||||
|
||||
Known setup:
|
||||
|
||||
- Access org: `openclaw-crabbox.cloudflareaccess.com`.
|
||||
- Access org: `crabbox-openclaw.cloudflareaccess.com`.
|
||||
- Access enabled.
|
||||
- Current IdPs: one-time PIN and GitHub.
|
||||
- GitHub IdP name: `GitHub OpenClaw`.
|
||||
- GitHub IdP restriction: org `openclaw`.
|
||||
- Fallback Access app: `Crabbox Coordinator` on `crabbox.clawd.bot`.
|
||||
- Fallback Access policy readback verifies the GitHub org include rule for `openclaw`.
|
||||
- Service-token Access app: `Crabbox Coordinator Service Token` on `crabbox-access.openclaw.ai`.
|
||||
- Service-token Access policy: `CLI service token`, `non_identity`, include the local Crabbox CLI service token.
|
||||
|
||||
Required env:
|
||||
|
||||
@ -85,22 +91,22 @@ Current local status:
|
||||
- The GitHub OAuth client ID and secret may be stored locally as `CRABBOX_GITHUB_OAUTH_*` and deployed to the Worker as `CRABBOX_GITHUB_CLIENT_*`.
|
||||
- Cloudflare Access service-token CLI credentials can be stored locally as `CRABBOX_ACCESS_CLIENT_ID` and `CRABBOX_ACCESS_CLIENT_SECRET`; `CRABBOX_ACCESS_TOKEN` can carry an already minted Access JWT for protected fallback routes.
|
||||
- Crabbox browser-login OAuth secrets are deployed as Worker secrets `CRABBOX_GITHUB_CLIENT_ID`, `CRABBOX_GITHUB_CLIENT_SECRET`, and `CRABBOX_SESSION_SECRET`.
|
||||
- Worker route is attached for `crabbox.openclaw.ai/*`.
|
||||
- Worker routes are attached for `crabbox.openclaw.ai/*` and `crabbox-access.openclaw.ai/*`.
|
||||
- `CRABBOX_COORDINATOR`, `CRABBOX_PROFILE`, `CRABBOX_CONFIG`, `CRABBOX_FLEET_CONFIG`, `CRABBOX_SSH_KEY`, `CRABBOX_NO_COLOR`, and `CRABBOX_LOG` are optional CLI defaults and are not required to build the MVP.
|
||||
|
||||
The Cloudflare token `crabbox-deploy` is scoped to `Steipete@gmail.com's Account` and the Crabbox zones. It verifies access to Workers scripts, Access applications, Access identity providers, Access keys, DNS records, and zone Worker routes from both the local machine and MacBook Pro.
|
||||
The Cloudflare token `crabbox-deploy` is scoped to the OpenClaw Cloudflare account and the Crabbox/OpenClaw routes it manages. It verifies access to Workers scripts, Access applications, Access identity providers, Access keys, DNS records, and zone Worker routes from both the local machine and MacBook Pro.
|
||||
|
||||
## DNS Decision
|
||||
## DNS State
|
||||
|
||||
Preferred path:
|
||||
Current path:
|
||||
|
||||
1. Keep `openclaw.ai` in Cloudflare.
|
||||
2. Add proxied DNS for `crabbox.openclaw.ai`.
|
||||
3. Deploy Worker route `crabbox.openclaw.ai/*`.
|
||||
1. Keep the main `openclaw.ai` website on Vercel.
|
||||
2. Manage `crabbox.openclaw.ai` in the OpenClaw Cloudflare account.
|
||||
3. Proxy `crabbox.openclaw.ai/*` and `crabbox-access.openclaw.ai/*` to the `crabbox-coordinator` Worker.
|
||||
4. Set `CRABBOX_PUBLIC_URL=https://crabbox.openclaw.ai`.
|
||||
5. Configure the GitHub OAuth callback on `https://crabbox.openclaw.ai/v1/auth/github/callback`.
|
||||
|
||||
Temporary path:
|
||||
Fallback path:
|
||||
|
||||
1. Use the workers.dev URL for health checks if DNS is disrupted.
|
||||
2. Use `crabbox.clawd.bot` only as a legacy fallback.
|
||||
@ -272,11 +278,12 @@ Current deployed coordinator:
|
||||
|
||||
```text
|
||||
https://crabbox.openclaw.ai
|
||||
https://crabbox-coordinator.steipete.workers.dev
|
||||
https://crabbox-access.openclaw.ai
|
||||
https://crabbox-coordinator.services-91b.workers.dev
|
||||
crabbox.clawd.bot/* -> crabbox-coordinator fallback
|
||||
```
|
||||
|
||||
Current Worker secrets:
|
||||
Current Worker secrets and settings:
|
||||
|
||||
```text
|
||||
HETZNER_TOKEN
|
||||
@ -284,11 +291,18 @@ AWS_ACCESS_KEY_ID
|
||||
AWS_SECRET_ACCESS_KEY
|
||||
AWS_SESSION_TOKEN optional
|
||||
CRABBOX_SHARED_TOKEN
|
||||
CRABBOX_GITHUB_CLIENT_ID
|
||||
CRABBOX_GITHUB_CLIENT_SECRET
|
||||
CRABBOX_GITHUB_ALLOWED_ORG
|
||||
CRABBOX_GITHUB_ALLOWED_ORGS optional
|
||||
CRABBOX_GITHUB_ALLOWED_TEAMS optional
|
||||
CRABBOX_DEFAULT_ORG
|
||||
CRABBOX_SESSION_SECRET
|
||||
```
|
||||
|
||||
## Verified OpenClaw Run
|
||||
|
||||
Warm-run command from `/Users/steipete/Projects/openclaw` through the Cloudflare coordinator:
|
||||
Historical warm-run command from an OpenClaw checkout through the Cloudflare coordinator:
|
||||
|
||||
```sh
|
||||
CI=1 /usr/bin/time -p /Users/steipete/Projects/crabbox/bin/crabbox run --id cbx_f60f47cbc879 -- pnpm test:changed:max
|
||||
@ -301,6 +315,14 @@ Result:
|
||||
- Runner class: requested `beast`, actual fallback `cpx62`.
|
||||
- Sync path: rsync overlay plus remote Git hydrate for shallow checkout merge-base support.
|
||||
|
||||
Current live smoke command:
|
||||
|
||||
```sh
|
||||
CRABBOX_LIVE=1 CRABBOX_LIVE_REPO=/Users/steipete/Projects/clawdbot6 /Users/steipete/Projects/crabbox/scripts/live-smoke.sh
|
||||
```
|
||||
|
||||
The smoke covers brokered AWS, direct Hetzner, Blacksmith Testbox delegation, slug reuse, status/inspect/cache/history/logs, stop, and final active-lease cleanup checks.
|
||||
|
||||
## Local, MacBook Pro, And Mac Studio
|
||||
|
||||
The same required env should exist on the local machine, MacBook Pro, and Mac Studio. Do not commit these values.
|
||||
|
||||
@ -127,7 +127,7 @@ Build in this order:
|
||||
|
||||
9. Access/auth
|
||||
- Primary org: GitHub `openclaw`.
|
||||
- Cloudflare Access org: `openclaw-crabbox.cloudflareaccess.com`.
|
||||
- Cloudflare Access org: `crabbox-openclaw.cloudflareaccess.com`.
|
||||
- Cloudflare OTP remains available for early fallback.
|
||||
- GitHub OAuth app exists under the `openclaw` org as `Crabbox Access`.
|
||||
- GitHub IdP exists in Cloudflare Access as `GitHub OpenClaw`.
|
||||
@ -172,12 +172,14 @@ And proves:
|
||||
## Known Current Infra Facts
|
||||
|
||||
- Direct CLI execution is implemented and verified. It can create/reuse a Hetzner server, bootstrap it, sync a local checkout with rsync, hydrate shallow Git history enough for changed-test detection, run commands over SSH, stream output, and release/delete leases.
|
||||
- The Cloudflare coordinator and Durable Object lease store are implemented and deployed. The CLI uses them when `CRABBOX_COORDINATOR` is set, and falls back to direct Hetzner otherwise.
|
||||
- Intended primary domain: `crabbox.openclaw.ai`.
|
||||
- The Cloudflare coordinator and Durable Object lease store are implemented and deployed. The CLI uses them when a broker URL is configured, and direct provider mode remains a debug fallback.
|
||||
- Primary domain: `crabbox.openclaw.ai`.
|
||||
- Access-protected service-token domain: `crabbox-access.openclaw.ai`.
|
||||
- Current Cloudflare-manageable fallback domain: `crabbox.clawd.bot`.
|
||||
- `openclaw.ai` must be visible as a Cloudflare zone before `crabbox.openclaw.ai/*` can be attached as a Worker route. Current public DNS is on Namecheap nameservers.
|
||||
- Workers.dev fallback: `https://crabbox-coordinator.services-91b.workers.dev`.
|
||||
- `crabbox.openclaw.ai/*` is attached as a Worker route in the OpenClaw Cloudflare account. The main `openclaw.ai` website can stay on Vercel; only the Crabbox subdomain needs to route to Cloudflare/Workers.
|
||||
- Cloudflare account ID and Crabbox Cloudflare token are available in local and MacBook Pro `~/.profile`.
|
||||
- The current Crabbox Cloudflare token is `crabbox-deploy`, scoped to `Steipete@gmail.com's Account` and the `clawd.bot` zone.
|
||||
- The current Crabbox Cloudflare token is `crabbox-deploy`, scoped to the OpenClaw Cloudflare account and the routes/zones Crabbox manages.
|
||||
- The current Crabbox Cloudflare token verifies Workers scripts, Access apps, Access IdPs, Access keys, DNS records, and zone Worker routes.
|
||||
- Cloudflare Access is enabled.
|
||||
- Current Access IdPs are OTP and GitHub.
|
||||
@ -185,7 +187,7 @@ And proves:
|
||||
- Crabbox browser login uses a GitHub OAuth callback at `/v1/auth/github/callback` and stores OAuth client values as Worker secrets.
|
||||
- Cloudflare Access GitHub IdP `GitHub OpenClaw` exists.
|
||||
- Cloudflare Access app `Crabbox Coordinator` exists for `crabbox.clawd.bot`.
|
||||
- Worker `crabbox-coordinator` is deployed at `https://crabbox-coordinator.steipete.workers.dev` and routed from `crabbox.clawd.bot/*`. The canonical target is `https://crabbox.openclaw.ai` once the Cloudflare zone is delegated.
|
||||
- Worker `crabbox-coordinator` is deployed at `https://crabbox-coordinator.services-91b.workers.dev`, routed from `crabbox.openclaw.ai/*` and `crabbox-access.openclaw.ai/*`, and optionally reachable through fallback routes.
|
||||
- Coordinator auth supports GitHub browser-login user tokens plus shared-token operator automation. Shared-token auth uses `CRABBOX_COORDINATOR_TOKEN` locally and `CRABBOX_SHARED_TOKEN` in the Worker.
|
||||
- Hetzner token is available in local and Mac Studio `~/.profile`.
|
||||
- The Hetzner account currently hits a dedicated-core quota/resource limit for `ccx63`, `ccx53`, and `ccx43`. The `beast` class falls back to `cpx62` until quota is raised.
|
||||
@ -197,9 +199,7 @@ And proves:
|
||||
## Next Implementation Milestones
|
||||
|
||||
1. Raise Hetzner dedicated-core quota so `beast` can use `ccx63` instead of falling back to `cpx62`.
|
||||
2. Add GitHub org/team allowlisting for browser-login user tokens.
|
||||
3. Delegate `openclaw.ai` to Cloudflare or provide a token that can create/manage that zone, then attach `crabbox.openclaw.ai/*`.
|
||||
4. Add Cloudflare Access service-token support for non-browser CLI use on fallback routes.
|
||||
5. Add one-shot `run --profile` cleanup semantics coverage in integration tests.
|
||||
6. Add coordinator drain controls beyond release/delete.
|
||||
7. Re-run OpenClaw `pnpm test:changed:max` on `ccx63` and compare against the current Crabbox baseline.
|
||||
2. Add one-shot `run --profile` cleanup semantics coverage in integration tests.
|
||||
3. Add coordinator drain controls beyond release/delete.
|
||||
4. Re-run OpenClaw `pnpm test:changed:max` on `ccx63` and compare against the current Crabbox baseline after quota is raised.
|
||||
5. Add generated CLI docs or a docs drift check so command pages cannot silently diverge from actual flags.
|
||||
|
||||
@ -23,15 +23,42 @@ Run these before a release or after changing secrets:
|
||||
go test ./...
|
||||
npm run check --prefix worker
|
||||
npm test --prefix worker
|
||||
node scripts/build-docs-site.mjs
|
||||
npm run docs:check
|
||||
bin/crabbox doctor
|
||||
bin/crabbox whoami
|
||||
bin/crabbox status --json
|
||||
bin/crabbox list --json
|
||||
bin/crabbox usage --scope all --json
|
||||
bin/crabbox history --limit 5
|
||||
```
|
||||
|
||||
`crabbox doctor` checks local prerequisites and coordinator reachability. `crabbox whoami` verifies identity. `crabbox status` confirms the broker can answer lease state. `crabbox usage` proves the cost accounting path is reachable. `crabbox history` proves run history is reachable.
|
||||
`crabbox doctor` checks local prerequisites and coordinator reachability. `crabbox whoami` verifies identity. `crabbox list` confirms the broker can answer lease state. `crabbox usage` proves the cost accounting path is reachable. `crabbox history` proves run history is reachable.
|
||||
|
||||
When broker/provider credentials are available and infra changed, run the live smoke:
|
||||
|
||||
```sh
|
||||
CRABBOX_LIVE=1 CRABBOX_LIVE_REPO=/path/to/openclaw scripts/live-smoke.sh
|
||||
```
|
||||
|
||||
To narrow the live matrix while debugging, set `CRABBOX_LIVE_PROVIDERS`:
|
||||
|
||||
```sh
|
||||
CRABBOX_LIVE=1 CRABBOX_LIVE_PROVIDERS=aws CRABBOX_LIVE_REPO=/path/to/openclaw scripts/live-smoke.sh
|
||||
CRABBOX_LIVE=1 CRABBOX_LIVE_PROVIDERS=hetzner CRABBOX_LIVE_REPO=/path/to/openclaw scripts/live-smoke.sh
|
||||
CRABBOX_LIVE=1 CRABBOX_LIVE_PROVIDERS=blacksmith-testbox CRABBOX_LIVE_REPO=/path/to/openclaw scripts/live-smoke.sh
|
||||
```
|
||||
|
||||
For direct-provider smoke, disable the coordinator with a scratch config and run the same commands manually:
|
||||
|
||||
```sh
|
||||
tmp="$(mktemp)"
|
||||
printf 'provider: hetzner\n' > "$tmp"
|
||||
CRABBOX_CONFIG="$tmp" CRABBOX_COORDINATOR= bin/crabbox warmup --provider hetzner --class standard --ttl 15m --idle-timeout 4m
|
||||
CRABBOX_CONFIG="$tmp" CRABBOX_COORDINATOR= bin/crabbox run --provider hetzner --id <slug> --no-sync -- echo direct-hetzner-ok
|
||||
CRABBOX_CONFIG="$tmp" CRABBOX_COORDINATOR= bin/crabbox stop --provider hetzner <slug>
|
||||
rm -f "$tmp"
|
||||
```
|
||||
|
||||
Use `--provider aws` with AWS SDK credentials for the direct AWS equivalent.
|
||||
|
||||
## Deployment
|
||||
|
||||
@ -78,7 +105,13 @@ The canonical Worker URL is:
|
||||
https://crabbox.openclaw.ai
|
||||
```
|
||||
|
||||
The `crabbox.openclaw.ai/*` route is attached to the coordinator Worker. Bearer-token CLI automation talks to the Worker with `CRABBOX_SHARED_TOKEN`/`CRABBOX_COORDINATOR_TOKEN`; GitHub browser login stores a user-scoped signed token. Access-protected fallback routes can also use `CRABBOX_ACCESS_CLIENT_ID` plus `CRABBOX_ACCESS_CLIENT_SECRET`, or `CRABBOX_ACCESS_TOKEN` for an already minted Access JWT.
|
||||
The Access-protected Worker URL is:
|
||||
|
||||
```text
|
||||
https://crabbox-access.openclaw.ai
|
||||
```
|
||||
|
||||
The `crabbox.openclaw.ai/*` route is attached to the coordinator Worker for normal CLI and browser-login use. `crabbox-access.openclaw.ai/*` is attached to the same Worker behind Cloudflare Access for service-token proof and hardened automation. Bearer-token CLI automation talks to the Worker with `CRABBOX_SHARED_TOKEN`/`CRABBOX_COORDINATOR_TOKEN`; GitHub browser login stores a user-scoped signed token. Access-protected routes also require `CRABBOX_ACCESS_CLIENT_ID` plus `CRABBOX_ACCESS_CLIENT_SECRET`, or `CRABBOX_ACCESS_TOKEN` for an already minted Access JWT.
|
||||
|
||||
Use `crabbox config show` to confirm which URL and provider the CLI will use:
|
||||
|
||||
@ -129,7 +162,6 @@ Before handing off:
|
||||
|
||||
- `go test ./...`
|
||||
- Worker format, lint, typecheck, tests, and build.
|
||||
- `node scripts/build-docs-site.mjs`
|
||||
- docs link check, when a link checker is available.
|
||||
- `npm run docs:check`
|
||||
- `git diff --check`
|
||||
- live `crabbox doctor` if broker credentials are available.
|
||||
|
||||
@ -85,7 +85,7 @@ CRABBOX_MAX_MONTHLY_USD_PER_ORG
|
||||
CRABBOX_DEFAULT_ORG
|
||||
```
|
||||
|
||||
The CLI sends `X-Crabbox-Owner` from `CRABBOX_OWNER`, Git author/committer email env, or local `git config user.email`. It sends `X-Crabbox-Org` from `CRABBOX_ORG` when set. Cloudflare Access email still wins when present.
|
||||
For signed GitHub login tokens, owner/org is embedded in the token that the Worker forwards to the Fleet Durable Object. In shared-token automation, the CLI sends `X-Crabbox-Owner` from `CRABBOX_OWNER`, Git author/committer email env, or local `git config user.email`, and sends `X-Crabbox-Org` from `CRABBOX_ORG` when set. If a fallback route forwards Cloudflare Access identity, that Access email wins over shared-token owner headers.
|
||||
|
||||
If a new lease would exceed a configured active-lease or monthly reserved-cost limit, the coordinator returns `cost_limit_exceeded` and does not provision the machine.
|
||||
|
||||
|
||||
@ -47,6 +47,7 @@ Bootstrap is intentionally tiny: OpenSSH, CA certificates, curl, Git, rsync, jq,
|
||||
- Git manifest, rsync plan, fingerprints, guardrails: `internal/cli/repo.go`
|
||||
- Sync plan command: `internal/cli/sync_plan.go`
|
||||
- SSH command output and direct SSH touch behavior: `internal/cli/ssh.go`, `internal/cli/ssh_cmd.go`
|
||||
- Per-lease SSH known_hosts and ControlMaster config: `internal/cli/ssh.go`
|
||||
- GitHub Actions hydrate/register/dispatch bridge: `internal/cli/actions.go`
|
||||
- Cache stats/purge/warm commands: `internal/cli/cache.go`
|
||||
- Run history/log commands and retained run logs: `internal/cli/history.go`, `internal/cli/runlog.go`
|
||||
@ -74,4 +75,5 @@ Bootstrap is intentionally tiny: OpenSSH, CA certificates, curl, Git, rsync, jq,
|
||||
- CI gate: `.github/workflows/ci.yml`
|
||||
- Release workflow and Homebrew tap fallback: `.github/workflows/release.yml`
|
||||
- GoReleaser archives and Homebrew formula config: `.goreleaser.yaml`
|
||||
- Docs site builder and Pages deployment: `scripts/build-docs-site.mjs`, `.github/workflows/pages.yml`
|
||||
- Docs link check, site builder, and Pages deployment: `scripts/check-docs-links.mjs`, `scripts/build-docs-site.mjs`, `.github/workflows/pages.yml`
|
||||
- Live provider smoke coverage: `scripts/live-smoke.sh`
|
||||
|
||||
@ -13,7 +13,7 @@ Start with:
|
||||
```sh
|
||||
bin/crabbox doctor
|
||||
bin/crabbox config show
|
||||
bin/crabbox status --json
|
||||
bin/crabbox list --json
|
||||
bin/crabbox usage --scope all --json
|
||||
```
|
||||
|
||||
@ -44,6 +44,27 @@ Fixes:
|
||||
- for self-hosted GitHub browser login, create your own GitHub OAuth app and set its callback URL to `https://<your-coordinator-host>/v1/auth/github/callback`;
|
||||
- ensure the Worker's `CRABBOX_PUBLIC_URL` uses the same public origin as that GitHub OAuth callback.
|
||||
|
||||
## SSH Host Key Or Control Socket Fails
|
||||
|
||||
Symptoms:
|
||||
|
||||
- SSH warns that host identification changed after a provider reused an IP;
|
||||
- a reused warm lease connects to the wrong ControlMaster socket;
|
||||
- paths under `~/Library/Application Support` appear split at the space.
|
||||
|
||||
Checks:
|
||||
|
||||
```sh
|
||||
bin/crabbox inspect --id blue-lobster --json
|
||||
bin/crabbox ssh --id blue-lobster
|
||||
```
|
||||
|
||||
Fixes:
|
||||
|
||||
- upgrade to a build that quotes SSH config values with spaces;
|
||||
- keep per-lease keys under the Crabbox config `testboxes/<lease>` directory;
|
||||
- avoid manually overriding `UserKnownHostsFile` or `ControlPath` unless debugging SSH itself.
|
||||
|
||||
## Lease Rejected By Cost Control
|
||||
|
||||
Symptoms:
|
||||
@ -194,7 +215,7 @@ Symptoms:
|
||||
Checks:
|
||||
|
||||
```sh
|
||||
node scripts/build-docs-site.mjs
|
||||
npm run docs:check
|
||||
gh run list --workflow pages.yml
|
||||
```
|
||||
|
||||
@ -203,3 +224,4 @@ Fixes:
|
||||
- enable GitHub Pages for the repository or organization;
|
||||
- rerun the Pages workflow after Pages is allowed;
|
||||
- keep Markdown links relative so the static builder can rewrite them.
|
||||
- fix broken internal Markdown links before checking whether Pages itself is down.
|
||||
|
||||
@ -15,6 +15,7 @@
|
||||
},
|
||||
"scripts": {
|
||||
"check": "node --check index.js && node --test index.test.js",
|
||||
"docs:check": "node scripts/check-docs-links.mjs && node scripts/build-docs-site.mjs",
|
||||
"test": "node --test index.test.js"
|
||||
},
|
||||
"files": [
|
||||
|
||||
@ -11,7 +11,10 @@ const sections = [
|
||||
["Start", ["README.md", "how-it-works.md", "architecture.md", "orchestrator.md", "cli.md"]],
|
||||
["Features", rels("features")],
|
||||
["Commands", rels("commands")],
|
||||
["Operate", ["operations.md", "observability.md", "troubleshooting.md", "performance.md", "infrastructure.md", "security.md", "mvp-plan.md"]],
|
||||
[
|
||||
"Operate",
|
||||
["operations.md", "observability.md", "troubleshooting.md", "performance.md", "infrastructure.md", "security.md", "mvp-plan.md"],
|
||||
],
|
||||
];
|
||||
|
||||
fs.rmSync(outDir, { recursive: true, force: true });
|
||||
|
||||
77
scripts/check-docs-links.mjs
Normal file
77
scripts/check-docs-links.mjs
Normal file
@ -0,0 +1,77 @@
|
||||
#!/usr/bin/env node
|
||||
import fs from "node:fs";
|
||||
import path from "node:path";
|
||||
|
||||
const root = process.cwd();
|
||||
const files = [path.join(root, "README.md"), ...walk(path.join(root, "docs"))];
|
||||
const failures = [];
|
||||
|
||||
for (const file of files) {
|
||||
const markdown = fs.readFileSync(file, "utf8");
|
||||
const headings = headingAnchors(markdown);
|
||||
const links = markdown.matchAll(/\[[^\]]+\]\(([^)]+)\)/g);
|
||||
for (const match of links) {
|
||||
const href = match[1].trim();
|
||||
if (!href || href.startsWith("http://") || href.startsWith("https://") || href.startsWith("mailto:")) {
|
||||
continue;
|
||||
}
|
||||
const [rawPath, rawAnchor] = href.split("#", 2);
|
||||
const linkPath = stripAngleBrackets(rawPath);
|
||||
const target = linkPath ? path.resolve(path.dirname(file), linkPath) : file;
|
||||
if (!fs.existsSync(target)) {
|
||||
failures.push(`${rel(file)} links to missing ${href}`);
|
||||
continue;
|
||||
}
|
||||
if (rawAnchor && target.endsWith(".md")) {
|
||||
const targetHeadings = target === file ? headings : headingAnchors(fs.readFileSync(target, "utf8"));
|
||||
if (!targetHeadings.has(rawAnchor)) {
|
||||
failures.push(`${rel(file)} links to missing heading ${href}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (failures.length) {
|
||||
console.error(failures.join("\n"));
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
console.log(`checked ${files.length} markdown files: internal links ok`);
|
||||
|
||||
function walk(dir) {
|
||||
return fs
|
||||
.readdirSync(dir, { withFileTypes: true })
|
||||
.flatMap((entry) => {
|
||||
const full = path.join(dir, entry.name);
|
||||
if (entry.isDirectory()) return walk(full);
|
||||
return entry.name.endsWith(".md") ? [full] : [];
|
||||
})
|
||||
.sort();
|
||||
}
|
||||
|
||||
function headingAnchors(markdown) {
|
||||
const anchors = new Set();
|
||||
for (const match of markdown.matchAll(/^#{1,6}\s+(.+)$/gm)) {
|
||||
anchors.add(slugify(match[1]));
|
||||
}
|
||||
return anchors;
|
||||
}
|
||||
|
||||
function slugify(text) {
|
||||
return text
|
||||
.toLowerCase()
|
||||
.replace(/`([^`]+)`/g, "$1")
|
||||
.replace(/<[^>]+>/g, "")
|
||||
.replace(/[^\w\s-]/g, "")
|
||||
.trim()
|
||||
.replace(/\s+/g, "-");
|
||||
}
|
||||
|
||||
function stripAngleBrackets(text) {
|
||||
if (text.startsWith("<") && text.endsWith(">")) return text.slice(1, -1);
|
||||
return text;
|
||||
}
|
||||
|
||||
function rel(file) {
|
||||
return path.relative(root, file).replaceAll(path.sep, "/");
|
||||
}
|
||||
@ -16,6 +16,10 @@
|
||||
"pattern": "crabbox.openclaw.ai/*",
|
||||
"zone_name": "openclaw.ai",
|
||||
},
|
||||
{
|
||||
"pattern": "crabbox-access.openclaw.ai/*",
|
||||
"zone_name": "openclaw.ai",
|
||||
},
|
||||
],
|
||||
"durable_objects": {
|
||||
"bindings": [
|
||||
|
||||
Loading…
Reference in New Issue
Block a user