openclaw/crabpot
openclaw/crabpot
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 4 Wiki Activity
7d425df57e
crabpot/docs/operations.md
Vincent Koc 4540f3efc0
test(contract): gate issue evidence coverage
2026-04-25 12:53:30 -07:00

2.3 KiB
Raw Blame History

Operations

Adding a fixture

  1. Add an entry to crabpot.config.json.
  2. Run npm test.
  3. Run node scripts/sync-fixtures.mjs --materialize.
  4. Review .gitmodules and the pinned submodule commit.
  5. Commit the manifest and submodule pointer together.

Updating fixtures

git submodule update --remote --recursive plugins/<fixture-id>
npm test
npm run report

Then inspect the diff. A fixture update is only useful if it either:

  • tracks a plugin release we care about,
  • adds coverage for a new seam,
  • reproduces a compatibility break,
  • or proves a contract migration path still works.

CI model

Use a cheap default workflow first:

  • validate the manifest
  • ensure .gitmodules agrees with crabpot.config.json
  • initialize pinned submodules
  • inspect expected hooks, registrations, and manifest contracts
  • run registration-capture smoke checks
  • generate the compatibility report in check mode
  • gate issue evidence, P1 probe coverage, and compat-record reconciliation

Add heavier lanes later for SDK compilation against specific OpenClaw refs:

  • openclaw@main
  • latest stable release tag
  • active beta branch or release candidate

Reading the report

Use reports/crabpot-report.md for full human review, reports/crabpot-report.json for downstream tooling, and reports/crabpot-issues.md for the short issue/probe queue.

Decision classes mean:

  • core-compat-adapter: preserve or add OpenClaw compatibility behavior.
  • plugin-upstream-fix: open an upstream plugin issue or migration PR.
  • inspector-follow-up: add deeper inspector coverage before deciding.
  • intentional-break: document a versioned migration and accept the break.
  • no-action: the seam is covered by current default checks.

Hard breakages should block contract changes. Warnings are old or sensitive seams that currently still work but need compat coverage before OpenClaw changes the relevant API.

Run the quality gate directly when changing classifiers:

npm run contract:coverage
node scripts/check-contract-coverage.mjs --openclaw ../openclaw

Issue severity means:

  • P0: current hard breakage.
  • P1: high-risk contract seam where OpenClaw should not change behavior until the inspector has a probe or a compat adapter.
  • P2: compatibility debt that should stay visible but does not block all contract work.
  • P3: low-risk follow-up.