diff --git a/.openclaw-sync/source.json b/.openclaw-sync/source.json index 08b0838f0..7198bbafc 100644 --- a/.openclaw-sync/source.json +++ b/.openclaw-sync/source.json @@ -1,5 +1,5 @@ { "repository": "openclaw/openclaw", - "sha": "42fed1f205fee47db1b04a0ca259a6f83f9dac1a", - "syncedAt": "2026-05-02T19:28:51.427Z" + "sha": "b9eb31b54cfaffcfd116486e0e25fe610cc73c60", + "syncedAt": "2026-05-02T19:58:28.682Z" } diff --git a/docs/ci.md b/docs/ci.md index dd587467b..50cd661b9 100644 --- a/docs/ci.md +++ b/docs/ci.md @@ -383,10 +383,9 @@ The scheduled live/E2E workflow runs the full release-path Docker suite daily. ## QA Lab -QA Lab has dedicated CI lanes outside the main smart-scoped workflow. +QA Lab has dedicated CI lanes outside the main smart-scoped workflow. Agentic parity is nested under the broad QA and release harnesses, not a standalone PR workflow. Use `Full Release Validation` with `rerun_group=qa-parity` when parity should ride with a broad validation run. -- The `Parity gate` workflow runs on matching PR changes and manual dispatch; it builds the private QA runtime and compares the mock GPT-5.5 and Opus 4.6 agentic packs. -- The `QA-Lab - All Lanes` workflow runs nightly on `main` and on manual dispatch; it fans out the mock parity gate, live Matrix lane, and live Telegram and Discord lanes as parallel jobs. Live jobs use the `qa-live-shared` environment, and Telegram/Discord use Convex leases. +- The `QA-Lab - All Lanes` workflow runs nightly on `main` and on manual dispatch; it fans out the mock parity lane, live Matrix lane, and live Telegram and Discord lanes as parallel jobs. Live jobs use the `qa-live-shared` environment, and Telegram/Discord use Convex leases. Release checks run Matrix and Telegram live transport lanes with the deterministic mock provider and mock-qualified models (`mock-openai/gpt-5.5` and `mock-openai/gpt-5.5-alt`) so the channel contract is isolated from live model latency and normal provider-plugin startup. The live transport gateway disables memory search because QA parity covers memory behavior separately; provider connectivity is covered by the separate live model, native provider, and Docker provider suites. @@ -394,7 +393,7 @@ Matrix uses `--profile fast` for scheduled and release gates, adding `--fail-fas `OpenClaw Release Checks` also runs the release-critical QA Lab lanes before release approval; its QA parity gate runs the candidate and baseline packs as parallel lane jobs, then downloads both artifacts into a small report job for the final parity comparison. -Do not put the PR landing path behind `Parity gate` unless the change actually touches QA runtime, model-pack parity, or a surface the parity workflow owns. For normal channel, config, docs, or unit-test fixes, treat it as an optional signal and follow the scoped CI/check evidence instead. +For normal PRs, follow scoped CI/check evidence instead of treating parity as a required status. ## CodeQL diff --git a/docs/concepts/qa-e2e-automation.md b/docs/concepts/qa-e2e-automation.md index 36c3e9065..8e8b11122 100644 --- a/docs/concepts/qa-e2e-automation.md +++ b/docs/concepts/qa-e2e-automation.md @@ -32,7 +32,7 @@ script aliases; both forms are supported. | `qa run` | Bundled QA self-check; writes a Markdown report. | | `qa suite` | Run repo-backed scenarios against the QA gateway lane. Aliases: `pnpm openclaw qa suite --runner multipass` for a disposable Linux VM. | | `qa coverage` | Print the markdown scenario-coverage inventory (`--json` for machine output). | -| `qa parity-report` | Compare two `qa-suite-summary.json` files and write the agentic parity-gate report. | +| `qa parity-report` | Compare two `qa-suite-summary.json` files and write the agentic parity report. | | `qa character-eval` | Run the character QA scenario across multiple live models with a judged report. See [Reporting](#reporting). | | `qa manual` | Run a one-off prompt against the selected provider/model lane. | | `qa ui` | Start the QA debugger UI and local QA bus (alias: `pnpm qa:lab:ui`). | diff --git a/docs/help/testing.md b/docs/help/testing.md index 4a0a8e971..ad72328b4 100644 --- a/docs/help/testing.md +++ b/docs/help/testing.md @@ -109,18 +109,21 @@ When you only need one failing case, prefer narrowing live tests via the allowli These commands sit beside the main test suites when you need QA-lab realism: -CI runs QA Lab in dedicated workflows. `Parity gate` runs on matching PRs and -from manual dispatch with mock providers. `QA-Lab - All Lanes` runs nightly on -`main` and from manual dispatch with the mock parity gate, live Matrix lane, -Convex-managed live Telegram lane, and Convex-managed live Discord lane as -parallel jobs. Scheduled QA and release checks pass Matrix `--profile fast` -explicitly, while the Matrix CLI and manual workflow input default remain -`all`; manual dispatch can shard `all` into `transport`, `media`, `e2ee-smoke`, -`e2ee-deep`, and `e2ee-cli` jobs. `OpenClaw Release Checks` runs parity plus -the fast Matrix and Telegram lanes before release approval, using -`mock-openai/gpt-5.5` for release transport checks so they stay deterministic -and avoid normal provider-plugin startup. These live transport gateways disable -memory search; memory behavior stays covered by the QA parity suites. +CI runs QA Lab in dedicated workflows. Agentic parity is nested under +`QA-Lab - All Lanes` and release validation, not a standalone PR workflow. +Broad validation should use `Full Release Validation` with +`rerun_group=qa-parity` or the release-checks QA group. `QA-Lab - All Lanes` +runs nightly on `main` and from manual dispatch with the mock parity lane, live +Matrix lane, Convex-managed live Telegram lane, and Convex-managed live Discord +lane as parallel jobs. Scheduled QA and release checks pass Matrix +`--profile fast` explicitly, while the Matrix CLI and manual workflow input +default remain `all`; manual dispatch can shard `all` into `transport`, +`media`, `e2ee-smoke`, `e2ee-deep`, and `e2ee-cli` jobs. `OpenClaw Release +Checks` runs parity plus the fast Matrix and Telegram lanes before release +approval, using `mock-openai/gpt-5.5` for release transport checks so they stay +deterministic and avoid normal provider-plugin startup. These live transport +gateways disable memory search; memory behavior stays covered by the QA parity +suites. Full release live media shards use `ghcr.io/openclaw/openclaw-live-media-runner:ubuntu-24.04`, which already has diff --git a/docs/reference/RELEASING.md b/docs/reference/RELEASING.md index c9f0cebec..118b05e1d 100644 --- a/docs/reference/RELEASING.md +++ b/docs/reference/RELEASING.md @@ -173,7 +173,7 @@ the maintainer-only release runbook. plugins. - Release checks now run in a separate manual workflow: `OpenClaw Release Checks` -- `OpenClaw Release Checks` also runs the QA Lab mock parity gate plus the fast +- `OpenClaw Release Checks` also runs the QA Lab mock parity lane plus the fast live Matrix profile and Telegram QA lane before release approval. The live lanes use the `qa-live-shared` environment; Telegram also uses Convex CI credential leases. Run the manual `QA-Lab - All Lanes` workflow with @@ -443,7 +443,7 @@ package mechanics. Release QA Lab coverage includes: -- mock parity gate comparing the OpenAI candidate lane against the Opus 4.6 +- mock parity lane comparing the OpenAI candidate lane against the Opus 4.6 baseline using the agentic parity pack - fast live Matrix QA profile using the `qa-live-shared` environment - live Telegram QA lane using Convex CI credential leases