From a004031785439b0ea8d504c414937682f0cf2e57 Mon Sep 17 00:00:00 2001 From: "openclaw-docs-sync[bot]" Date: Thu, 9 Apr 2026 00:04:56 +0000 Subject: [PATCH] chore(sync): mirror docs from openclaw/openclaw@39cc6b7dc78c0a80552454ad18b9d5bab86551ab --- .openclaw-sync/source.json | 4 ++-- docs/concepts/qa-e2e-automation.md | 18 +++++++++++------- docs/providers/qwen.md | 4 +++- 3 files changed, 16 insertions(+), 10 deletions(-) diff --git a/.openclaw-sync/source.json b/.openclaw-sync/source.json index 5a5263228..462621a63 100644 --- a/.openclaw-sync/source.json +++ b/.openclaw-sync/source.json @@ -1,5 +1,5 @@ { "repository": "openclaw/openclaw", - "sha": "78b6bedd104666ef4e9be51fd4f47a4718164fb4", - "syncedAt": "2026-04-08T23:48:41.027Z" + "sha": "39cc6b7dc78c0a80552454ad18b9d5bab86551ab", + "syncedAt": "2026-04-09T00:04:55.484Z" } diff --git a/docs/concepts/qa-e2e-automation.md b/docs/concepts/qa-e2e-automation.md index d4777afef..27eca3a5c 100644 --- a/docs/concepts/qa-e2e-automation.md +++ b/docs/concepts/qa-e2e-automation.md @@ -94,13 +94,13 @@ pnpm openclaw qa character-eval \ --model minimax/MiniMax-M2.7,thinking=high \ --model zai/glm-5.1,thinking=high \ --model moonshot/kimi-k2.5,thinking=high \ - --model qwen/qwen3.6-plus,thinking=high \ - --model xiaomi/mimo-v2-pro,thinking=high \ + --model qwen/qwen3.5-plus,thinking=high \ --model google/gemini-3.1-pro-preview,thinking=high \ --judge-model openai/gpt-5.4,thinking=xhigh,fast \ --judge-model anthropic/claude-opus-4-6,thinking=high \ - --concurrency 8 \ - --judge-concurrency 8 + --blind-judge-models \ + --concurrency 16 \ + --judge-concurrency 16 ``` The command runs local QA gateway child processes, not Docker. Character eval @@ -109,6 +109,10 @@ such as chat, workspace help, and small file tasks. The candidate model should not be told that it is being evaluated. The command preserves each full transcript, records basic run stats, then asks the judge models in fast mode with `xhigh` reasoning to rank the runs by naturalness, vibe, and humor. +Use `--blind-judge-models` when comparing providers: the judge prompt still gets +every transcript and run status, but candidate refs are replaced with neutral +labels such as `candidate-01`; the report maps rankings back to real refs after +parsing. Candidate runs default to `high` thinking, with `xhigh` for OpenAI models that support it. Override a specific candidate inline with `--model provider/model,thinking=`. `--thinking ` still sets a @@ -120,14 +124,14 @@ single candidate or judge needs an override. Pass `--fast` only when you want to force fast mode on for every candidate model. Candidate and judge durations are recorded in the report for benchmark analysis, but judge prompts explicitly say not to rank by speed. -Candidate and judge model runs both default to concurrency 8. Lower +Candidate and judge model runs both default to concurrency 16. Lower `--concurrency` or `--judge-concurrency` when provider limits or local gateway pressure make a run too noisy. When no candidate `--model` is passed, the character eval defaults to `openai/gpt-5.4`, `openai/gpt-5.2`, `anthropic/claude-opus-4-6`, `anthropic/claude-sonnet-4-6`, `minimax/MiniMax-M2.7`, `zai/glm-5.1`, -`moonshot/kimi-k2.5`, `qwen/qwen3.6-plus`, `xiaomi/mimo-v2-pro`, and -`google/gemini-3.1-pro-preview`. +`moonshot/kimi-k2.5`, `qwen/qwen3.5-plus`, and +`google/gemini-3.1-pro-preview` when no `--model` is passed. When no `--judge-model` is passed, the judges default to `openai/gpt-5.4,thinking=xhigh,fast` and `anthropic/claude-opus-4-6,thinking=high`. diff --git a/docs/providers/qwen.md b/docs/providers/qwen.md index 82c7205ce..d5d890449 100644 --- a/docs/providers/qwen.md +++ b/docs/providers/qwen.md @@ -88,7 +88,9 @@ requiring the built-in `qwen` provider id specifically. ## Built-in catalog -OpenClaw currently ships this bundled Qwen catalog: +OpenClaw currently ships this bundled Qwen catalog. The configured catalog is +endpoint-aware: Coding Plan configs omit models that are only known to work on +the Standard endpoint. | Model ref | Input | Context | Notes | | --------------------------- | ----------- | --------- | -------------------------------------------------- |