From 2ae1f257881b8bab522cb84f192a2b778f8cacdd Mon Sep 17 00:00:00 2001 From: "openclaw-docs-sync[bot]" Date: Wed, 8 Apr 2026 16:06:11 +0000 Subject: [PATCH] chore(sync): mirror docs from openclaw/openclaw@4a51a1031d39f88d00f4f82615310ffd2527ce8d --- .openclaw-sync/source.json | 4 +-- docs/concepts/qa-e2e-automation.md | 44 ++++++++++++++++++++++++------ 2 files changed, 38 insertions(+), 10 deletions(-) diff --git a/.openclaw-sync/source.json b/.openclaw-sync/source.json index b80a94aa9..e068067b1 100644 --- a/.openclaw-sync/source.json +++ b/.openclaw-sync/source.json @@ -1,5 +1,5 @@ { "repository": "openclaw/openclaw", - "sha": "b77db8c0b610f975c15a418433c57fb9116b5473", - "syncedAt": "2026-04-08T16:03:45.732Z" + "sha": "4a51a1031d39f88d00f4f82615310ffd2527ce8d", + "syncedAt": "2026-04-08T16:06:11.151Z" } diff --git a/docs/concepts/qa-e2e-automation.md b/docs/concepts/qa-e2e-automation.md index 9cc32899d..5358f5b90 100644 --- a/docs/concepts/qa-e2e-automation.md +++ b/docs/concepts/qa-e2e-automation.md @@ -87,17 +87,45 @@ refs and write a judged Markdown report: ```bash pnpm openclaw qa character-eval \ - --model openai/gpt-5.4 \ - --model anthropic/claude-opus-4-6 \ - --model minimax/MiniMax-M2.7 \ - --judge-model openai/gpt-5.4 + --model openai/gpt-5.4,thinking=xhigh \ + --model openai/gpt-5.2,thinking=xhigh \ + --model anthropic/claude-opus-4-6,thinking=high \ + --model anthropic/claude-sonnet-4-6,thinking=high \ + --model minimax/MiniMax-M2.7,thinking=high \ + --model zai/glm-5.1,thinking=high \ + --model moonshot/kimi-k2.5,thinking=high \ + --model qwen/qwen3.6-plus,thinking=high \ + --model xiaomi/mimo-v2-pro,thinking=high \ + --model google/gemini-3.1-pro-preview,thinking=high \ + --judge-model openai/gpt-5.4,thinking=xhigh,fast \ + --judge-model anthropic/claude-opus-4-6,thinking=high ``` -The command runs local QA gateway child processes, not Docker. It preserves each -full transcript, records basic run stats, then asks the judge model in fast mode -with `xhigh` reasoning to rank the runs by naturalness, vibe, and humor. +The command runs local QA gateway child processes, not Docker. Character eval +scenarios should set the persona through `SOUL.md`, then run ordinary user turns +such as chat, workspace help, and small file tasks. The candidate model should +not be told that it is being evaluated. The command preserves each full +transcript, records basic run stats, then asks the judge models in fast mode with +`xhigh` reasoning to rank the runs by naturalness, vibe, and humor. +Candidate runs default to `high` thinking, with `xhigh` for OpenAI models that +support it. Override a specific candidate inline with +`--model provider/model,thinking=`. `--thinking ` still sets a +global fallback, and the older `--model-thinking ` form is +kept for compatibility. +OpenAI candidate refs default to fast mode so priority processing is used where +the provider supports it. Add `,fast`, `,no-fast`, or `,fast=false` inline when a +single candidate or judge needs an override. Pass `--fast` only when you want to +force fast mode on for every candidate model. Candidate and judge durations are +recorded in the report for benchmark analysis, but judge prompts explicitly say +not to rank by speed. When no candidate `--model` is passed, the character eval defaults to -`openai/gpt-5.4` and `anthropic/claude-opus-4-6`. +`openai/gpt-5.4`, `openai/gpt-5.2`, `anthropic/claude-opus-4-6`, +`anthropic/claude-sonnet-4-6`, `minimax/MiniMax-M2.7`, `zai/glm-5.1`, +`moonshot/kimi-k2.5`, `qwen/qwen3.6-plus`, `xiaomi/mimo-v2-pro`, and +`google/gemini-3.1-pro-preview`. +When no `--judge-model` is passed, the judges default to +`openai/gpt-5.4,thinking=xhigh,fast` and +`anthropic/claude-opus-4-6,thinking=high`. ## Related docs