openclaw/projectclownfish
openclaw/projectclownfish
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: add autonomous cluster planning

Browse Source
This commit is contained in:
Vincent Koc 2026-04-25 04:25:23 -07:00
parent 53502f490e
commit 8ed31d94de
No known key found for this signature in database
16 changed files with 703 additions and 34 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
.github/workflows/cluster-worker.yml vendored
View File

@ -15,6 +15,7 @@ on:
options:
- plan
- execute
- autonomous
runner:
description: "Runner label, e.g. ubuntu-latest or a Blacksmith label"
required: true
@ -52,7 +53,7 @@ jobs:
GH_TOKEN: ${{ secrets.CLOWNFISH_GH_TOKEN }}
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
CLOWNFISH_ALLOWED_OWNER: ${{ vars.CLOWNFISH_ALLOWED_OWNER || 'openclaw' }}
CLOWNFISH_ALLOW_EXECUTE: ${{ inputs.mode == 'execute' && vars.CLOWNFISH_ALLOW_EXECUTE || '0' }}
CLOWNFISH_ALLOW_EXECUTE: ${{ (inputs.mode == 'execute' || inputs.mode == 'autonomous') && vars.CLOWNFISH_ALLOW_EXECUTE || '0' }}
CLOWNFISH_CODEX_BYPASS: ${{ inputs.codex_bypass && '1' || '0' }}
CLOWNFISH_MODEL: ${{ inputs.model }}
steps:
@ -89,7 +90,7 @@ jobs:
npm run worker -- "${args[@]}"
- name: Apply safe closure actions
if: ${{ inputs.mode == 'execute' && !inputs.dry_run }}
if: ${{ (inputs.mode == 'execute' || inputs.mode == 'autonomous') && !inputs.dry_run }}
run: npm run apply-result -- "${{ inputs.job }}" --latest
- name: Upload worker artifacts

9
.github/workflows/validate.yml vendored
View File

@ -30,5 +30,10 @@ jobs:
- name: Dry-run worker
run: npm run worker -- jobs/openclaw/cluster-example.md --mode plan --dry-run
- name: Check apply-result script
run: node --check scripts/apply-result.mjs
- name: Build offline autonomous artifact
run: npm run build-fix-artifact -- jobs/openclaw/autonomous-example.md --offline
- name: Check scripts
run: |
node --check scripts/apply-result.mjs
node --check scripts/plan-cluster.mjs

2
AGENTS.md
View File

@ -4,7 +4,7 @@ projectclownfish farms one GitHub issue/PR cluster to one isolated Codex worker.
## Hard Rules
- Default to `plan`; do not execute GitHub mutations unless the job says `mode: execute` and `CLOWNFISH_ALLOW_EXECUTE=1`.
- Default to `plan`; do not execute GitHub mutations unless the job says `mode: execute` or `mode: autonomous` and `CLOWNFISH_ALLOW_EXECUTE=1`.
- Re-fetch live GitHub state before any close, label, comment, merge, or fix action.
- If canonical choice is unclear, checks are failing, a PR has conflicts, or the cluster changed materially, stop with `needs_human`.
- Never print tokens, secrets, or full environment dumps.

13
README.md
View File

@ -23,6 +23,7 @@ Validate and render locally:
```bash
npm run validate:job -- jobs/openclaw/cluster-001.md
npm run render -- jobs/openclaw/cluster-001.md --mode plan
npm run build-fix-artifact -- jobs/openclaw/autonomous-example.md --offline
```
Run locally without calling Codex:
@ -74,12 +75,14 @@ Optional:
`plan` produces action recommendations only.
`execute` is gated by all of these:
`execute` and `autonomous` are gated by all of these:
- workflow input `mode=execute`
- job frontmatter `mode: execute`
- workflow input `mode=execute` or `mode=autonomous`
- job frontmatter with the same mode
- `CLOWNFISH_ALLOW_EXECUTE=1`
In execute mode Codex still returns JSON only. Projectclownfish applies safe closures deterministically from that JSON, using the ClawSweeper-style live-state and idempotency checks.
In execute and autonomous mode Codex still returns JSON only. Projectclownfish applies safe closures deterministically from that JSON, using the ClawSweeper-style live-state and idempotency checks.
Start with `plan` over a batch of clusters. Promote only boring, obvious work to `execute`.
`autonomous` also builds a live cluster preflight and fix artifact. It may recommend canonical fixes, merge paths, and post-merge closeouts, but direct GitHub mutations still flow through `apply-result`.
Start with `plan` over a batch of clusters. Promote boring, obvious closeout work to `execute`; use `autonomous` for clusters where duplicate closeout and canonical fix planning should happen together.

18
docs/OPERATIONS.md
View File

@ -16,9 +16,9 @@
```
4. Review artifacts from GitHub Actions.
5. Change selected jobs to `mode: execute`.
5. Change selected jobs to `mode: execute` or `mode: autonomous`.
6. Set repo variable `CLOWNFISH_ALLOW_EXECUTE=1` only for the execution window.
7. Dispatch execute jobs for reviewed clusters only. Execute workers still return JSON; `apply-result` performs the GitHub mutations afterward.
7. Dispatch execute/autonomous jobs for reviewed clusters only. Workers still return JSON; `apply-result` performs safe GitHub mutations afterward.
8. Reset `CLOWNFISH_ALLOW_EXECUTE=0`.
## Auto-Closure
@ -28,6 +28,7 @@
It only applies closure actions when all of these are true:
- the job and result are both `mode: execute`;
- or the job and result are both `mode: autonomous`;
- `CLOWNFISH_ALLOW_EXECUTE=1`;
- the job allows both `comment` and `close`;
- the action is `close_duplicate`, `close_superseded`, or `close_fixed_by_candidate`;
@ -37,6 +38,17 @@ It only applies closure actions when all of these are true:
The applicator writes an idempotency marker into the close comment before closing. Re-runs skip already-applied comments/closures instead of posting twice.
## Autonomous Flow
`npm run build-fix-artifact -- <job.md>` hydrates the job refs, linked refs, current `main`, PR files, commits, and checks, then writes:
- `cluster-plan.json`: live cluster inventory and canonical candidates;
- `fix-artifact.json`: drive plan, gates, permissions, and per-item matrix.
Autonomous workers receive those artifacts in the prompt. They can emit instant close actions for high-confidence duplicate/superseded/fixed-by-candidate items, and they can emit `build_fix_artifact` when a canonical fix PR is needed.
They still must not mutate GitHub directly. Missing checkout, failing checks, conflicts, unclear canonical choice, or stale item state means `needs_human`.
## Runner Strategy
Use `ubuntu-latest` for correctness smoke tests.
@ -62,7 +74,7 @@ Do not put tokens in job files.
## Promotion Rules
Promote from `plan` to `execute` only when:
Promote from `plan` to `execute` or `autonomous` only when:
- the canonical item is clear;
- no unique reports are being closed;

46
jobs/openclaw/autonomous-example.md Normal file
View File

@ -0,0 +1,46 @@
---
repo: openclaw/openclaw
cluster_id: example-autonomous-cron-timeout
mode: autonomous
allowed_actions:
- comment
- label
- close
- fix
- raise_pr
blocked_actions:
- force_push
- bypass_checks
require_human_for:
- failing_checks
- conflicting_prs
- unclear_canonical
- broad_code_delta
- missing_target_checkout
canonical:
- "#40868"
candidates:
- "#40868"
- "#41272"
cluster_refs:
- "#40868"
- "#41272"
allow_instant_close: true
allow_fix_pr: true
allow_merge: false
allow_post_merge_close: true
canonical_hint: "#40868 is the likely canonical report; verify live state."
notes: "Example only. Replace with a real cluster exported from ghcrawl or curated by hand."
---
# Autonomous Cluster Task
Use the dedupe autonomous workflow to classify the cluster, close boring covered duplicates through structured actions, and build a fix artifact if the canonical issue still needs code work.
## Goal
Find the canonical fix path, preserve contributor credit, and leave an auditable artifact that can be replayed or escalated.
## Context
Paste ghcrawl cluster summary, LLM key summaries, top touched files, reproduction notes, candidate PRs, and operator notes here.

2
package.json
View File

@ -7,6 +7,8 @@
"validate": "node scripts/validate-all.mjs",
"validate:job": "node scripts/validate-job.mjs",
"render": "node scripts/render-prompt.mjs",
"plan-cluster": "node scripts/plan-cluster.mjs",
"build-fix-artifact": "node scripts/plan-cluster.mjs",
"worker": "node scripts/run-worker.mjs",
"apply-result": "node scripts/apply-result.mjs",
"dispatch": "node scripts/dispatch-jobs.mjs"

49
prompts/autonomous.md Normal file
View File

@ -0,0 +1,49 @@
# Autonomous Mode
Autonomous mode is stricter than execute mode. You may do broader reasoning, but you still must not mutate GitHub directly.
Scope:
- Start only from refs in the job file and refs linked from those item bodies, comments, review threads, closing refs, commits, or PR descriptions.
- Do not run broad GitHub search unless the job explicitly says so.
- Use the provided cluster preflight artifact and fix artifact as your starting inventory.
- If the cluster changed materially since preflight, return `needs_human`.
Before drive mode:
1. Fetch current `main` for the target repo and decide whether the behavior is already fixed, obsolete, or still real.
2. Hydrate every provided and linked issue/PR with bodies, comments, labels, state, checks, review state, linked closing refs, and touched files when available.
3. Classify each item as `canonical`, `duplicate`, `related`, `superseded`, `independent`, `fixed_by_candidate`, or `needs_human`.
4. Identify the canonical path:
- already merged PR/commit on `main`;
- open PR that is mergeable or repairable;
- new fix PR needed because the bug is real and no viable PR exists.
5. Do not emit closure actions until the canonical path is explicit.
Instant close actions:
- Emit `close_duplicate`, `close_superseded`, or `close_fixed_by_candidate` only for high-confidence covered items.
- Include `target_updated_at`, `target_kind`, `canonical` or `candidate_fix`, contributor-credit preserving `comment`, evidence, and a stable `idempotency_key`.
- Leave independent or unclear reports open as `keep_independent`, `keep_related`, or `needs_human`.
Fix artifact actions:
- If no viable canonical PR exists and the bug still reproduces, emit `fix_needed` plus `build_fix_artifact`.
- The `build_fix_artifact` action must include affected surfaces, likely files, linked issues/PRs, validation commands, changelog requirement, credit notes, and a PR title/body plan.
- If a target checkout is unavailable or unsafe, do not pretend to patch. Return the artifact and mark implementation as `needs_human`.
Merge and post-merge close:
- Recommend `merge_canonical` only when checks, review state, conflicts, changelog, and validation are clean.
- Recommend `post_merge_close` only after a canonical fix is merged or already present on current `main`.
- Preserve contributor credit in all closeout comments.
Required result shape:
- `canonical`, `canonical_issue`, or `canonical_pr` with full URL when known.
- Per-item action matrix in `actions`.
- Evidence and command/result summary in action evidence.
- `fix_artifact` object when a fix path is needed.
- `needs_human` entries for anything blocked by ambiguity, stale state, failing checks, missing checkout, or missing permissions.
Return structured JSON only. Do not close, comment, label, merge, push, or open PRs directly.

1
prompts/worker-system.md
View File

@ -24,6 +24,7 @@ Execution guard:
- In `plan` mode, do not mutate GitHub.
- In `execute` mode, do not mutate GitHub directly; emit structured actions for the applicator.
- In `autonomous` mode, do not mutate GitHub directly; emit structured actions and fix artifacts for Projectclownfish scripts to apply.
- If any safety condition is not met, return `needs_human`.
Final answer must match `schemas/codex-result.schema.json`.

18
schemas/codex-result.schema.json
View File

@ -15,7 +15,7 @@
"type": "string"
},
"mode": {
"enum": ["plan", "execute"]
"enum": ["plan", "execute", "autonomous"]
},
"summary": {
"type": "string"
@ -37,7 +37,11 @@
"keep_related",
"keep_independent",
"merge_candidate",
"merge_canonical",
"fix_needed",
"build_fix_artifact",
"open_fix_pr",
"post_merge_close",
"needs_human",
"close",
"close_duplicate",
@ -102,6 +106,18 @@
"items": {
"type": "string"
}
},
"canonical": {
"type": "string"
},
"canonical_issue": {
"type": "string"
},
"canonical_pr": {
"type": "string"
},
"fix_artifact": {
"type": "object"
}
}
}

26
schemas/job.schema.json
View File

@ -14,7 +14,7 @@
"minLength": 1
},
"mode": {
"enum": ["plan", "execute"]
"enum": ["plan", "execute", "autonomous"]
},
"allowed_actions": {
"type": "array",
@ -49,6 +49,30 @@
"pattern": "^#?[0-9]+$"
}
},
"cluster_refs": {
"type": "array",
"items": {
"type": "string"
}
},
"allow_instant_close": {
"type": "boolean"
},
"allow_fix_pr": {
"type": "boolean"
},
"allow_merge": {
"type": "boolean"
},
"allow_post_merge_close": {
"type": "boolean"
},
"canonical_hint": {
"type": "string"
},
"target_checkout": {
"type": "string"
},
"notes": {
"type": "string"
}

9
scripts/apply-result.mjs
View File

@ -40,8 +40,8 @@ if (errors.length > 0) {
assertAllowedOwner(job.frontmatter.repo, process.env.CLOWNFISH_ALLOWED_OWNER);
if (job.frontmatter.mode !== "execute") {
throw new Error("refusing apply: job frontmatter mode is not execute");
if (!["execute", "autonomous"].includes(job.frontmatter.mode)) {
throw new Error("refusing apply: job frontmatter mode is not execute or autonomous");
}
if (process.env.CLOWNFISH_ALLOW_EXECUTE !== "1") {
throw new Error("refusing apply: CLOWNFISH_ALLOW_EXECUTE must be 1");
@ -63,9 +63,12 @@ if (result.cluster_id !== job.frontmatter.cluster_id) {
`result cluster ${result.cluster_id} does not match job cluster ${job.frontmatter.cluster_id}`,
);
}
if (result.mode !== "execute") {
if (!["execute", "autonomous"].includes(result.mode)) {
throw new Error(`refusing apply: result mode is ${result.mode}`);
}
if (result.mode !== job.frontmatter.mode) {
throw new Error(`refusing apply: result mode ${result.mode} does not match job mode ${job.frontmatter.mode}`);
}
const report = {
repo: result.repo,

2
scripts/dispatch-jobs.mjs
View File

@ -11,7 +11,7 @@ const workflow = args.workflow ?? "cluster-worker.yml";
const files = args._;
if (files.length === 0) {
console.error("usage: node scripts/dispatch-jobs.mjs <job.md> [...] [--mode plan|execute] [--runner label]");
console.error("usage: node scripts/dispatch-jobs.mjs <job.md> [...] [--mode plan|execute|autonomous] [--runner label]");
process.exit(2);
}

65
scripts/lib.mjs
View File

@ -86,10 +86,17 @@ export function validateJob(job) {
if (typeof fm.repo === "string" && !/^[A-Za-z0-9_.-]+\/[A-Za-z0-9_.-]+$/.test(fm.repo)) {
errors.push("repo must be owner/repo");
}
if (fm.mode && !["plan", "execute"].includes(fm.mode)) {
errors.push("mode must be plan or execute");
if (fm.mode && !["plan", "execute", "autonomous"].includes(fm.mode)) {
errors.push("mode must be plan, execute, or autonomous");
}
for (const key of ["allowed_actions", "blocked_actions", "require_human_for", "canonical", "candidates"]) {
for (const key of [
"allowed_actions",
"blocked_actions",
"require_human_for",
"canonical",
"candidates",
"cluster_refs",
]) {
if (fm[key] !== undefined && !Array.isArray(fm[key])) {
errors.push(`${key} must be a list`);
}
@ -104,6 +111,26 @@ export function validateJob(job) {
errors.push(`candidate refs must look like #123: ${ref}`);
}
}
for (const ref of fm.cluster_refs ?? []) {
if (!isGithubRef(ref)) {
errors.push(`cluster_refs must look like #123 or a GitHub issue/PR URL: ${ref}`);
}
}
for (const key of [
"allow_instant_close",
"allow_fix_pr",
"allow_merge",
"allow_post_merge_close",
]) {
if (fm[key] !== undefined && typeof fm[key] !== "boolean") {
errors.push(`${key} must be true or false`);
}
}
for (const key of ["canonical_hint", "target_checkout"]) {
if (fm[key] !== undefined && typeof fm[key] !== "string") {
errors.push(`${key} must be a string`);
}
}
return errors;
}
@ -120,10 +147,15 @@ function requireArray(errors, object, key) {
}
}
export function renderPrompt(job, requestedMode) {
export function renderPrompt(job, requestedMode, context = {}) {
const mode = requestedMode ?? job.frontmatter.mode;
const modePrompt = mode === "execute" ? "prompts/execute.md" : "prompts/plan-only.md";
return [
const modePrompt =
mode === "autonomous"
? "prompts/autonomous.md"
: mode === "execute"
? "prompts/execute.md"
: "prompts/plan-only.md";
const parts = [
readText("prompts/worker-system.md"),
readText(modePrompt),
"## Dedupe policy",
@ -136,9 +168,28 @@ export function renderPrompt(job, requestedMode) {
"```md",
job.raw.trim(),
"```",
];
for (const [title, filePath] of [
["Cluster preflight artifact", context.clusterPlanPath],
["Fix artifact", context.fixArtifactPath],
]) {
if (!filePath) continue;
const absolute = path.resolve(filePath);
parts.push(`## ${title}`, `Path: \`${path.relative(repoRoot(), absolute)}\``, "```json", fs.readFileSync(absolute, "utf8").trim(), "```");
}
parts.push(
"## Required final output",
"Return JSON matching `schemas/codex-result.schema.json` and nothing else.",
].join("\n\n");
);
return parts.join("\n\n");
}
function isGithubRef(value) {
const text = String(value ?? "");
return /^#?[0-9]+$/.test(text) || /^https:\/\/github\.com\/[^/]+\/[^/]+\/(?:issues|pull)\/[0-9]+/.test(text);
}
export function parseArgs(argv) {

438
scripts/plan-cluster.mjs Normal file
View File

@ -0,0 +1,438 @@
#!/usr/bin/env node
import fs from "node:fs";
import path from "node:path";
import { execFileSync } from "node:child_process";
import {
assertAllowedOwner,
makeRunDir,
parseArgs,
parseJob,
repoRoot,
validateJob,
} from "./lib.mjs";
const MAX_LINKED_REFS = Number(process.env.CLOWNFISH_MAX_LINKED_REFS ?? 25);
const args = parseArgs(process.argv.slice(2));
const jobPath = args._[0];
const offline = Boolean(args.offline);
if (!jobPath) {
console.error("usage: node scripts/plan-cluster.mjs <job.md> [--run-dir dir] [--offline]");
process.exit(2);
}
const job = parseJob(jobPath);
const errors = validateJob(job);
if (errors.length > 0) {
console.error(errors.join("\n"));
process.exit(1);
}
assertAllowedOwner(job.frontmatter.repo, process.env.CLOWNFISH_ALLOWED_OWNER);
const runDir = args["run-dir"]
? path.resolve(String(args["run-dir"]))
: makeRunDir(job, `${job.frontmatter.mode}-cluster-plan`);
fs.mkdirSync(runDir, { recursive: true });
const seedRefs = uniqueRefs([
...(job.frontmatter.canonical ?? []),
...(job.frontmatter.candidates ?? []),
...(job.frontmatter.cluster_refs ?? []),
].map((ref) => normalizeRef(job.frontmatter.repo, ref)));
const externalRefs = seedRefs.filter((ref) => ref.repo !== job.frontmatter.repo);
const seedNumbers = seedRefs
.filter((ref) => ref.repo === job.frontmatter.repo)
.map((ref) => ref.number);
const items = new Map();
const linkedRefs = new Map();
const pending = [...new Set(seedNumbers)].map((number) => ({ number, depth: 0 }));
let linkedHydrateCount = 0;
const branch = offline ? offlineMainBranch(job.frontmatter.repo) : fetchMainBranch(job.frontmatter.repo);
while (pending.length > 0) {
const next = pending.shift();
const number = next?.number;
if (!number || items.has(number)) continue;
const item = offline
? offlineItem(job.frontmatter.repo, number, job)
: hydrateItem(job.frontmatter.repo, number);
items.set(number, item);
if (offline || next.depth > 0) continue;
for (const linked of extractLinkedRefs(job.frontmatter.repo, item)) {
const key = `${linked.repo}#${linked.number}`;
linkedRefs.set(key, linked);
const alreadyPending = pending.some((entry) => entry.number === linked.number);
if (
linked.repo === job.frontmatter.repo &&
!items.has(linked.number) &&
!alreadyPending &&
linkedHydrateCount < MAX_LINKED_REFS
) {
pending.push({ number: linked.number, depth: next.depth + 1 });
linkedHydrateCount += 1;
}
}
}
const itemList = [...items.values()].sort((left, right) => left.number - right.number);
const plan = {
repo: job.frontmatter.repo,
cluster_id: job.frontmatter.cluster_id,
mode: job.frontmatter.mode,
source_job: job.relativePath,
generated_at: new Date().toISOString(),
offline,
main: branch,
scope: {
seed_refs: seedRefs.map(formatNormalizedRef),
linked_refs: [...linkedRefs.values()].map(formatNormalizedRef).sort(),
external_refs: externalRefs.map(formatNormalizedRef).sort(),
expansion_policy:
"Autonomous mode may hydrate only job-provided refs and first-hop refs linked from those items.",
max_linked_refs: MAX_LINKED_REFS,
},
items: itemList.map((item) => summarizeItem(item, job)),
canonical_candidates: canonicalCandidates(itemList, job),
safety_gates: [
"re-fetch live state before every close/comment/label/merge/fix action",
"stop with needs_human when canonical choice is unclear",
"stop with needs_human when checks fail, conflicts exist, or cluster state changes",
"preserve contributor credit in every closeout comment",
],
};
const fixArtifact = buildFixArtifact(plan, job);
const clusterPlanPath = path.join(runDir, "cluster-plan.json");
const fixArtifactPath = path.join(runDir, "fix-artifact.json");
fs.writeFileSync(clusterPlanPath, `${JSON.stringify(plan, null, 2)}\n`);
fs.writeFileSync(fixArtifactPath, `${JSON.stringify(fixArtifact, null, 2)}\n`);
console.log(
JSON.stringify(
{
cluster_plan: path.relative(repoRoot(), clusterPlanPath),
fix_artifact: path.relative(repoRoot(), fixArtifactPath),
items: itemList.length,
offline,
},
null,
2,
),
);
function hydrateItem(repo, number) {
const issue = ghJson(["api", `repos/${repo}/issues/${number}`]);
const comments = ghPaged(`repos/${repo}/issues/${number}/comments`);
const pullRequest = issue.pull_request ? ghJson(["api", `repos/${repo}/pulls/${number}`]) : null;
const files = pullRequest ? ghPaged(`repos/${repo}/pulls/${number}/files`) : [];
const commits = pullRequest ? ghPaged(`repos/${repo}/pulls/${number}/commits`) : [];
const checks = pullRequest ? ghPrChecks(repo, number) : [];
return {
repo,
number,
ref: `#${number}`,
kind: pullRequest ? "pull_request" : "issue",
state: issue.state,
title: issue.title,
html_url: issue.html_url,
author: issue.user?.login,
author_association: issue.author_association,
labels: (issue.labels ?? []).map((label) => label.name ?? label).filter(Boolean),
created_at: issue.created_at,
updated_at: issue.updated_at,
closed_at: issue.closed_at,
body: issue.body ?? "",
body_excerpt: excerpt(issue.body),
comments: comments.map((comment) => ({
author: comment.user?.login,
author_association: comment.author_association,
created_at: comment.created_at,
updated_at: comment.updated_at,
body: comment.body ?? "",
body_excerpt: excerpt(comment.body),
})),
pull_request: pullRequest
? {
draft: pullRequest.draft,
mergeable: pullRequest.mergeable,
mergeable_state: pullRequest.mergeable_state,
base_ref: pullRequest.base?.ref,
head_ref: pullRequest.head?.ref,
head_repo: pullRequest.head?.repo?.full_name,
head_sha: pullRequest.head?.sha,
additions: pullRequest.additions,
deletions: pullRequest.deletions,
changed_files: pullRequest.changed_files,
files: files.map((file) => ({
filename: file.filename,
status: file.status,
additions: file.additions,
deletions: file.deletions,
})),
commits: commits.map((commit) => ({
sha: commit.sha,
message: firstLine(commit.commit?.message),
author: commit.author?.login ?? commit.commit?.author?.name,
})),
checks,
}
: null,
};
}
function summarizeItem(item, job) {
return {
repo: item.repo,
ref: item.ref,
number: item.number,
kind: item.kind,
state: item.state,
title: item.title,
url: item.html_url,
author: item.author,
author_association: item.author_association,
labels: item.labels,
created_at: item.created_at,
updated_at: item.updated_at,
closed_at: item.closed_at,
body_excerpt: item.body_excerpt,
comments_count: item.comments.length,
classification_hint: classificationHint(item, job),
pull_request: item.pull_request
? {
draft: item.pull_request.draft,
mergeable: item.pull_request.mergeable,
mergeable_state: item.pull_request.mergeable_state,
base_ref: item.pull_request.base_ref,
head_ref: item.pull_request.head_ref,
head_repo: item.pull_request.head_repo,
head_sha: item.pull_request.head_sha,
changed_files: item.pull_request.changed_files,
additions: item.pull_request.additions,
deletions: item.pull_request.deletions,
files: item.pull_request.files,
commits: item.pull_request.commits,
checks: item.pull_request.checks,
}
: null,
};
}
function buildFixArtifact(plan, job) {
return {
repo: plan.repo,
cluster_id: plan.cluster_id,
mode: plan.mode,
generated_at: plan.generated_at,
source_job: plan.source_job,
target_checkout: job.frontmatter.target_checkout ?? null,
permissions: {
allow_instant_close: job.frontmatter.allow_instant_close === true,
allow_fix_pr: job.frontmatter.allow_fix_pr === true,
allow_merge: job.frontmatter.allow_merge === true,
allow_post_merge_close: job.frontmatter.allow_post_merge_close === true,
},
canonical_candidates: plan.canonical_candidates,
item_matrix: plan.items.map((item) => ({
ref: item.ref,
kind: item.kind,
state: item.state,
updated_at: item.updated_at,
hint: item.classification_hint,
})),
drive_plan: {
instant_close:
job.frontmatter.allow_instant_close === true
? "Worker may emit close_duplicate, close_superseded, or close_fixed_by_candidate actions only with live target_updated_at and canonical/candidate evidence."
: "Disabled by job frontmatter.",
canonical_fix:
job.frontmatter.allow_fix_pr === true
? "If no viable canonical PR exists and the bug still reproduces, emit fix_needed plus a build_fix_artifact action with files, tests, changelog, and PR plan."
: "Worker may identify canonical fixes but must not plan a fix PR.",
merge:
job.frontmatter.allow_merge === true
? "Worker may recommend merge_canonical only after checks, review state, conflicts, and changelog are clean."
: "Merge recommendations must stay non-mutating.",
post_merge_close:
job.frontmatter.allow_post_merge_close === true
? "After canonical fix confirmation, worker may emit post_merge_close closeout actions for covered refs."
: "Post-merge closure disabled by job frontmatter.",
},
required_validation: [
"prove current main behavior",
"hydrate every provided and linked item before classification",
"show canonical URL or explain needs_human",
"include targeted tests and changelog plan for fix artifacts",
"include full GitHub URLs in closure rationale",
],
};
}
function canonicalCandidates(items, job) {
const canonicalNumbers = new Set((job.frontmatter.canonical ?? []).map((ref) => normalizeRef(job.frontmatter.repo, ref).number));
return items
.filter((item) => canonicalNumbers.has(item.number) || item.kind === "pull_request")
.map((item) => ({
ref: item.ref,
kind: item.kind,
state: item.state,
title: item.title,
url: item.html_url,
hint: classificationHint(item, job),
checks: item.pull_request?.checks ?? [],
}));
}
function classificationHint(item, job) {
const canonicalNumbers = new Set((job.frontmatter.canonical ?? []).map((ref) => normalizeRef(job.frontmatter.repo, ref).number));
if (canonicalNumbers.has(item.number)) return "canonical_hint";
if (item.state !== "open") return "already_closed";
if (item.kind === "pull_request" && item.pull_request?.draft === false) return "open_pr_candidate";
if (item.kind === "pull_request") return "draft_pr_candidate";
return "open_issue_candidate";
}
function extractLinkedRefs(defaultRepo, item) {
const texts = [
item.title,
item.body,
...item.comments.map((comment) => comment.body),
item.pull_request?.commits?.map((commit) => commit.message).join("\n"),
];
return uniqueRefs(texts.flatMap((text) => refsFromText(defaultRepo, text)));
}
function refsFromText(defaultRepo, text) {
const refs = [];
const ownerRepo = defaultRepo.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
const urlPattern = new RegExp(
`https://github\\.com/(${ownerRepo}|[A-Za-z0-9_.-]+/[A-Za-z0-9_.-]+)/(?:issues|pull)/(\\d+)`,
"g",
);
for (const match of String(text ?? "").matchAll(urlPattern)) {
refs.push(normalizeRef(defaultRepo, `https://github.com/${match[1]}/issues/${match[2]}`));
}
for (const match of String(text ?? "").matchAll(/(^|[^A-Za-z0-9_])#(\d+)\b/g)) {
refs.push({ repo: defaultRepo, number: Number(match[2]) });
}
return refs;
}
function normalizeRef(defaultRepo, value) {
const text = String(value ?? "").trim();
const shorthand = text.match(/^#?(\d+)$/);
if (shorthand) return { repo: defaultRepo, number: Number(shorthand[1]) };
const url = text.match(/^https:\/\/github\.com\/([^/]+\/[^/]+)\/(?:issues|pull)\/(\d+)/);
if (url) return { repo: url[1], number: Number(url[2]) };
return { repo: defaultRepo, number: 0 };
}
function uniqueRefs(refs) {
const seen = new Set();
const out = [];
for (const ref of refs) {
if (!ref?.repo || !ref.number) continue;
const key = `${ref.repo}#${ref.number}`;
if (seen.has(key)) continue;
seen.add(key);
out.push(ref);
}
return out;
}
function formatNormalizedRef(ref) {
return ref.repo === job.frontmatter.repo ? `#${ref.number}` : `https://github.com/${ref.repo}/issues/${ref.number}`;
}
function fetchMainBranch(repo) {
const branch = ghJson(["api", `repos/${repo}/branches/main`]);
return {
name: "main",
sha: branch.commit?.sha,
url: branch._links?.html,
};
}
function offlineMainBranch(repo) {
return {
name: "main",
sha: null,
url: `https://github.com/${repo}/tree/main`,
note: "offline mode did not fetch current main",
};
}
function offlineItem(repo, number, job) {
return {
repo,
number,
ref: `#${number}`,
kind: "unknown",
state: "unknown",
title: `offline seed #${number}`,
html_url: `https://github.com/${repo}/issues/${number}`,
author: null,
author_association: null,
labels: [],
created_at: null,
updated_at: null,
closed_at: null,
body: job.body,
body_excerpt: excerpt(job.body),
comments: [],
pull_request: null,
};
}
function ghJson(ghArgs) {
const text = execFileSync("gh", ghArgs, {
cwd: repoRoot(),
encoding: "utf8",
env: process.env,
maxBuffer: 64 * 1024 * 1024,
stdio: ["ignore", "pipe", "pipe"],
}).trim();
return JSON.parse(text || "null");
}
function ghPaged(apiPath) {
const pages = ghJson(["api", apiPath, "--paginate", "--slurp"]);
if (!Array.isArray(pages)) return [];
return pages.flatMap((page) => (Array.isArray(page) ? page : []));
}
function ghPrChecks(repo, number) {
try {
const text = execFileSync(
"gh",
["pr", "checks", String(number), "--repo", repo, "--json", "name,state,bucket,link"],
{
cwd: repoRoot(),
encoding: "utf8",
env: process.env,
maxBuffer: 16 * 1024 * 1024,
stdio: ["ignore", "pipe", "pipe"],
},
).trim();
return JSON.parse(text || "[]");
} catch (error) {
return [{ error: firstLine(error?.stderr || error?.message || String(error)) }];
}
}
function excerpt(text, limit = 1200) {
const value = String(text ?? "").replace(/\s+/g, " ").trim();
if (value.length <= limit) return value;
return `${value.slice(0, limit - 3)}...`;
}
function firstLine(text) {
return String(text ?? "").split(/\r?\n/)[0] ?? "";
}

34
scripts/run-worker.mjs
View File

@ -19,11 +19,11 @@ const dryRun = Boolean(args["dry-run"] || process.env.CLOWNFISH_DRY_RUN === "1")
const model = args.model ?? process.env.CLOWNFISH_MODEL ?? "gpt-5.4";
if (!jobPath) {
console.error("usage: node scripts/run-worker.mjs <job.md> --mode plan|execute [--dry-run]");
console.error("usage: node scripts/run-worker.mjs <job.md> --mode plan|execute|autonomous [--dry-run]");
process.exit(2);
}
if (!["plan", "execute"].includes(mode)) {
console.error("mode must be plan or execute");
if (!["plan", "execute", "autonomous"].includes(mode)) {
console.error("mode must be plan, execute, or autonomous");
process.exit(2);
}
@ -36,12 +36,12 @@ if (errors.length > 0) {
assertAllowedOwner(job.frontmatter.repo, process.env.CLOWNFISH_ALLOWED_OWNER);
if (mode === "execute") {
if (job.frontmatter.mode !== "execute") {
throw new Error("refusing execute: job frontmatter mode is not execute");
if ((mode === "execute" || mode === "autonomous") && !dryRun) {
if (job.frontmatter.mode !== mode) {
throw new Error(`refusing ${mode}: job frontmatter mode is not ${mode}`);
}
if (process.env.CLOWNFISH_ALLOW_EXECUTE !== "1") {
throw new Error("refusing execute: CLOWNFISH_ALLOW_EXECUTE must be 1");
throw new Error(`refusing ${mode}: CLOWNFISH_ALLOW_EXECUTE must be 1`);
}
}
@ -49,7 +49,25 @@ const runDir = makeRunDir(job, mode);
const promptPath = path.join(runDir, "prompt.md");
const resultPath = path.join(runDir, "result.json");
const transcriptPath = path.join(runDir, "codex.jsonl");
const prompt = renderPrompt(job, mode);
const promptContext = {};
if (mode === "autonomous") {
const plannerArgs = ["scripts/plan-cluster.mjs", jobPath, "--run-dir", runDir];
if (dryRun) plannerArgs.push("--offline");
const planner = spawnSync(process.execPath, plannerArgs, {
cwd: repoRoot(),
encoding: "utf8",
env: process.env,
});
if (planner.status !== 0) {
console.error(planner.stderr || planner.stdout);
process.exit(planner.status ?? 1);
}
promptContext.clusterPlanPath = path.join(runDir, "cluster-plan.json");
promptContext.fixArtifactPath = path.join(runDir, "fix-artifact.json");
}
const prompt = renderPrompt(job, mode, promptContext);
fs.writeFileSync(promptPath, prompt);