feat: add safe closure applicator

This commit is contained in:
Vincent Koc 2026-04-25 04:14:37 -07:00
parent 48255ae333
commit 53502f490e
No known key found for this signature in database
13 changed files with 500 additions and 19 deletions

View File

@ -38,6 +38,8 @@ on:
permissions:
contents: read
issues: write
pull-requests: write
env:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
@ -86,6 +88,10 @@ jobs:
fi
npm run worker -- "${args[@]}"
- name: Apply safe closure actions
if: ${{ inputs.mode == 'execute' && !inputs.dry_run }}
run: npm run apply-result -- "${{ inputs.job }}" --latest
- name: Upload worker artifacts
if: always()
uses: actions/upload-artifact@v4

View File

@ -29,3 +29,6 @@ jobs:
- name: Dry-run worker
run: npm run worker -- jobs/openclaw/cluster-example.md --mode plan --dry-run
- name: Check apply-result script
run: node --check scripts/apply-result.mjs

1
.gitignore vendored
View File

@ -4,5 +4,6 @@ node_modules/
!.env.example
.projectclownfish/runs/*
!.projectclownfish/runs/.gitkeep
.projectclownfish/payloads/
results/**/*.local.*
*.log

View File

@ -31,6 +31,14 @@ Run locally without calling Codex:
npm run worker -- jobs/openclaw/cluster-001.md --mode plan --dry-run
```
Apply a reviewed execute result:
```bash
CLOWNFISH_ALLOW_EXECUTE=1 npm run apply-result -- jobs/openclaw/cluster-001.md --latest
```
`apply-result` is the only path that mutates GitHub. It re-fetches the target issue/PR, verifies `target_updated_at`, skips maintainer-authored items, posts an idempotent close comment, then closes only duplicate, superseded, or fixed-by-candidate actions.
Dispatch one worker:
```bash
@ -72,4 +80,6 @@ Optional:
- job frontmatter `mode: execute`
- `CLOWNFISH_ALLOW_EXECUTE=1`
In execute mode Codex still returns JSON only. Projectclownfish applies safe closures deterministically from that JSON, using the ClawSweeper-style live-state and idempotency checks.
Start with `plan` over a batch of clusters. Promote only boring, obvious work to `execute`.

View File

@ -18,9 +18,25 @@
4. Review artifacts from GitHub Actions.
5. Change selected jobs to `mode: execute`.
6. Set repo variable `CLOWNFISH_ALLOW_EXECUTE=1` only for the execution window.
7. Dispatch execute jobs for reviewed clusters only.
7. Dispatch execute jobs for reviewed clusters only. Execute workers still return JSON; `apply-result` performs the GitHub mutations afterward.
8. Reset `CLOWNFISH_ALLOW_EXECUTE=0`.
## Auto-Closure
`npm run apply-result -- <job.md> --latest` is the deterministic mutation path.
It only applies closure actions when all of these are true:
- the job and result are both `mode: execute`;
- `CLOWNFISH_ALLOW_EXECUTE=1`;
- the job allows both `comment` and `close`;
- the action is `close_duplicate`, `close_superseded`, or `close_fixed_by_candidate`;
- the action includes a canonical/candidate fix ref and live `target_updated_at`;
- GitHub still reports the same `updated_at`;
- the target is open and not maintainer-authored.
The applicator writes an idempotency marker into the close comment before closing. Re-runs skip already-applied comments/closures instead of posting twice.
## Runner Strategy
Use `ubuntu-latest` for correctness smoke tests.
@ -52,4 +68,5 @@ Promote from `plan` to `execute` only when:
- no unique reports are being closed;
- comments preserve contributor credit;
- idempotency keys are present;
- `target_updated_at` was fetched from live GitHub state;
- high-risk work is marked `needs_human`.

View File

@ -4,6 +4,7 @@ Only close when:
- the item is open;
- it is a true duplicate or superseded by a clear canonical item;
- it is clearly covered by a candidate fix that should own validation and follow-up;
- a clear comment has been posted first;
- the comment preserves credit and gives a reopen path;
- the action is allowed by the job frontmatter.
@ -24,3 +25,5 @@ Never close:
- active maintainer discussion;
- assigned work in progress;
- contributor PR with useful code that should be merged or credited.
Auto-closure payloads must include `target_updated_at`. The applicator will re-fetch live GitHub state and skip the close if the target changed after review.

View File

@ -8,6 +8,7 @@ Prefer these outcomes:
- `duplicate`: same root cause, same user-visible failure, no unique remaining work.
- `related`: same area or symptom family, but meaningfully different root cause or scope.
- `superseded`: PR or issue replaced by a better candidate.
- `fixed_by_candidate`: issue/report is covered by a specific candidate PR or fix path that should own validation.
- `independent`: should not be closed or merged as part of this cluster.
- `needs_human`: ambiguous, risky, changed live state, failing checks, unclear author credit, or broad code delta.
@ -20,3 +21,5 @@ Evidence order:
5. Cluster notes and ghcrawl summaries.
Do not close based on title similarity alone.
When recommending auto-closure, include the canonical or candidate fix ref and the live target `updated_at` value. Missing live state should become `needs_human`, not a close.

View File

@ -8,6 +8,7 @@
"validate:job": "node scripts/validate-job.mjs",
"render": "node scripts/render-prompt.mjs",
"worker": "node scripts/run-worker.mjs",
"apply-result": "node scripts/apply-result.mjs",
"dispatch": "node scripts/dispatch-jobs.mjs"
},
"engines": {

View File

@ -1,24 +1,27 @@
# Execute Mode
Execute only the actions that are explicitly allowed by the job.
Execute mode still returns structured JSON first. Do not mutate GitHub directly.
Before each mutation:
The runner applies safe closure actions after your JSON passes validation. Your job is to classify the cluster and emit auditable actions that the deterministic GitHub applicator can replay.
1. re-fetch live state;
2. check if the action already happened;
3. build an idempotency key;
4. perform the smallest safe mutation;
5. record the before/after state.
For each target action, include:
Allowed mutation commands may include:
- `target`: issue/PR ref like `#123`
- `action`: one of `keep_canonical`, `keep_related`, `keep_independent`, `merge_candidate`, `fix_needed`, `needs_human`, `close_duplicate`, `close_superseded`, or `close_fixed_by_candidate`
- `classification`: one of `canonical`, `duplicate`, `related`, `superseded`, `independent`, `fixed_by_candidate`, or `needs_human`
- `target_kind`: `issue` or `pull_request`
- `target_updated_at`: the live GitHub `updatedAt`/`updated_at` value you fetched for the target
- `canonical`, `duplicate_of`, or `candidate_fix` when the close depends on another issue/PR
- `comment`: the exact close comment you recommend, preserving contributor credit and linking the canonical or candidate fix
- `idempotency_key`: stable key such as `projectclownfish:<cluster_id>:<target>:<action>:<canonical-or-fix>`
- `evidence`: short concrete evidence strings
- `gh issue comment`
- `gh issue close`
- `gh issue edit --add-label`
- `gh pr comment`
- `gh pr close`
- `gh pr merge`
The applicator only auto-closes:
Never force-push, rewrite contributor branches, or bypass failing checks unless the job explicitly says so and the policy allows it.
- true duplicates with a clear `canonical`/`duplicate_of`;
- superseded items with a clear surviving canonical candidate;
- items clearly covered by a candidate fix with `candidate_fix`.
Return structured JSON only.
Everything else should be `planned` as non-mutating or escalated as `needs_human`.
Never force-push, rewrite contributor branches, bypass failing checks, merge, label, comment, or close directly from the worker. Return structured JSON only.

View File

@ -15,10 +15,13 @@ For each item, decide one action:
- keep canonical
- close duplicate
- close superseded
- close fixed by candidate
- keep related
- keep independent
- merge candidate
- fix needed
- needs human
Use the same action fields as execute mode when possible: `classification`, `target_kind`, `target_updated_at`, `canonical`, `duplicate_of`, `candidate_fix`, `evidence`, and a stable `idempotency_key`. In plan mode these are recommendations only.
Return structured JSON only.

View File

@ -23,7 +23,7 @@ Before action:
Execution guard:
- In `plan` mode, do not mutate GitHub.
- In `execute` mode, mutate only if the job allows the action and the evidence is clear.
- In `execute` mode, do not mutate GitHub directly; emit structured actions for the applicator.
- If any safety condition is not met, return `needs_human`.
Final answer must match `schemas/codex-result.schema.json`.

View File

@ -31,7 +31,21 @@
"type": "string"
},
"action": {
"type": "string"
"type": "string",
"enum": [
"keep_canonical",
"keep_related",
"keep_independent",
"merge_candidate",
"fix_needed",
"needs_human",
"close",
"close_duplicate",
"close_superseded",
"close_fixed_by_candidate",
"label",
"comment"
]
},
"status": {
"enum": ["planned", "executed", "skipped", "blocked", "failed"]
@ -39,6 +53,44 @@
"idempotency_key": {
"type": "string"
},
"classification": {
"enum": [
"canonical",
"duplicate",
"related",
"superseded",
"independent",
"fixed_by_candidate",
"needs_human"
]
},
"target_kind": {
"enum": ["issue", "pull_request"]
},
"target_updated_at": {
"type": "string"
},
"canonical": {
"type": "string",
"pattern": "^#?[0-9]+$"
},
"duplicate_of": {
"type": "string",
"pattern": "^#?[0-9]+$"
},
"candidate_fix": {
"type": "string",
"pattern": "^#?[0-9]+$"
},
"comment": {
"type": "string"
},
"evidence": {
"type": "array",
"items": {
"type": ["string", "object"]
}
},
"reason": {
"type": "string"
}

379
scripts/apply-result.mjs Normal file
View File

@ -0,0 +1,379 @@
#!/usr/bin/env node
import fs from "node:fs";
import path from "node:path";
import { execFileSync } from "node:child_process";
import { createHash } from "node:crypto";
import { assertAllowedOwner, parseArgs, parseJob, repoRoot, validateJob } from "./lib.mjs";
const MAINTAINER_AUTHOR_ASSOCIATIONS = new Set(["OWNER", "MEMBER", "COLLABORATOR"]);
const CLOSE_ACTIONS = new Set([
"close",
"close_duplicate",
"close_superseded",
"close_fixed_by_candidate",
]);
const CLOSE_CLASSIFICATIONS = new Set(["duplicate", "superseded", "fixed_by_candidate"]);
const args = parseArgs(process.argv.slice(2));
const jobPath = args._[0];
const resultPathArg = args._[1];
const latest = Boolean(args.latest);
const dryRun = Boolean(args["dry-run"] || process.env.CLOWNFISH_APPLY_DRY_RUN === "1");
const allowMissingUpdatedAt = Boolean(args["allow-missing-updated-at"]);
const reportPathArg = args["report"];
if (!jobPath) {
console.error("usage: node scripts/apply-result.mjs <job.md> [result.json] [--latest] [--dry-run]");
process.exit(2);
}
if (!resultPathArg && !latest) {
console.error("result path is required unless --latest is set");
process.exit(2);
}
const job = parseJob(jobPath);
const errors = validateJob(job);
if (errors.length > 0) {
console.error(errors.join("\n"));
process.exit(1);
}
assertAllowedOwner(job.frontmatter.repo, process.env.CLOWNFISH_ALLOWED_OWNER);
if (job.frontmatter.mode !== "execute") {
throw new Error("refusing apply: job frontmatter mode is not execute");
}
if (process.env.CLOWNFISH_ALLOW_EXECUTE !== "1") {
throw new Error("refusing apply: CLOWNFISH_ALLOW_EXECUTE must be 1");
}
if (!job.frontmatter.allowed_actions.includes("close")) {
throw new Error("refusing apply: job does not allow close");
}
if (!job.frontmatter.allowed_actions.includes("comment")) {
throw new Error("refusing apply: job does not allow comment");
}
const resultPath = resultPathArg ? path.resolve(resultPathArg) : findLatestResultPath();
const result = JSON.parse(fs.readFileSync(resultPath, "utf8"));
if (result.repo !== job.frontmatter.repo) {
throw new Error(`result repo ${result.repo} does not match job repo ${job.frontmatter.repo}`);
}
if (result.cluster_id !== job.frontmatter.cluster_id) {
throw new Error(
`result cluster ${result.cluster_id} does not match job cluster ${job.frontmatter.cluster_id}`,
);
}
if (result.mode !== "execute") {
throw new Error(`refusing apply: result mode is ${result.mode}`);
}
const report = {
repo: result.repo,
cluster_id: result.cluster_id,
dry_run: dryRun,
result_path: path.relative(repoRoot(), resultPath),
applied_at: new Date().toISOString(),
actions: [],
};
for (const action of result.actions ?? []) {
report.actions.push(applyAction({ job, result, action, dryRun, allowMissingUpdatedAt }));
}
const reportPath =
typeof reportPathArg === "string"
? path.resolve(reportPathArg)
: path.join(path.dirname(resultPath), "apply-report.json");
fs.writeFileSync(reportPath, `${JSON.stringify(report, null, 2)}\n`);
console.log(JSON.stringify(report, null, 2));
function findLatestResultPath() {
const runsRoot = path.join(repoRoot(), ".projectclownfish", "runs");
if (!fs.existsSync(runsRoot)) {
throw new Error("no run directory exists");
}
const candidates = [];
for (const runName of fs.readdirSync(runsRoot)) {
const candidate = path.join(runsRoot, runName, "result.json");
if (!fs.existsSync(candidate)) continue;
candidates.push({ path: candidate, mtimeMs: fs.statSync(candidate).mtimeMs });
}
candidates.sort((left, right) => right.mtimeMs - left.mtimeMs);
if (!candidates[0]) throw new Error("no result.json files found");
return candidates[0].path;
}
function applyAction({ job, result, action, dryRun, allowMissingUpdatedAt }) {
const target = normalizeIssueRef(action.target);
const actionName = String(action.action ?? "");
const classification = normalizeClassification(action);
const canonical = normalizeIssueRef(action.canonical ?? action.duplicate_of);
const candidateFix = normalizeIssueRef(action.candidate_fix ?? action.fixed_by ?? action.fix_candidate);
const idempotencyKey =
typeof action.idempotency_key === "string" && action.idempotency_key.trim()
? action.idempotency_key.trim()
: defaultIdempotencyKey(result.cluster_id, target, actionName, classification);
const base = {
target: `#${target}`,
action: actionName,
classification,
canonical: canonical ? `#${canonical}` : undefined,
candidate_fix: candidateFix ? `#${candidateFix}` : undefined,
idempotency_key: idempotencyKey,
};
if (!target) return { ...base, status: "failed", reason: "target must look like #123" };
if (!CLOSE_ACTIONS.has(actionName)) {
return { ...base, status: "skipped", reason: "action is not an auto-closure action" };
}
if (!CLOSE_CLASSIFICATIONS.has(classification)) {
return {
...base,
status: "blocked",
reason: "auto-closure requires duplicate, superseded, or fixed_by_candidate classification",
};
}
if (!job.frontmatter.candidates.map(normalizeIssueRef).includes(target)) {
return { ...base, status: "blocked", reason: "target is not listed in job candidates" };
}
if ((classification === "duplicate" || classification === "superseded") && !canonical) {
return { ...base, status: "blocked", reason: "closure requires canonical or duplicate_of" };
}
if (classification === "fixed_by_candidate" && !candidateFix) {
return { ...base, status: "blocked", reason: "closure requires candidate_fix" };
}
if (canonical === target || candidateFix === target) {
return { ...base, status: "blocked", reason: "target cannot close against itself" };
}
const live = fetchIssue(result.repo, target);
const kind = live.pull_request ? "pull_request" : "issue";
const authorAssociation = normalizeAuthorAssociation(live.author_association);
if (MAINTAINER_AUTHOR_ASSOCIATIONS.has(authorAssociation)) {
return {
...base,
status: "blocked",
reason: `target author association is ${authorAssociation}`,
live_state: live.state,
};
}
const expectedUpdatedAt = action.target_updated_at ?? action.live_updated_at;
if (!expectedUpdatedAt && !allowMissingUpdatedAt) {
return {
...base,
status: "blocked",
reason: "missing target_updated_at; rerun the worker against live GitHub state",
live_state: live.state,
live_updated_at: live.updated_at,
};
}
if (expectedUpdatedAt && expectedUpdatedAt !== live.updated_at) {
return {
...base,
status: "blocked",
reason: "target changed since worker review",
expected_updated_at: expectedUpdatedAt,
live_updated_at: live.updated_at,
live_state: live.state,
};
}
const comment = renderCloseComment({ action, classification, result, target, live });
const marker = idempotencyMarker(result.cluster_id, target, idempotencyKey);
const body = comment.includes(marker) ? comment : `${comment.trim()}\n\n${marker}`;
const existingComment = findExistingComment(result.repo, target, marker, body);
if (live.state !== "open") {
return {
...base,
status: existingComment ? "executed" : "skipped",
reason: existingComment ? "already closed with matching projectclownfish comment" : "already closed",
live_state: live.state,
};
}
if (dryRun) {
return {
...base,
status: "planned",
reason: "dry run",
live_state: live.state,
live_updated_at: live.updated_at,
comment,
};
}
if (!existingComment) {
postIssueComment(result.repo, target, body);
}
closeIssueOrPullRequest(result.repo, target, kind, classification);
return {
...base,
status: "executed",
reason: closeReasonText(classification),
live_state: "closed",
live_updated_at: live.updated_at,
};
}
function normalizeIssueRef(value) {
const match = String(value ?? "").match(/^#?(\d+)$/);
return match ? Number(match[1]) : 0;
}
function normalizeClassification(action) {
const raw = String(action.classification ?? action.close_reason ?? action.reason ?? "").toLowerCase();
if (raw.includes("fixed") || raw.includes("candidate")) return "fixed_by_candidate";
if (raw.includes("superseded") || raw.includes("supersede")) return "superseded";
if (raw.includes("duplicate") || raw.includes("dupe")) return "duplicate";
if (action.action === "close_fixed_by_candidate") return "fixed_by_candidate";
if (action.action === "close_superseded") return "superseded";
if (action.action === "close_duplicate") return "duplicate";
return raw;
}
function defaultIdempotencyKey(clusterId, target, actionName, classification) {
return sha256(`${clusterId}:${target}:${actionName}:${classification}`).slice(0, 24);
}
function idempotencyMarker(clusterId, target, key) {
return `<!-- projectclownfish:close:${clusterId}:#${target}:${key} -->`;
}
function renderCloseComment({ action, classification, result, target, live }) {
if (typeof action.comment === "string" && action.comment.trim()) return action.comment;
const canonical = normalizeIssueRef(action.canonical ?? action.duplicate_of);
const candidateFix = normalizeIssueRef(action.candidate_fix ?? action.fixed_by ?? action.fix_candidate);
const title = typeof live.title === "string" ? live.title : `#${target}`;
const evidence = Array.isArray(action.evidence) ? action.evidence : [];
const evidenceLines = evidence
.slice(0, 5)
.map((item) => `- ${typeof item === "string" ? item : (item.detail ?? JSON.stringify(item))}`);
const reason = action.reason ? String(action.reason).trim() : closeReasonText(classification);
const lines = [`Thanks for this. Projectclownfish reviewed this cluster and is closing #${target}.`];
lines.push("");
if (classification === "duplicate" && canonical) {
lines.push(
`This appears to duplicate #${canonical}. I'm keeping #${canonical} as the canonical thread so fixes, validation, and follow-up stay in one place.`,
);
} else if (classification === "superseded" && canonical) {
lines.push(
`This is superseded by #${canonical}. I'm keeping that thread as the canonical path so the useful context and contributor credit stay visible.`,
);
} else if (classification === "fixed_by_candidate" && candidateFix) {
lines.push(
`This is covered by candidate fix #${candidateFix}. I'm closing this thread so validation and follow-up stay attached to that fix path.`,
);
} else {
lines.push(reason);
}
lines.push("");
lines.push(`Cluster: \`${result.cluster_id}\``);
lines.push(`Reviewed item: #${target} - ${title}`);
if (evidenceLines.length) lines.push("", "Evidence:", ...evidenceLines);
lines.push(
"",
"If this has a different reproduction path or still reproduces after the canonical fix lands, reply and we can reopen or split it back out.",
);
return lines.join("\n");
}
function closeReasonText(classification) {
switch (classification) {
case "duplicate":
return "duplicate of the canonical thread";
case "superseded":
return "superseded by the canonical candidate";
case "fixed_by_candidate":
return "covered by the candidate fix";
default:
return "closed by projectclownfish";
}
}
function fetchIssue(repo, number) {
return ghJson(["api", `repos/${repo}/issues/${number}`]);
}
function findExistingComment(repo, number, marker, body) {
const comments = ghPaged(`repos/${repo}/issues/${number}/comments`);
return comments.find((comment) => comment.body?.includes(marker) || comment.body === body);
}
function postIssueComment(repo, number, body) {
const payloadPath = writePayload(`comment-${number}`, { body });
ghWithRetry(["api", `repos/${repo}/issues/${number}/comments`, "--method", "POST", "--input", payloadPath]);
}
function closeIssueOrPullRequest(repo, number, kind, classification) {
if (kind === "pull_request") {
ghWithRetry(["pr", "close", String(number), "--repo", repo]);
return;
}
const stateReason = classification === "fixed_by_candidate" ? "completed" : "not_planned";
const payloadPath = writePayload(`close-${number}`, { state: "closed", state_reason: stateReason });
ghWithRetry(["api", `repos/${repo}/issues/${number}`, "--method", "PATCH", "--input", payloadPath]);
}
function writePayload(name, value) {
const dir = path.join(repoRoot(), ".projectclownfish", "payloads");
fs.mkdirSync(dir, { recursive: true });
const file = path.join(dir, `${name}-${Date.now()}.json`);
fs.writeFileSync(file, JSON.stringify(value), "utf8");
return file;
}
function ghJson(ghArgs) {
const text = ghWithRetry(ghArgs);
return JSON.parse(text || "null");
}
function ghPaged(apiPath) {
const pages = ghJson(["api", apiPath, "--paginate", "--slurp"]);
if (!Array.isArray(pages)) return [];
return pages.flatMap((page) => (Array.isArray(page) ? page : []));
}
function ghWithRetry(ghArgs, attempts = 6) {
let lastError;
for (let attempt = 0; attempt < attempts; attempt += 1) {
try {
return execFileSync("gh", ghArgs, {
cwd: repoRoot(),
encoding: "utf8",
env: process.env,
maxBuffer: 64 * 1024 * 1024,
stdio: ["ignore", "pipe", "pipe"],
}).trim();
} catch (error) {
lastError = error;
if (!shouldRetryGh(error) || attempt === attempts - 1) throw error;
sleepMs(Math.min(120_000, 10_000 * 2 ** attempt));
}
}
throw lastError;
}
function shouldRetryGh(error) {
const stderr = String(error?.stderr ?? "");
const message = `${error instanceof Error ? error.message : String(error)}\n${stderr}`;
return (
message.includes("was submitted too quickly") ||
message.includes("secondary rate") ||
message.includes("API rate limit exceeded")
);
}
function sleepMs(milliseconds) {
Atomics.wait(new Int32Array(new SharedArrayBuffer(4)), 0, 0, milliseconds);
}
function normalizeAuthorAssociation(value) {
return typeof value === "string" && value.trim() ? value.trim().toUpperCase() : "NONE";
}
function sha256(text) {
return createHash("sha256").update(text).digest("hex");
}