openclaw/Kova
openclaw/Kova
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

refactor: simplify profile gate coverage

Browse Source
This commit is contained in:
Shakker 2026-05-04 23:49:17 +01:00
parent dc011f4305
commit 3d0f45b0c0
No known key found for this signature in database
12 changed files with 187 additions and 440 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
docs/CONTRACT_REGISTRY.md
View File

@ -31,8 +31,8 @@ Then:
2. Add missing metric ids to `metrics/known.json`.
3. Add state fixture hooks or traits in `states/*.json` only when the
requirement needs a new user condition.
4. Add profile coverage in `profiles/*.json` only when the surface should be
part of that profile.
4. Add profile requirement coverage in `profiles/*.json` only when the surface
requirement should be part of that profile.
5. Run `node bin/kova.mjs plan --json`.
6. Run a dry-run for the scenario.
7. Add self-check coverage if the surface introduces new evidence parsing.
@ -68,8 +68,8 @@ lists.
Then:
1. Pair the state with compatible scenarios or profile entries.
2. Add profile trait/state coverage only when it is required for release
confidence.
2. Add or update surface requirements when this state becomes required proof
for a surface.
3. Run `node bin/kova.mjs plan --json`.
4. Dry-run at least one scenario/state pair.
5. Execute a disposable scenario when the state lifecycle mutates files,
@ -89,6 +89,8 @@ Self-check and plan validation must fail for:
- scenario/state pairs that violate requirement state contracts or hard
incompatibility blocks
- profile entries that require unknown surfaces or states
- profile gate coverage that uses derived policy fields instead of
`requirements` or `platforms`
If a new surface or state needs exceptions to these rules, the contract is too
loose. Tighten the JSON or add a focused validator.

36
docs/REPORT_SCHEMA.md
View File

@ -339,25 +339,25 @@ definitions, state fixture definitions, surface definitions, process-role
definitions, profile summaries, platform metadata, and supports filtering with
`--scenario`, `--state`, and `--profile`.
Every scenario must declare a `surface`. Registry validation fails before plan,
run, or matrix output if a scenario references an unknown surface, a surface
references an unknown process role, or a profile references an unknown
scenario/state/surface.
Every scenario must declare a `surface` and the requirement ids it proves.
Registry validation fails before plan, run, or matrix output if a scenario
references an unknown surface or requirement, a surface references an unknown
process role, or a profile references an unknown scenario/state/requirement.
Every state must declare traits, compatible surfaces, incompatible surfaces,
risk area, owner area, setup evidence, and cleanup guarantees. Registry
validation rejects unknown state traits, unknown surface references, and profile
entries that pair a scenario with a state that is not allowed for the scenario's
surface.
Every state must declare traits, risk area, owner area, setup evidence, and
cleanup guarantees. Positive surface compatibility is owned by surface
requirements. Registry validation rejects unknown state traits, unknown hard
incompatibility references, and profile entries that pair a scenario with a
state that does not satisfy the scenario's proved requirements.
Plan JSON includes `coverage`:
- `surfaces`: each surface with scenario count and mapped scenarios
- `scenarioSurfaceMap`: direct scenario-to-surface mappings
- `surfacesWithoutScenarios`: declared surfaces with no scenario yet
- `profiles`: per-profile selected surfaces, scenarios, states, required
coverage, coverage gaps, state trait coverage, state/surface pairs, and
trait/surface coverage
- `profiles`: per-profile selected surfaces, scenarios, states, requirement
coverage, derived required coverage, coverage gaps, state trait coverage,
state/surface pairs, and trait/surface coverage
`kova matrix plan --json` also includes `resolvedCoverage`. This is the pre-run
contract resolver for the selected profile, target, filters, scenarios, and
@ -455,6 +455,9 @@ the existing matrix runner and adds:
"blocking": ["darwin-arm64"],
"warning": ["linux-x64", "linux-arm64", "wsl2"]
},
"requirements": {
"blocking": ["release-runtime-startup:baseline"]
},
"states": {
"blocking": ["fresh"]
},
@ -494,10 +497,11 @@ Filtered gate slices are partial. They can produce `DO_NOT_SHIP` when a selected
blocking scenario fails, but they cannot produce `SHIP` because required gate
coverage is missing. A passing filtered slice remains `PARTIAL`.
Release profiles may define explicit platform/surface/scenario/state/trait and
state-surface coverage. Profiles may also define requirement coverage using
`surface:requirement` ids. Missing blocking coverage prevents `SHIP`; missing
warning coverage creates warning cards. Platform coverage keys include
Release profiles define explicit platform coverage and requirement coverage
using `surface:requirement` ids. Surface, scenario, state, trait, and
state-surface coverage views are derived from resolved obligations for report
compatibility. Missing blocking requirement/platform coverage prevents `SHIP`;
missing warning coverage creates warning cards. Platform coverage keys include
`darwin-arm64`, `linux-x64`, `linux-arm64`, and `wsl2` where detectable.
Gate cards are concise fixer records. They include severity, scenario/state,

44
profiles/diagnostic.json
View File

@ -125,50 +125,6 @@
"gate": {
"id": "openclaw-diagnostic",
"coverage": {
"surfaces": {
"blocking": [
"release-runtime-startup",
"gateway-performance",
"bundled-runtime-deps",
"agent-cli-local-turn",
"agent-gateway-rpc-turn",
"dashboard-session-send-turn",
"tui-message-turn",
"openai-compatible-turn"
]
},
"states": {
"blocking": [
"fresh",
"missing-plugin-index",
"many-bundled-plugins",
"mock-openai-provider"
]
},
"stateSurfaces": {
"blocking": [
"release-runtime-startup:fresh",
"gateway-performance:many-bundled-plugins",
"bundled-runtime-deps:missing-plugin-index",
"agent-cli-local-turn:mock-openai-provider",
"agent-gateway-rpc-turn:mock-openai-provider",
"dashboard-session-send-turn:mock-openai-provider",
"tui-message-turn:mock-openai-provider",
"openai-compatible-turn:mock-openai-provider"
]
},
"scenarios": {
"blocking": [
"release-runtime-startup",
"gateway-performance",
"bundled-runtime-deps",
"agent-cold-warm-message",
"agent-gateway-rpc-turn",
"dashboard-session-send-turn",
"tui-message-turn",
"openai-compatible-turn"
]
},
"requirements": {
"blocking": [
"release-runtime-startup:baseline",

26
profiles/official-plugins.json
View File

@ -5,32 +5,6 @@
"gate": {
"id": "openclaw-official-plugins",
"coverage": {
"states": {
"blocking": [
"official-plugins"
]
},
"traits": {
"blocking": [
"official-plugin",
"plugin-pressure"
]
},
"stateSurfaces": {
"blocking": [
"official-plugin-install:official-plugins"
]
},
"surfaces": {
"blocking": [
"official-plugin-install"
]
},
"scenarios": {
"blocking": [
"official-plugin-install"
]
},
"requirements": {
"blocking": [
"official-plugin-install:baseline"

122
profiles/release.json
View File

@ -226,128 +226,6 @@
"wsl2"
]
},
"states": {
"blocking": [
"fresh",
"onboarded-user",
"old-release-user",
"old-release-2026-4-20-user",
"old-release-2026-4-24-user",
"plugin-index",
"many-bundled-plugins",
"official-plugins"
]
},
"traits": {
"blocking": [
"fresh-user",
"existing-user",
"old-release",
"plugin-pressure",
"provider-pressure"
],
"warning": [
"filesystem-pressure",
"memory-pressure",
"failure-state"
]
},
"stateSurfaces": {
"blocking": [
"release-runtime-startup:fresh",
"fresh-install:fresh",
"fresh-install:onboarded-user",
"upgrade-existing-user:old-release-user",
"upgrade-existing-user:old-release-2026-4-20-user",
"upgrade-existing-user:old-release-2026-4-24-user",
"bundled-runtime-deps:missing-plugin-index",
"plugin-lifecycle:plugin-index",
"plugin-lifecycle:external-plugin",
"official-plugin-install:official-plugins",
"gateway-performance:many-bundled-plugins",
"agent-cli-local-turn:mock-openai-provider",
"agent-gateway-rpc-turn:mock-openai-provider",
"dashboard-session-send-turn:mock-openai-provider",
"tui-message-turn:mock-openai-provider",
"openai-compatible-turn:mock-openai-provider",
"provider-models:model-auth-missing",
"dashboard:fresh",
"tui:fresh"
],
"warning": [
"failure-containment:broken-plugin-deps",
"soak:large-workspace",
"workspace-scan:large-workspace",
"mcp-runtime:fresh",
"browser-automation:fresh",
"media-understanding:fresh",
"network-offline:fresh",
"cross-platform-smoke:slow-filesystem"
]
},
"surfaces": {
"blocking": [
"release-runtime-startup",
"fresh-install",
"upgrade-existing-user",
"bundled-runtime-deps",
"plugin-lifecycle",
"plugin-external-install",
"official-plugin-install",
"plugin-remove",
"plugin-update",
"plugin-bad-manifest",
"plugin-missing-runtime-deps",
"bundled-plugin-startup",
"provider-models",
"agent-cli-local-turn",
"agent-gateway-rpc-turn",
"dashboard-session-send-turn",
"tui-message-turn",
"openai-compatible-turn",
"dashboard",
"tui",
"gateway-performance"
],
"warning": [
"failure-containment",
"soak",
"workspace-scan",
"mcp-runtime",
"browser-automation",
"media-understanding",
"network-offline",
"cross-platform-smoke"
]
},
"scenarios": {
"blocking": [
"release-runtime-startup",
"fresh-install",
"upgrade-existing-user",
"upgrade-from-2026-4-20",
"upgrade-from-2026-4-24",
"bundled-runtime-deps",
"plugin-lifecycle",
"official-plugin-install",
"provider-models",
"agent-cold-warm-message",
"agent-gateway-rpc-turn",
"dashboard-session-send-turn",
"tui-message-turn",
"openai-compatible-turn",
"dashboard-readiness",
"tui-responsiveness",
"gateway-performance"
],
"warning": [
"workspace-scan-pressure",
"mcp-runtime-start-stop",
"browser-automation-smoke",
"media-understanding-timeout",
"agent-network-offline"
]
},
"requirements": {
"blocking": [
"release-runtime-startup:baseline",

115
src/matrix/coverage-policy.mjs Normal file
View File

@ -0,0 +1,115 @@
export const coveragePolicyKeys = [
"surfaces",
"platforms",
"states",
"traits",
"scenarios",
"stateSurfaces",
"requirements"
];
export const primaryCoveragePolicyKeys = ["platforms", "requirements"];
export const derivedCoveragePolicyKeys = ["surfaces", "scenarios", "states", "traits", "stateSurfaces"];
export function normalizeCoveragePolicy(coverage) {
const input = coverage && typeof coverage === "object" ? coverage : {};
return Object.fromEntries(coveragePolicyKeys.map((key) => [key, normalizeCoverageSet(input[key])]));
}
export function deriveCoveragePolicy(coverage, obligations = []) {
const policy = normalizeCoveragePolicy(coverage);
const requirementSeverity = requirementSeverityByKey(policy.requirements);
for (const obligation of obligations ?? []) {
const key = requirementKey(obligation.surface, obligation.requirement);
const severity = requirementSeverity.get(key);
if (!severity) {
continue;
}
add(policy.surfaces, severity, obligation.surface);
add(policy.scenarios, severity, obligation.scenario);
add(policy.states, severity, obligation.state);
add(policy.stateSurfaces, severity, obligation.surface && obligation.state ? `${obligation.surface}:${obligation.state}` : null);
for (const trait of obligation.stateTraits ?? []) {
add(policy.traits, severity, trait);
}
}
return sortCoveragePolicy(policy);
}
export function buildEntryCoverageObligations(profile, { scenarios, states }) {
const scenarioById = new Map((scenarios ?? []).map((scenario) => [scenario.id, scenario]));
const stateById = new Map((states ?? []).map((state) => [state.id, state]));
const obligations = [];
for (const entry of profile?.entries ?? []) {
const scenario = scenarioById.get(entry.scenario);
const state = stateById.get(entry.state);
if (!scenario) {
continue;
}
for (const requirement of scenario.proves ?? []) {
obligations.push({
surface: scenario.surface,
requirement,
scenario: scenario.id,
state: entry.state,
stateTraits: state?.traits ?? [],
status: "planned"
});
}
}
return obligations;
}
export function coverageIdsFromSet(set) {
return [...new Set([...(set?.blocking ?? []), ...(set?.warning ?? [])])].sort();
}
function normalizeCoverageSet(value) {
const input = value && typeof value === "object" ? value : {};
return {
blocking: normalizeStringList(input.blocking),
warning: normalizeStringList(input.warning)
};
}
function normalizeStringList(value) {
return Array.isArray(value) ? value.filter((item) => typeof item === "string" && item.length > 0) : [];
}
function requirementSeverityByKey(requirements) {
const severities = new Map();
for (const value of requirements.warning) {
severities.set(value, "warning");
}
for (const value of requirements.blocking) {
severities.set(value, "blocking");
}
return severities;
}
function requirementKey(surface, requirement) {
return `${surface}:${requirement}`;
}
function add(set, severity, value) {
if (typeof value !== "string" || value.length === 0) {
return;
}
if (!set[severity].includes(value)) {
set[severity].push(value);
}
}
function sortCoveragePolicy(policy) {
return Object.fromEntries(Object.entries(policy).map(([key, value]) => [
key,
{
blocking: derivedCoveragePolicyKeys.includes(key) ? [...value.blocking].sort() : value.blocking,
warning: derivedCoveragePolicyKeys.includes(key) ? [...value.warning].sort() : value.warning
}
]));
}

31
src/matrix/coverage.mjs
View File

@ -1,4 +1,9 @@
import { platformCoverageKeys } from "../platform.mjs";
import {
buildEntryCoverageObligations,
coverageIdsFromSet,
deriveCoveragePolicy
} from "./coverage-policy.mjs";
export function buildCoverage({ surfaces, scenarios, states, profiles, platform }) {
const scenarioSurfaceMap = scenarios
@ -52,13 +57,17 @@ function profileCoverage(profile, { scenarios, states, platform }) {
}
}
const requiredSurfaces = coverageIds(profile, "surfaces");
const requiredScenarios = coverageIds(profile, "scenarios");
const requiredStates = coverageIds(profile, "states");
const requiredTraits = coverageIds(profile, "traits");
const requiredStateSurfaces = coverageIds(profile, "stateSurfaces");
const requiredRequirements = coverageIds(profile, "requirements");
const requiredPlatforms = coverageIds(profile, "platforms");
const derivedPolicy = deriveCoveragePolicy(
profile.gate?.coverage,
buildEntryCoverageObligations(profile, { scenarios, states })
);
const requiredSurfaces = coverageIdsFromSet(derivedPolicy.surfaces);
const requiredScenarios = coverageIdsFromSet(derivedPolicy.scenarios);
const requiredStates = coverageIdsFromSet(derivedPolicy.states);
const requiredTraits = coverageIdsFromSet(derivedPolicy.traits);
const requiredStateSurfaces = coverageIdsFromSet(derivedPolicy.stateSurfaces);
const requiredRequirements = coverageIdsFromSet(derivedPolicy.requirements);
const requiredPlatforms = coverageIdsFromSet(derivedPolicy.platforms);
const coveredTraits = coveredStateTraits(profile, states);
const currentPlatformKeys = platformCoverageKeys(platform);
@ -161,14 +170,6 @@ function traitSurfaceCoverage(profile, { scenarios, states }) {
.map(([trait, surfaces]) => [trait, [...surfaces].sort()]));
}
function coverageIds(profile, key) {
const coverage = profile.gate?.coverage?.[key];
if (!coverage) {
return [];
}
return [...new Set([...(coverage.blocking ?? []), ...(coverage.warning ?? [])])].sort();
}
function byScenario(left, right) {
return left.scenario.localeCompare(right.scenario);
}

79
src/matrix/gate.mjs
View File

@ -1,4 +1,5 @@
import { platformCoverageKeys } from "../platform.mjs";
import { deriveCoveragePolicy } from "./coverage-policy.mjs";
export function preflightGateRun({ entries, flags }) {
if (flags?.gate !== true || flags?.execute !== true) {
@ -22,7 +23,7 @@ function scenarioUsesSourceEnv(scenario) {
}
export function evaluateGate(report, profile, options = {}) {
const policy = normalizeGatePolicy(profile);
const policy = normalizeGatePolicy(profile, options);
const purpose = profile?.purpose ?? "release";
const records = report.records ?? [];
const cards = [];
@ -155,7 +156,7 @@ function isPartialGate(report) {
return (controls?.include?.length ?? 0) > 0 || (controls?.exclude?.length ?? 0) > 0;
}
function normalizeGatePolicy(profile) {
function normalizeGatePolicy(profile, options = {}) {
const gate = profile?.gate && typeof profile.gate === "object" ? profile.gate : {};
const entries = Array.isArray(profile?.entries) ? profile.entries : [];
const warning = normalizePolicyEntries(gate.warning ?? []);
@ -166,63 +167,19 @@ function normalizeGatePolicy(profile) {
id: typeof gate.id === "string" && gate.id ? gate.id : `${profile?.id ?? "matrix"}-gate`,
blocking,
warning,
coverage: normalizeCoveragePolicy(gate.coverage)
coverage: deriveCoveragePolicy(gate.coverage, options.resolvedCoverage?.obligations ?? [])
};
}
function normalizeCoveragePolicy(coverage) {
const input = coverage && typeof coverage === "object" ? coverage : {};
return {
surfaces: normalizeCoverageSet(input.surfaces),
platforms: normalizeCoverageSet(input.platforms),
states: normalizeCoverageSet(input.states),
traits: normalizeCoverageSet(input.traits),
scenarios: normalizeCoverageSet(input.scenarios),
stateSurfaces: normalizeCoverageSet(input.stateSurfaces),
requirements: normalizeCoverageSet(input.requirements)
};
}
function normalizeCoverageSet(value) {
const input = value && typeof value === "object" ? value : {};
return {
blocking: normalizeStringList(input.blocking),
warning: normalizeStringList(input.warning)
};
}
function normalizeStringList(value) {
return Array.isArray(value) ? value.filter((item) => typeof item === "string" && item.length > 0) : [];
}
function buildCoverageCards(report, policy, partial, options = {}) {
const cards = [];
const records = report.records ?? [];
const resolvedCoverage = options.resolvedCoverage ?? null;
const platformKeys = platformCoverageKeys(report.platform);
const scenarioKeys = new Set(records.map((record) => record.scenario).filter(Boolean));
const stateKeys = new Set(records.map((record) => record.state?.id).filter(Boolean));
const surfaceKeys = new Set(records.map((record) => record.surface ?? record.measurements?.surface).filter(Boolean));
const traitKeys = new Set(records.flatMap((record) => record.state?.traits ?? []).filter(Boolean));
const stateSurfaceKeys = new Set(records
.map((record) => {
const surface = record.surface ?? record.measurements?.surface;
const state = record.state?.id;
return surface && state ? `${surface}:${state}` : null;
})
.filter(Boolean));
const requirementKeys = new Set((resolvedCoverage?.obligations ?? [])
.filter((obligation) => obligation.status === "planned")
.map((obligation) => `${obligation.surface}:${obligation.requirement}`)
.filter((value) => !value.endsWith(":null")));
addCoverageCards(cards, {
kind: "surface",
expected: policy.coverage.surfaces,
observed: surfaceKeys,
partial,
statusText: `${surfaceKeys.size} surface(s) present`
});
addCoverageCards(cards, {
kind: "platform",
expected: policy.coverage.platforms,
@ -230,34 +187,6 @@ function buildCoverageCards(report, policy, partial, options = {}) {
partial,
statusText: report.platform ? `${report.platform.os}/${report.platform.arch}` : "unknown platform"
});
addCoverageCards(cards, {
kind: "scenario",
expected: policy.coverage.scenarios,
observed: scenarioKeys,
partial,
statusText: `${scenarioKeys.size} scenario(s) present`
});
addCoverageCards(cards, {
kind: "state",
expected: policy.coverage.states,
observed: stateKeys,
partial,
statusText: `${stateKeys.size} state(s) present`
});
addCoverageCards(cards, {
kind: "trait",
expected: policy.coverage.traits,
observed: traitKeys,
partial,
statusText: `${traitKeys.size} state trait(s) present`
});
addCoverageCards(cards, {
kind: "state-surface",
expected: policy.coverage.stateSurfaces,
observed: stateSurfaceKeys,
partial,
statusText: `${stateSurfaceKeys.size} state/surface pair(s) present`
});
addCoverageCards(cards, {
kind: "requirement",
expected: policy.coverage.requirements,

7
src/registries/profiles.mjs
View File

@ -153,7 +153,12 @@ function validateCoverage(coverage, prefix, errors) {
errors.push(`${prefix} must be an object`);
return;
}
for (const key of ["surfaces", "scenarios", "states", "traits", "stateSurfaces", "platforms", "requirements"]) {
for (const key of ["surfaces", "scenarios", "states", "traits", "stateSurfaces"]) {
if (coverage[key] !== undefined) {
errors.push(`${prefix}.${key} is derived from entries and requirements; use ${prefix}.requirements instead`);
}
}
for (const key of ["platforms", "requirements"]) {
const value = coverage[key];
if (value === undefined) {
continue;

17
src/registries/surface-requirements.mjs
View File

@ -61,20 +61,3 @@ export function scenarioSupportsState({ scenario, surface, state }) {
reason: `state '${state?.id ?? "unknown"}' does not satisfy scenario '${scenario.id}' requirement state ids or traits`
};
}
export function surfaceSupportsState({ surface, state }) {
const requirements = surface?.requirements ?? [];
if (requirements.length === 0) {
return {
ok: false,
reason: `surface '${surface?.id ?? "unknown"}' has no requirements`
};
}
if (requirements.some((requirement) => stateSatisfiesRequirement(state, requirement).ok)) {
return { ok: true, reason: null };
}
return {
ok: false,
reason: `state '${state?.id ?? "unknown"}' does not satisfy surface '${surface.id}' requirement state ids or traits`
};
}

65
src/registries/validate.mjs
View File

@ -5,7 +5,6 @@ import {
knownTargetKinds,
requirementsForScenario,
scenarioSupportsState,
surfaceSupportsState,
targetKindsForRequirements
} from "./surface-requirements.mjs";
@ -201,12 +200,7 @@ function validateProfileReferences(profile, refs, errors) {
}
}
validateCoverageRefs(profile, refs, errors, "surfaces", refs.surfaceIds);
validateCoverageRefs(profile, refs, errors, "scenarios", refs.scenarioIds);
validateCoverageRefs(profile, refs, errors, "states", refs.stateIds);
validateCoverageRefs(profile, refs, errors, "traits", refs.traitIds);
validatePlatformCoverageRefs(profile, errors);
validateStateSurfaceCoverageRefs(profile, refs, errors);
validateRequirementCoverageRefs(profile, refs, errors);
validateCalibrationRefs(profile, refs, errors);
}
@ -272,51 +266,6 @@ function validateScenarioStatePair({ profileId, location, scenarioId, stateId, r
}
}
function validateStateSurfaceCoverageRefs(profile, refs, errors) {
const coverage = profile.gate?.coverage?.stateSurfaces;
if (!coverage) {
return;
}
for (const level of ["blocking", "warning"]) {
for (const value of coverage[level] ?? []) {
const [surface, state, extra] = String(value).split(":");
if (!surface || !state || extra !== undefined) {
errors.push(`profile '${profile.id}' gate.coverage.stateSurfaces.${level} must use surface:state, got '${value}'`);
continue;
}
if (!refs.surfaceIds.has(surface)) {
errors.push(`profile '${profile.id}' gate.coverage.stateSurfaces.${level} references unknown surface '${surface}'`);
}
if (!refs.stateIds.has(state)) {
errors.push(`profile '${profile.id}' gate.coverage.stateSurfaces.${level} references unknown state '${state}'`);
}
validateStateSurfacePair({
profileId: profile.id,
location: `gate.coverage.stateSurfaces.${level}`,
surfaceId: surface,
stateId: state,
refs,
errors
});
}
}
}
function validateStateSurfacePair({ profileId, location, surfaceId, stateId, refs, errors }) {
const surface = refs.surfaceById.get(surfaceId);
const state = refs.stateById.get(stateId);
if (!surface || !state) {
return;
}
const stateResult = surfaceSupportsState({ surface, state });
if (!stateResult.ok) {
errors.push(`profile '${profileId}' ${location} requires '${surface.id}:${state.id}', but ${stateResult.reason}`);
}
if ((state.incompatibleSurfaces ?? []).includes(surface.id)) {
errors.push(`profile '${profileId}' ${location} requires explicitly incompatible state/surface pair '${surface.id}:${state.id}'`);
}
}
function validateRequirementCoverageRefs(profile, refs, errors) {
const coverage = profile.gate?.coverage?.requirements;
if (!coverage) {
@ -342,20 +291,6 @@ function validateRequirementCoverageRefs(profile, refs, errors) {
}
}
function validateCoverageRefs(profile, _refs, errors, key, allowedIds) {
const coverage = profile.gate?.coverage?.[key];
if (!coverage) {
return;
}
for (const level of ["blocking", "warning"]) {
for (const id of coverage[level] ?? []) {
if (!allowedIds.has(id)) {
errors.push(`profile '${profile.id}' gate.coverage.${key}.${level} references unknown ${key.slice(0, -1)} '${id}'`);
}
}
}
}
function idSet(items) {
return new Set(items.map((item) => item.id));
}

75
src/selfcheck.mjs
View File

@ -4464,6 +4464,26 @@ function stateRegistryValidationCheck() {
}
assertEqual(rejectedPurpose, true, "unknown profile purpose rejected");
let rejectedDerivedCoverage = false;
try {
validateProfileShape({
id: "profile",
title: "Bad Profile Coverage",
objective: "Invalid derived profile coverage.",
entries: [{ scenario: "scenario", state: "state" }],
gate: {
coverage: {
surfaces: {
blocking: ["surface"]
}
}
}
}, "bad-profile-coverage.json");
} catch (error) {
rejectedDerivedCoverage = /coverage\.surfaces is derived/.test(error.message);
}
assertEqual(rejectedDerivedCoverage, true, "derived profile coverage rejected");
let rejectedRequirement = false;
try {
validateRegistryReferences({
@ -4504,61 +4524,6 @@ function stateRegistryValidationCheck() {
}
assertEqual(rejectedRequirement, true, "invalid surface requirement and scenario proof rejected");
let rejectedCoveragePair = false;
try {
validateRegistryReferences({
scenarios: [{
id: "scenario",
surface: "known-surface",
states: [],
targetKinds: [],
processRoles: []
}],
states: [{
id: "state",
traits: ["fresh-user"],
incompatibleSurfaces: ["known-surface"]
}],
profiles: [{
id: "profile",
entries: [],
gate: {
coverage: {
stateSurfaces: {
blocking: ["known-surface:state"]
}
}
}
}],
surfaces: [
{
id: "known-surface",
processRoles: [],
requirements: [{
id: "baseline",
states: ["state"],
targetKinds: ["runtime"],
metrics: []
}]
},
{
id: "other-surface",
processRoles: [],
requirements: [{
id: "baseline",
states: ["state"],
targetKinds: ["runtime"],
metrics: []
}]
}
],
processRoles: []
});
} catch (error) {
rejectedCoveragePair = /explicitly incompatible state\/surface pair/.test(error.message);
}
assertEqual(rejectedCoveragePair, true, "invalid coverage state/surface pair rejected");
let rejectedMetric = false;
try {
validateRegistryReferences({