create-test

• SKILL.mdcore · 13,252 bytes

# Create a test

## Goal

Every rule the owner cares about has a cheap, deterministic check that runs on every verification round, so the agent cannot drift from it unnoticed. A test costs nothing per run, never tires and never changes its mind, which is why tests carry the enforcement and memory carries only the context. One correction should produce as many narrow tests as the feedback supports, because each test catches one way of getting it wrong and the model finds new ways. A test is proved before it is trusted: it fails on the artifact the owner rejected and passes on the version the owner accepted, or it has not shown it can tell the difference. Tests are written against the kind of artifact, not only the one file that was corrected, so the rule holds for every future artifact of that kind. A property that no code can check is handed to a verifier agent rather than faked with a test that calls a model. The measure of this skill is that the owner never has to give the same correction twice.

## Objective

Restate the feedback as observable properties of an artifact, one per line. Walk the catalog of deterministic checks and list every check that applies to each property, with the ones rejected and why. Implement each kept check as one pytest function in a file named by the FB number, using only the standard library, reading artifacts through the shared helpers. Prove each test against the rejected and accepted artifacts through fixtures when they exist. Run the suite the way the current phase requires and leave every existing test untouched. Report the test paths so the orchestrator can write them into the entry's `verified-by`, and name every property that needs a verifier instead.

## Tasks

1. Restate the feedback as one or more observable properties of an artifact: something a reader could point at in the file and say "there, that is the violation". "Captions must not shout" becomes "no caption text is in all capitals"; "the intro is too long" becomes "the first section is under 120 words". A property that cannot be pointed at is not yet a property; refine it, or hand it to a verifier in task 6.
2. Walk the catalog below for each property and list every check that applies. Deterministic checks are cheap, so prefer several narrow tests over one broad one; three tests that each catch one failure mode beat one that tries to catch all. Record the catalog items rejected and why, because the report is how the orchestrator knows the search was done.