Policies
Protect Test Integrity
Protect Test Integrity: what it does, its label on each agent, what it misses, and examples.
Catalog policy protect-test-integrity, version 0.0.3, labelled blocks on Claude Code.
What it does
Blocks weakening tests to turn them green: a deleted test file (commit only), a net loss of assertions, or an added vacuous assertion (assert True, expect(true).toBe(true)). Added skips are block-test-skips. Runs: commit (incl. an agent's commit: CHOCK_AGENT_COMMIT, CLAUDECODE=1, AI_AGENT or agent_commit_env), agent write (vs disk), turn's end (vs HEAD). Waiver: 'chock: allow test-integrity' on an added line. Person's commit: honoured. Agent: only if already in HEAD; it asks a person.
On each agent
| Agent | Label | What it does there |
|---|---|---|
| Claude Code | Blocks | Blocks on an agent's file writes and at turn end |
| Cursor | Blocks | Blocks on an agent's file writes and at turn end |
| Codex | Blocks | Blocks on an agent's file writes and at turn end. If the hook errors, the action goes through (not witnessed; the vendor's row is only partly live-run) |
| GitHub Copilot | Blocks | Blocks at turn end only. It does not check a file write when it happens. If the hook errors, the action goes through (not witnessed; the vendor's row is only partly live-run) |
| Devin | Blocks | Blocks at turn end only. It does not check a file write when it happens. If the hook errors, the action goes through (vendor docs only, not witnessed) |
| Grok | Not graded | Not graded yet for Grok |
| goose | Not graded | Not graded yet for goose |
| Antigravity | Not graded | Not graded yet for Antigravity |
| OpenHands | Not graded | Not graded yet for OpenHands |
| Kimi Code | Not graded | Not graded yet for Kimi Code |
| Factory Droid | Not graded | Not graded yet for Factory Droid |
What it misses
This policy does not state what it misses.
Examples from its eval suite
Refused: A failing test is made green by cutting two of its three assertions.
Allowed: A test is refactored: assertions are reworded and moved into a helper, and none is lost.
More
Built from engine 5a4b6287a46e and catalog 1e60d211c1bb.