Coverage: measure, depth, clauses

For LLMs7 sections

“How much of the act is formalized?” has a computed answer, not an impression. One command measures each package, and a family of recorded checks keeps the numbers from going down.

Environment: law audit is available in the public release (S); the two reports below come from the checkout launcher ./law, run inside each package directory. Marks follow the topic legend.

Two real runs

Both reports below come from the checkout build, which ships the lint catalog; the public composition reports the form section as not_run instead (see Sixteen shape checks on every audit).

A vocabulary package has no rules, so its report is short. Package vocab.geo, audited from its own directory against the checkout’s compiled snapshot:

$ cd packs/vocab/geo && ../../../law audit --clir ../../../corpus/clir
law audit vocab.geo — formalization measure §33.1 (E-0249), profile static
CLIR lowered from sources (imports: ../../../corpus/clir); freshness: verified
levels not computed by this implementation: STRUCTURED, SEMANTIC

form (LawQL catalog lints on fresh snapshot, DECISION-0394): detectors 16, findings 0
dangling premises: 0 (unsupplied by package 0, enum variants in body 0)
one-way premises: 0 of 0 rules

All sixteen shape detectors ran and found nothing, and no rule has a premise that nothing supplies, so nothing in this report asks for a fix. There is no text-coverage line, because this vocabulary package quotes no act.

A source package with rules reports text coverage against its pinned bytes plus the lint findings. Package id.halal:

$ cd corpus/laws/id/halal && ../../../../law audit --clir ../../../../corpus/clir
law audit id.halal — formalization measure §33.1 (E-0249), profile static
CLIR lowered from sources (imports: ../../../../corpus/clir); freshness: verified
levels not computed by this implementation: STRUCTURED, SEMANTIC

— urn:id:clir:halal#UUJPH_ID_TEXT —
  measure undefined: unit convention is not supported (unit: unsupported)
  text coverage: 22305 of 24734 bytes (90.1 %), fragments 69

form (LawQL catalog lints on fresh snapshot, DECISION-0394): detectors 16, findings 12: accepted 1, debt 11
dangling premises: 4 (unsupplied by package 0, enum variants in body 4)
one-way premises: 5 of 124 rules

Read the second report line by line:

the share computation for this edition is undefined, because its unit convention is not supported; the tool reports the gap instead of inventing shares;
text coverage is still computed: 90.1% of the pinned bytes are quoted, across 69 fragments;
the form check found 12 findings across 16 detectors; one is accepted with a reason and eleven are carried as debt;
five of 124 rules have premises that flow only one way.

What it justifies: the remaining 10% of pinned bytes and the eleven debt findings are the work list for the next pass on this package.

The formalization measure

law audit computes the per-act formalization measure and executes nothing. Each article lands on one level: quoted only, anchored to a rule, or executable through one. The two levels above that — structured and semantic reading — are named by the schema but not computed by this tool release, and the command says so on every run (the levels not computed line above).

The measure is computed by two independent implementations. Their reports are compared byte for byte in the shared conformance suite, with shared unit conventions. So the numbers cannot quietly depend on which implementation produced them.

Minimum levels the check enforces

The full profile records a minimum value per act and refuses any drop below it; growth is allowed (full-profile CI check, G). The recorded minimums cover three measures:

formalization depth — the per-level shares per act; a share that falls below its recorded value fails the run;
label coverage — a per-language record over dictionary entries and norms, including norms that carry no source anchor, so unanchored norms cannot hide from the count;
clause coverage (opt-in per act) — normative fragments of clause kind need an authored deriving node anchored on exactly their fragment, configured through the act’s clause ledger.

A second family of checks covers the scenario side (G): the casus matrix, the scenario and goal manifests, scenario discovery, dataset pairs, the question catalog, the task-guide catalog, and verbalization totality. Each asks the same question in its own vocabulary: is every promised example present, discoverable, and rendered?

The registry check that stays out of the pipeline

A companion internal tool prints packages missing from the hand-kept registries and exits nonzero on gaps (I). It is deliberately not wired into continuous integration. It runs during authoring, when a human is present to judge whether a gap is a real omission or a package that should never have been registered. Automation that raised an alarm on every experimental package would teach authors to ignore it.

When `law audit` refuses

law audit needs a package to stand in. Run outside any package, it refuses (this transcript is real, diagnostic on stderr):

$ cd /tmp/audit-empty && /Users/rifatjumagulov/Downloads/law-dsl/law audit
law audit: REFUSAL: LPK-E0201: /private/tmp/audit-empty: no law.toml here or above — pass the project via --project
  → run from the project or pass --project <dir>; a new one via `law init`

The hint names a --project flag that law audit does not accept (unknown flag --project is the real reply). Run the command from inside the package directory instead.

A package whose dependencies are not materialized is refused until they are, or until a compiled snapshot is presented explicitly with --clir, as in the runs above:

$ cd packs/vocab/geo && ../../../law audit
law audit: REFUSAL AUDIT_REFUSED: import context not built: no pinned deps/ in law.lock; materialize deps/ or present --clir <CLIR dir>

Reproduction details

Toolchain, hashes, and full transcripts

Commands on this page were run against law 0.1.1, semantics law.core/0.2, standard library 0.2.0 (full law version blocks under Reproduction details on the pinning page).