docs← Back to article

Markdown for LLMs

Queries: pitfalls

The source Markdown for this article. Copy it into your assistant or download it as a text file.

Download this articlePlain text ↗
# Queries: pitfalls

Wrong forms, silent outcomes, diagnostics, typical mistakes of
formalizers and AI agents, how to detect them. Items marked
"confirmed by run" were reproduced during this research on the
installed `law` (semantics `law.core/0.2`).

## 1. `when not P` instead of `when not_known(P)` (confirmed by run)

Strict `not` requires a refutation: a missing fact gives `NEITHER`,
and the rule stays silent even in the "positive" case. During this research the
first version of the `research.queries.truth_why_not` package with `when not
has_debt(r)` gave `NEITHER` in case 01:

```text
FAIL [research.queries.truth_why_not#authored] tests/01-borrow-granted.lawtest / urn:query:research-queries-01
      truth_status == TRUE_ONLY: в документе NEITHER
```

The "unless otherwise established" form is only `not_known(P)` (only in
a rule body). Detection: a `TRUE_ONLY` positive test next to a `NEITHER`
negative — a vacuous positive turns red at once instead of staying silent.

## 2. Wrong-arity question — `QUERY_INVALID`, not `NEITHER`

A question literal is checked against the presented program's signature at any
depth — in literal kinds, in a formula, in a `collect` generator,
in a nested question. A mismatch is rejected BEFORE
computation with an error object `{"error": {"code": "QUERY_INVALID", …}}`.
A predicate the program does NOT declare is not judged: open world —
the question is legal, `NEITHER` is the answer. An AI agent seeing `NEITHER`
fixes facts when the argument count in the question needs fixing. Detection:
a short-arity conformance scenario; the refusal text names the predicate and
the declared vs given arity.

## 3. `focused_truth` in 0.2 — a refusal without a result

The kind executes in 0.3 semantics; in 0.1 and 0.2 the question is rejected
with a `FOCUS_UNSUPPORTED_SEMANTICS` issue (fatal) without a result, and the
answer document is a separate focused-evaluation form, which the
full audit-document consumer does not accept. The AI
mistake is substituting the neighbouring `truth` "until better times": implementations must not substitute a non-marginal kind with a neighbour,
and the same binds the author. Detection: a conformance test pins the 0.2 refusal.

## 4. `position_count` with two arguments (confirmed by run)

The observation takes two names and an integer — `position_count(Template,
Status, N)`. The short `position_count(ReturnOnTime, 0)` form does not
lower:

```text
SKIP ... tests/04-no-member.lawtest
      тест не лоуверится (LDC-E1316: position_count(Шаблон, Статус, N) принимает два имени и целое (§267.4))
```

A SKIP-status step walks past the report as "not executed" (code 2), not
as a failure — the skip is easy to miss. Detection: require code 0 and
zero "not executed" in the `law test` total, not just no FAIL.

## 5. Expecting `truth_status` on `positions()` and `collect`

The `NORM_POSITION` and `COLLECTION` kinds have no `truth_status` axis:
mandatory dimensions are `applicability_status` + `normative_status` (+
supports) and the `value` collection respectively. A missing dimension
is absent, not equal to `NEITHER`. Expecting another axis is an observation
mistake, and reading a missing `truth_status` as "the law's silence" is a
conclusion mistake. Detection: the observation dictionary is closed; correct
expectation specimens are an activation-with-judgment scenario (four position axes)
and a minutes-between scenario (two collection axes).

## 6. Uppercase `bearer`/`holder` — lowercase only

The examples write `bearer`/`holder` only as lowercase keywords.
Older material keeps the capital form in places — that is data, not a model for new examples.

## 7. Silent `COMPUTED` instead of a cause: a term error visible only in the status

A `truth`/`why_not` result with `NEITHER` backed by a candidate-rule term error
carries a status (`MISSING_INPUT`, `MISSING_POLICY`, `NON_EXECUTABLE`,
`EXTERNAL_UNAVAILABLE`, `TYPE_ERROR`, `RUNTIME_ERROR`, `RESOURCE_LIMIT`) and
`missing_inputs` — while a test checking one `truth_status` stays
green even on "law not computed". Earlier builds answered
`COMPUTED` with empty `missing_inputs`. Detection: positive tests always as a
triple (`result_kind`, `truth_status`, `evaluation_status`); on `why_not` the
same plus reading `termErrors`. The opposite-signed exception:
`MISSING_PARAMETER_VALUE` is the law's own silence, the status stays
`COMPUTED`, the cause is a `warning` issue; an `error` level here
would be a computation refusal.

## 8. AI-agent traps in summary

- Writing a question "from memory" about a predicate missing from the program
  and reading `NEITHER` as a model refusal — that is the open-world
  answer; predicate absence is checked against sources, not against the status.
- Inventing question-literal values past the value reader (`1/0` in a
  `Decimal` slot) — a `QUERY_INVALID` refusal, indistinguishable in the
  report from an arity refusal without reading the message.
- Asking `duties`/`liberties`/`powers`/`immunities` instead of `positions()`
  "for precision" without checking the kind executes — a non-executed kind is
  rejected with a diagnostic, not substituted by a neighbour; what
  executes in 0.2 is pinned by conformance scenarios, not by the kind list.
- Losing `queryId`: two forms of one question under one ID give one
  canonical `query`; an ID mismatch is `LDC-E1356`,
  not "another answer".