eric/notarius
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Record every retry attempt outcome

Browse Source
This commit is contained in:
Eric Rakestraw
2026-07-17 15:23:27 +00:00
parent 35bffdf336
commit 236ccc62ad
9 changed files with 387 additions and 134 deletions
Download Patch File Download Diff File Expand all files Collapse all files

13
docs/internal/pipeline.md
View File

@@ -228,10 +228,15 @@ normally. Dependency fingerprints and debug content digests use the same stable
codec bytes that cross those boundaries.
Debug instrumentation wraps run, stage, attempt, validator, and structured LLM
boundaries. Every executed module retry has an attempt envelope containing its
candidate, accepted-attempt warnings, rejection or error, and only the LLM
calls made by that module attempt. Validator attempts retain independent scopes
under `validate/`. Debug-write failures are framework errors; debug data is
boundaries. Every executed chunk, extract, merge, and normalize attempt writes
one terminal envelope for acceptance, validator rejection, module or validator
error, or applicable candidate or final serialization error. The envelope
contains its attempt-local warnings, any available candidate and rejection,
and terminal error text; failures before a candidate exists omit that payload.
Only LLM calls made by the module operation belong to the module attempt.
Validator calls retain independent scopes under `validate/` and are not
duplicated into the module envelope. A failed terminal-envelope write is a
framework error and is joined with any primary attempt error. Debug data is
never used as a checkpoint source. Typed artifact debug envelopes are
domain-neutral, redact sensitive metadata and bytes through the common debug
policy, and record codec identity plus schema and content digests.

33
docs/operations.md
View File

@@ -128,15 +128,17 @@ checkpointing does not write debug output.
Debug artifacts include inputs and outputs for source, chunk, extract, merge,
normalize, and output work, structured LLM request and response data, validator
requests and results, timing, and retry attempt metadata. LLM calls made inside
a retry or validator attempt
write `prompt-000N.json`, `response-000N.json`, and
`response-content-000N.*` files under that attempt directory and are linked from
the attempt `llm_calls` array. Prompt content is written inline in the prompt
artifact. The response metadata and body use the paired files described above;
the body is pretty-printed JSON when possible and raw text otherwise. Merge and
normalize retries use these stable paths:
a module retry write `prompt-000N.json`, `response-000N.json`, and
`response-content-000N.*` files under that attempt directory and are linked
from its `llm_calls` array. Validator calls use separate attempt scopes under
`validate/` and are not duplicated into the module attempt. Prompt content is
written inline in the prompt artifact. The response metadata and body use the
paired files described above; the body is pretty-printed JSON when possible
and raw text otherwise. Retrying stages use these stable module-attempt paths:
```text
chunk/attempt-<NN>.json
extract/<lane-id>/chunk-<NNNNNN>/attempt-<NN>.json
merge/<lane-id>/attempt-<NN>.json
merge/<lane-id>/attempt-<NN>/prompt-<NNNN>.json
merge/<lane-id>/attempt-<NN>/response-<NNNN>.json
@@ -148,10 +150,19 @@ normalize/<lane-id>/attempt-<NN>/response-<NNNN>.json
normalize/<lane-id>/attempt-<NN>/response-content-<NNNN>.<ext>
```
Checkpoint-reused merge and normalize work retains the stage-level input and
output artifacts but has no retry-attempt artifacts because no module attempt
executed. Debug artifacts may contain source material, reference material,
prompt inputs, model outputs, and other sensitive data. Typed artifact
Every executed chunk, extract, merge, and normalize attempt has one terminal
envelope recording acceptance, validator rejection, or a module, validator,
candidate-serialization, or final-serialization error as applicable. It
includes attempt-local warnings and any available candidate or rejection. A
failure before a candidate exists has no candidate payload. If the envelope
cannot be persisted, the run reports that debug failure together with any
primary attempt error.
Checkpoint-reused chunk, extract, merge, and normalize work retains the
stage-level input and output artifacts but has no retry-attempt artifacts
because no module attempt executed. Debug artifacts may contain source
material, reference material, prompt inputs, model outputs, and other sensitive
data. Typed artifact
envelopes include domain-neutral codec identity, redacted metadata and content,
and digests of the stable codec bytes. API keys are not written, and obvious
credential-shaped values and sensitive map keys are redacted, but debug