Add a framework mechanism for prepared modules and validators to contribute checkpoint identity fingerprints
This commit is contained in:
@@ -158,7 +158,12 @@ reference into an immutable effective catalog and adds a generated
|
||||
canonical-name-only JSON input to every structured completion request. Overlay
|
||||
failures therefore stop construction before source parsing or an LLM call;
|
||||
campaign references remain separate disambiguation inputs and never become
|
||||
source evidence. The separate `internal/modules/dnd/codec/spells` package
|
||||
source evidence. Both the extractor and deterministic catalog validator expose
|
||||
the effective base-plus-overlay semantic digest as scoped prepared-component
|
||||
checkpoint identity. Raw overlay provenance independently covers file-byte
|
||||
changes, while the semantic digest also invalidates reuse when the embedded
|
||||
catalog or catalog composition changes. The separate
|
||||
`internal/modules/dnd/codec/spells` package
|
||||
owns the durable schema and stable JSON representation for artifact kind
|
||||
`dnd/spell-list`. The runner keeps the result typed through validators and later
|
||||
stages, using the codec only for checkpoint, debug, and output boundaries.
|
||||
|
||||
@@ -126,8 +126,16 @@ then output. It stops at the first error with pipeline, stage, lane, module, and
|
||||
validator context as applicable. It never invokes an operation method.
|
||||
|
||||
`PreparedPipeline` keeps private constructed executors and exposes cloned
|
||||
resolved input, chunk, lane, and output identities. `pipeline.RunInput` carries
|
||||
that prepared pipeline, raw source input, run identity and timing, optional
|
||||
resolved input, chunk, lane, and output identities. Prepared components may
|
||||
implement `pipeline.CheckpointFingerprintProvider` to contribute explicit
|
||||
semantic identities to checkpoint reuse. Preparation trims and validates each
|
||||
non-secret name and value, prefixes it with the component's stage, lane,
|
||||
module, and validator scope, rejects duplicates, and retains the resulting
|
||||
sorted collection behind a defensive-copy accessor. Fingerprints must be
|
||||
stable and must not contain source content, credentials, local paths,
|
||||
timestamps, or other invocation-specific values.
|
||||
|
||||
`pipeline.RunInput` carries that prepared pipeline, raw source input, run identity and timing, optional
|
||||
session and profile metadata, a chunk-plan store and mode, and checkpoint/debug
|
||||
collaborators. The runner
|
||||
parses source bytes through the already constructed input adapter. Later stage
|
||||
@@ -267,6 +275,14 @@ bytes, and decode failures become explicit reuse misses and execute the step
|
||||
normally. Dependency fingerprints and debug content digests use the same stable
|
||||
codec bytes that cross those boundaries.
|
||||
|
||||
The CLI includes prepared-component fingerprints in the run-wide checkpoint
|
||||
identity alongside resolved configuration, raw input, reference provenance,
|
||||
runtime overrides, and LLM-profile fingerprints. Module metadata is not used
|
||||
implicitly for cache identity: components opt in only with stable semantic
|
||||
values that can change accepted output. Adding or changing a component
|
||||
fingerprint intentionally produces a cold cache miss. Existing checkpoint
|
||||
schemas and paths remain unchanged.
|
||||
|
||||
Debug instrumentation wraps run, stage, attempt, validator, and structured LLM
|
||||
boundaries. Every executed chunk, extract, merge, and normalize attempt writes
|
||||
one terminal envelope for acceptance, validator rejection, module or validator
|
||||
|
||||
@@ -25,6 +25,9 @@ not `bypass`.
|
||||
`internal/framework/checkpoint` owns checkpoint identity, manifests, payload
|
||||
codecs, loader, and recorder. The CLI constructs a recorder whenever checkpoint
|
||||
recording is enabled and constructs a loader only for a `--resume` invocation.
|
||||
Identity incorporates explicit stable semantic fingerprints collected from
|
||||
prepared modules and validators in addition to configuration, input,
|
||||
references, runtime overrides, and LLM profiles.
|
||||
The serialized
|
||||
`workspace_schema_version` identifiers are frozen wire-compatibility fields;
|
||||
they do not describe a current public state surface.
|
||||
|
||||
@@ -89,6 +89,11 @@ Checkpoints use the selected root and the existing identity hierarchy:
|
||||
<checkpoint-root>/<pipeline-id>/<input-key>-<source-or-input-digest>/<pipeline-digest>/<identity-digest>/...
|
||||
```
|
||||
|
||||
The final identity digest includes stable semantic fingerprints explicitly
|
||||
contributed by prepared modules and validators. Adding or changing one of
|
||||
these fingerprints intentionally causes a cold cache miss; old checkpoint
|
||||
directories are left in place and are never migrated or deleted automatically.
|
||||
|
||||
An empty configured directory selects
|
||||
`<os.UserCacheDir>/notarius/checkpoints`. The root is exact when configured.
|
||||
Created directories and files use `0700` and `0600` permissions on supported
|
||||
|
||||
@@ -6,7 +6,10 @@ The accepted baseline implementation is complete. It provides effective
|
||||
SRD-plus-overlay catalog composition, canonical-name prompt grounding,
|
||||
deterministic catalog validation, the maintained `retries: 2` policy, and
|
||||
provenance and checkpoint identity coverage through the assembled offline
|
||||
workflow.
|
||||
workflow. Checkpoint identity includes independently scoped extractor and
|
||||
validator fingerprints of the effective catalog, closing the earlier gap in
|
||||
which raw overlay changes invalidated reuse but embedded catalog changes did
|
||||
not.
|
||||
|
||||
External quality evaluation is pending. This repository contains the
|
||||
maintained example and offline fake-LLM coverage, but no approved
|
||||
|
||||
Reference in New Issue
Block a user