Document validation defaults and cleanup roadmap
This commit is contained in:
@@ -75,7 +75,8 @@ prompt execution metadata.
|
||||
|
||||
`pipeline.Registries` holds concrete constructors for execution. A
|
||||
`pipeline.ModuleCatalog` exposes module specs for config validation and
|
||||
resolution.
|
||||
resolution. The catalog also exposes validator specs and central default
|
||||
validator-chain mappings without constructing modules or validators.
|
||||
|
||||
Every production module registers a `ModuleSpec` with:
|
||||
|
||||
@@ -91,6 +92,14 @@ instances. Input, validate, and output specs must not declare reference slots.
|
||||
|
||||
Capability checks prevent incompatible pipeline composition before a run starts.
|
||||
|
||||
Every production validator registers a `ValidatorSpec` with:
|
||||
|
||||
- `Key`: validator key used in config and manifests;
|
||||
- `ExecutionClass`: `deterministic` or `llm_backed`.
|
||||
|
||||
Default validator chains are keyed by workflow stage and module key. Production
|
||||
currently registers a default chain for `extract` module `dnd/spells` only.
|
||||
|
||||
## Runner Input And Output
|
||||
|
||||
`pipeline.RunInput` carries:
|
||||
@@ -118,8 +127,8 @@ The runner:
|
||||
2. builds the input adapter and parses the raw input into a source document;
|
||||
3. validates the source document;
|
||||
4. builds the chunker and produces source chunks, retrying when configured;
|
||||
5. validates source chunks against framework invariants and any registered raw
|
||||
chunk validators;
|
||||
5. validates source chunks against framework invariants and the resolved chunk
|
||||
validator chain;
|
||||
6. runs each selected artifact lane in sorted resolved order;
|
||||
7. builds the output encoder and validates logical output file names.
|
||||
8. passes accepted normalized raw outputs, rejected output records, warnings,
|
||||
@@ -181,25 +190,38 @@ Within an artifact lane, the runner:
|
||||
|
||||
## Validators
|
||||
|
||||
The current runner handoff is raw-output based. Extractors, mergers, and
|
||||
The runner handoff is raw-output based. Chunkers, extractors, mergers, and
|
||||
normalizers do not advertise validator chains through their module interfaces.
|
||||
Runner-side raw validation chains receive the raw module output plus stage,
|
||||
lane, module, source, chunk, schema, session, reference, LLM client/profile,
|
||||
binding option, and run metadata context. Merge validators also receive the
|
||||
ordered extract outputs used by the merge, and normalize validators receive the
|
||||
accepted merge output. Empty raw validation chains approve output by default.
|
||||
Response-schema provenance may include in-memory JSON schema bytes for
|
||||
validators. Those bytes are omitted from manifests, diagnostics, and encoded
|
||||
output files.
|
||||
Resolved validation chains receive the raw module output plus stage, lane,
|
||||
module, source, chunk, schema, session, reference, LLM client/profile, binding
|
||||
option, and run metadata context. Chunk validators receive the chunk result
|
||||
collection, merge validators receive the ordered extract outputs used by the
|
||||
merge, and normalize validators receive the accepted merge output. Empty chains
|
||||
approve output by default. Response-schema provenance may include in-memory JSON
|
||||
schema bytes for validators. Those bytes are omitted from manifests,
|
||||
diagnostics, and encoded output files.
|
||||
|
||||
Resolved validator chains come from central default mappings unless a
|
||||
stage-local config override is set on `chunk`, lane `extract`, lane `merge`, or
|
||||
lane `normalize`. Explicit empty overrides are valid and are recorded as empty
|
||||
chains in manifests.
|
||||
chains in manifests. Explicit non-empty overrides replace the default chain and
|
||||
preserve configured order.
|
||||
|
||||
The production default chain for `extract` module `dnd/spells` is:
|
||||
|
||||
1. `generic/valid_json`
|
||||
2. `generic/valid_json_schema`
|
||||
3. `extract/dnd/spells/shape`
|
||||
4. `extract/dnd/spells/source_refs`
|
||||
5. `extract/dnd/spells/source_relatedness`
|
||||
|
||||
No other production module currently has a default validator chain.
|
||||
|
||||
Validator rejection is a non-fatal run outcome: the rejected output is recorded
|
||||
in `RunOutput.Rejected` and does not pass to the next stage. Validator execution
|
||||
errors are framework-level errors and retry according to the relevant binding.
|
||||
Warning-only validators return approved results with warnings; those warnings
|
||||
are promoted only from successful attempts whose outputs are used.
|
||||
|
||||
## Warnings And Failures
|
||||
|
||||
|
||||
Reference in New Issue
Block a user