From 23c55f8925a05ea15fca63ac513d1dc219ddf60b Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Wed, 22 Jul 2026 09:19:13 -0500 Subject: [PATCH] Add implementation roadmap to clean up D&D extract module validation issues --- docs/roadmap/implementation.md | 483 +++++++------------------ docs/roadmap/ordered-pipeline-steps.md | 339 ----------------- 2 files changed, 129 insertions(+), 693 deletions(-) delete mode 100644 docs/roadmap/ordered-pipeline-steps.md diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md index 8559cb3..235e98a 100644 --- a/docs/roadmap/implementation.md +++ b/docs/roadmap/implementation.md @@ -1,399 +1,174 @@ -# Implementation Plan: Ordered Pipeline Follow-Up +# D&D Validation-Boundary Alignment Implementation Plan -## Status +Status: Proposed. -Completed on 2026-07-22. This document retains the implementation sequence for -historical context; current contracts are maintained in the canonical CLI, -configuration, operations, integration, and internal documentation linked from -the [development guide](../development.md). +Implement this plan in order. It repairs the immediate combat extraction +failure, makes D&D validation diagnostics consistently domain-owned, and then +aligns the spell and NPC private response boundaries with the same policy. -## Purpose +Do not change durable artifact schemas, artifact kinds, public Go types, +framework retry behavior, checkpoint formats, or the scene chunker. Existing +v1 private-schema identities may be corrected in place because Notarius has not +been run in production. Changed prompt and schema content will invalidate +development checkpoints through existing fingerprints. -Record the work that closed the correctness, observability, test, and -maintainability gaps in the implemented -[Ordered Pipeline Steps](ordered-pipeline-steps.md) feature. That feature -roadmap remains the authority for product intent, policy choices, acceptance -criteria, and exclusions. This document preserves the ordered, -decision-complete implementation sequence for the follow-up work. +## Cross-Stage Decisions -## Background +Private LLM schemas own the transport envelope: required fields, JSON types, +nullability, array/object shape, and unknown-field rejection. Deterministic +domain validators own semantic rules: supported enum values, nonblank values, +required non-empty collections, positive and resolvable source units, +catalog/identity policy, and normalized invariants. -The original implementation is complete in broad architecture. Pipelines now -resolve and prepare one canonical ordered-step model, execute hard barriers -between steps, hand accepted normalized artifacts to later operations as -canonical generated references, include generated identity in checkpoint and -output provenance, support `--recompute-step`, and use generated NPC output at -operation time in the D&D spell and combat consumers. Current documentation and -maintained examples describe that model. +For typed D&D artifacts, `generic/valid_json` remains first as a representation +sanity check. All deterministic domain validators that can reject a candidate +run next. `generic/valid_json_schema` runs after them as a durable-schema +backstop, followed by warning-only relatedness validators. This order gives +expected candidate failures bounded domain reason codes while retaining a final +check that typed encoding conforms to the durable contract. -The completed follow-up addressed these narrower gaps: +Do not expose raw JSON Schema errors or candidate values through the generic +validator. Do not add tests that require particular words or phrases to remain +in prompt prose. -- selective recomputation now hydrates an unselected predecessor from its - accepted normalized artifact without requiring its extract and merge state; -- the failed checkpoint decision is recorded before a required-predecessor - error returns; -- checkpoint reason codes are assigned explicitly rather than inferred from - human-readable prose; -- runner lane and checkpoint orchestration has explicit responsibility seams; - and -- CLI and resumed-producer acceptance coverage exercises the complete recovery - contract. - -No new product feature is introduced by this plan. Preserve configuration file -version 3 and checkpoint workspace schema `notarius.workspace.v3`; the fixes do -not require a new persistent format. - -## Instructions For Every Stage - -Before changing code, read `docs/development.md`, all documents under -`docs/policy/`, and the task-specific internal documents named there. Use the -repository's code knowledge graph for code discovery before falling back to -text search. - -Implement the stages in order. Each stage must leave the repository formatted, -building, and passing its focused tests. Preserve these invariants throughout: - -- The pipeline retains one input, chunk plan, output encoder, run identity, - checkpoint identity, failure boundary, worker budget, and provider scheduler. -- Every selected module and validator is constructed before source parsing. -- Steps schedule fixed extract, merge, and normalize lanes; this work must not - introduce module-to-module calls or a general DAG scheduler. -- Generated artifacts remain cloned operation-time context, never source - evidence. Never emit their content, source material, credentials, or local - paths in decisions, manifests, logs, or summaries. -- Public ordering remains step order, lane ID, and source/chunk order as - applicable. Refactoring must not expose completion order. -- Tests must follow `docs/policy/testing.md`: protect observable recovery, - orchestration, and CLI contracts without asserting private helper calls, - exact prose, goroutine choreography, or full-document snapshots. -- Update current-behavior documentation in the same stage that changes the - corresponding behavior. Keep detailed implementation sequencing only here. - -## Target Checkpoint Semantics - -Use these definitions consistently in all stages: - -- A **stage checkpoint** is internal resumable state for extract, merge, or - normalize. Ordinary resume may continue to reuse this state progressively. -- An **accepted lane artifact** is the one successful normalized artifact for a - `(step ID, lane ID, normalizer module)` under the current checkpoint identity. - It is the dependency exposed to a later step. -- An unselected required predecessor satisfies selective recomputation when its - accepted lane artifact can be validated and hydrated. Its extract and merge - stage checkpoints are not prerequisites for that handoff. -- A selected lane and its transitive dependents execute. They must never use - accepted-artifact hydration to bypass forced execution. -- If a required predecessor's accepted lane artifact is missing, rejected, - corrupt, non-canonical, or incompatible, fail the run before any dependent - lane starts. Do not implicitly rerun that predecessor. -- Ordinary resume without `--recompute-step` keeps its existing progressive - stage-reuse and cold-miss behavior. - -## Stage 1: Decompose Runner And Checkpoint Orchestration - -### Objective - -Create explicit, testable ownership seams for ordered-step coordination, -per-lane execution, and per-stage checkpoint handling without changing -observable behavior. +## Stage 1: Repair Combat Extraction ### Changes -1. Keep `Runner.Run` as the pipeline-wide coordinator. Extract a small - step-coordination helper responsible only for iterating prepared steps, - building generated reference sets at each barrier, invoking the lane engine, - and merging deterministic outcomes. -2. Split `runLanes` in `internal/framework/pipeline/runner_concurrent.go` into - helpers with these responsibilities: - - initialize lane state and resolve extract checkpoint state in lane order; - - run the bounded extract worker/continuation engine; - - collect terminal lane results and choose failures deterministically; and - - merge lane-local output into step output. - Preserve the existing bounded channels, cancellation, drain behavior, - chunk-first dispatch, lane ordering, and step barrier. -3. Split `continueTypedLane` in - `internal/framework/pipeline/runner_typed.go` into stage-specific merge and - normalize helpers. Each helper should own dependency construction, checkpoint - loading and canonical validation, execution/retry/validation when needed, - checkpoint recording, debug envelopes, and its typed result. Use small - result structs rather than long parallel return lists. -4. Centralize repeated checkpoint-decision flow in one pipeline helper that can - apply forced-execution policy, validate canonical stored artifacts, record - the final observable decision, and return a contextual error. Do not yet - change categories, reason codes, or required-predecessor semantics; Stages 2 - and 3 will change those deliberately. -5. Keep stage-specific code where the data shapes genuinely differ. Do not - introduce reflection, a generic stage state machine, or callbacks that hide - the fixed extract/merge/normalize lifecycle. +- Update the combat extraction instructions to enumerate the complete allowed + values: + - `turn_kind`: `turn`, `reaction`, `legendary_action`, `lair_action`, + `other`; + - action `category`: `attack`, `spell`, `movement`, `item`, + `ability_check`, `saving_throw`, `condition`, `other`. +- Keep the private combat schema structurally permissive and the durable schema + strict. Do not restore enum, minimum, `minLength`, or `minItems` constraints + to the private schema. +- Reorder the combat extraction default chain to: + `generic/valid_json`, combat shape, combat source references, + `generic/valid_json_schema`, combat source relatedness. +- Reorder the combat normalization default chain to: + `generic/valid_json`, combat shape, normalized invariants, combat source + references, `generic/valid_json_schema`, combat source relatedness. +- Preserve configured validator overrides as authoritative; only production + default composition changes. ### Tests -- Existing runner, checkpoint, barrier, ordering, cancellation, retry, debug, - and D&D integration tests must pass without expectation changes except moves - required by renamed private test fixtures. -- Add no tests for helper boundaries or collaborator call counts. Add a narrow - regression assertion only if the refactor exposes an observable behavior not - already protected. +- Add an assembled combat pipeline case whose raw LLM response contains an + unsupported turn kind or action category. Exhausted retries must produce a + non-fatal `invalid_combat_turn_shape` rejection owned by the combat shape + validator, not `json_schema_invalid` or a framework error. +- Retain coverage that a later valid retry succeeds and discarded-attempt + warnings/rejections do not become durable. +- Update registrar contract tests to assert the new extraction and + normalization order. +- Rely on behavioral enum/schema tests and prompt fingerprint coverage; do not + add prompt-word change-detector tests. -### Completion Gate +### Completion Check -Run `go test ./internal/framework/pipeline ./internal/framework/checkpoint -./internal/modules/integration` and the pipeline race tests. Review the diff to -confirm this stage changes structure only: serialized output, checkpoint state, -decision values, failure selection, and module invocation behavior must remain -unchanged. +Run `go test ./internal/modules/dnd/extract/combatturns +./internal/modules/dnd/validate/combatturns/... ./internal/modules/dnd/register +./internal/modules/integration` and `git diff --check`. -## Stage 2: Make Checkpoint Decisions Typed And Observable - -### Objective - -Assign stable decision categories and reason codes explicitly, and retain the -decision that causes a required-dependency failure. +## Stage 2: Make D&D Validator Ordering Consistent ### Changes -1. In `internal/framework/pipeline/checkpoint.go`, introduce string-backed - internal types and constants for checkpoint decision categories and reason - codes. Keep the current JSON strings and public artifact fields compatible. - Categories remain exactly `executed`, `reused`, `forced_recompute`, and - `dependency_invalidated`. -2. Replace prose inspection in `internal/framework/checkpoint/loader.go` with an - explicit decision constructor accepting category, reason code, and optional - detail. Remove every `strings.Contains` classification branch. Assign a code - at the validation site using this bounded vocabulary: - - `loading_disabled`, `checkpoint_missing`, `checkpoint_path_invalid`, - `checkpoint_read_failed`, and `checkpoint_decode_failed`; - - `workspace_schema_incompatible` and `identity_mismatch`; - - `stage_mismatch`, `step_mismatch`, `lane_mismatch`, `module_mismatch`, and - `status_not_reusable`; - - `dependency_mismatch` for the `dependency_invalidated` category; - - `artifact_payload_invalid`, `artifact_digest_mismatch`, - `artifact_codec_incompatible`, and `artifact_not_canonical`; and - - `checkpoint_reused` and `accepted_artifact_reused` for successful reuse, - and `recompute_step` for forced execution. - Retain an existing code not listed here only when a current external or - documented contract already depends on it. -3. Keep detail human-readable, UTF-8, bounded, and sanitized through one helper. - Detail may identify stage, step, lane, module, expected status, or schema - version, but must not include artifact/reference content, source content, - credentials, environment values, or local paths. Tests must assert codes and - safety properties, not exact detail prose. -4. Change the centralized runner decision flow from Stage 1 so the final loader - or canonical-validation decision is recorded before returning a - required-predecessor error. The contextual error must identify the step and - lane and include the stable reason code; it must not interpolate unsafe - loader detail. -5. Propagate typed values without lossy conversion through checkpoint events, - run manifests, debug summaries, and CLI diagnostics. Convert to strings only - at existing serialized boundaries unless changing an internal field type is - simpler and wire-compatible. -6. Update the canonical current-behavior owners in the same change: - `docs/internal/state.md` owns the internal decision flow, while - `docs/operations.md` owns operator diagnosis and any operator-visible reason - code table. Link rather than duplicate the table elsewhere. +- Move `generic/valid_json_schema` behind all rejecting domain validators in + every spell and NPC extraction and normalization default chain: + - spell extraction/normalization: shape, catalog, source references, schema, + then source relatedness; + - NPC extraction: shape, source references, schema, then source relatedness; + - NPC normalization: shape, identity, source references, schema, then source + relatedness. +- Keep `generic/valid_json` first and warning-only source relatedness last. +- Do not change validator implementations, reason codes, warning promotion, + retry counts, or user-provided chain order. +- Document the default-chain policy in the pipeline/module internals: domain + validators diagnose expected semantic failures and the generic schema + validator is the final rejecting representation backstop. ### Tests -- Add a table in `internal/framework/checkpoint` that exercises one - representative input per reason-code family and asserts category, code, - bounded valid UTF-8 detail, and absence of supplied secret/path sentinels. -- Add pipeline behavior tests proving a required-predecessor failure records the - underlying missing, corrupt, incompatible, or dependency-invalidated - decision before returning. -- Retain focused manifest/debug serialization assertions for category and code; - do not add exact-detail or full-manifest snapshots. +- Update production registrar tests for every affected chain. +- Add one representative spell and NPC assembled rejection proving that a + domain-invalid but encodable candidate is attributed to the owning domain + validator rather than the generic schema validator. Do not duplicate each + validator package's existing case matrix at integration level. +- Confirm explicitly configured validator chains retain their exact configured + order. -### Completion Gate +### Completion Check -Run `go test ./internal/framework/checkpoint ./internal/framework/pipeline -./internal/core/artifacts ./internal/core/debugbundle ./internal/cli`. Search the -production checkpoint package to confirm no decision category or reason code is -derived from diagnostic prose. +Run `go test ./internal/modules/dnd/register ./internal/modules/integration +./internal/framework/pipeline` and `git diff --check`. -## Stage 3: Hydrate Required Predecessors From Accepted Normalized Artifacts - -### Objective - -Make selective recomputation enforce the lane-level accepted-artifact contract: -a valid normalized producer artifact is sufficient even when its extract or -merge stage cache is unavailable. +## Stage 3: Align Spell and NPC Private Response Boundaries ### Changes -1. Extend the checkpoint loader contract with a dedicated accepted-normalized- - artifact lookup. Implement it in the filesystem loader and every no-op or - test implementation. The lookup receives step ID, lane ID, and normalizer - module key and returns the normalized artifact, its warnings, and an explicit - checkpoint decision. It must not require caller-supplied extract or merge - dependency fingerprints. -2. The filesystem lookup may reuse the existing normalize manifest and payload; - do not add a second persistent copy. It is reusable only when all of the - following hold: - - workspace schema is v3 and the non-empty stored checkpoint identity matches - the current invocation identity; - - stage, step, lane, normalizer module, and successful status match; - - the manifest output digest matches the payload; and - - the payload can subsequently be validated through the registered artifact - codec. - Skipping caller-supplied merge dependencies is safe only because the matched - non-empty checkpoint identity already binds the current input, resolved - topology and configuration, external references, runtime overrides, LLM - profiles, and component semantic fingerprints. Do not weaken or omit that - identity check. - A loader lacking a verifiable current identity must return an unavailable - decision rather than perform accepted-artifact reuse. -3. Add a pipeline hydration helper that decodes the returned serialized - artifact through the producer's registered codec, re-encodes it canonically, - and requires exact artifact kind, schema ID/name/version/digest, media type, - content bytes, and content digest. Return a runner-owned clone with producer - step, lane, module, and source identity. Any mismatch is an explicit bounded - decision and the bytes never reach a consumer. -4. Before normal execution of a lane marked in `RequireReusableLanes`, use the - accepted-artifact lookup: - - on success, mark the lane terminal without invoking extract, merge, - normalize, or their validators; - - append the accepted normalized output in the normal deterministic location - and restore only the warnings stored with that normalized checkpoint; - - record one `reused` normalize decision with a stable accepted-output reuse - reason code; do not synthesize extract/merge decisions, warnings, or - rejections that were not loaded; and - - allow the ordinary step barrier and generated handoff code to consume that - output exactly as it consumes a freshly executed output. -5. On missing, rejected, corrupt, non-canonical, or incompatible accepted state, - record the decision and fail the run before the dependent step begins. Do - not fall back to stage execution. Forced lanes must bypass this hydration - path and execute normally. -6. Leave ordinary resume unchanged when no lane is marked - `RequireReusableLanes`: it may reuse or recompute extract, merge, and - normalize progressively under the existing cold-miss rules. -7. Keep generated-reference fingerprints and provenance unchanged. Hydrating a - byte-identical producer must yield the same canonical handoff digest and - downstream dependency fingerprint as fresh execution. -8. Update `docs/internal/pipeline.md`, `docs/internal/state.md`, and - `docs/operations.md` in this stage to distinguish progressive stage reuse - from accepted normalized-artifact hydration and to document the fail-rather- - than-rerun rule for invalid required predecessors. +- Revise the existing v1 spell and NPC private schemas in place: + - retain required fields, JSON types, array/object structure, + `additionalProperties: false`, and omission of framework-assigned fields; + - remove `minLength`, `minItems`, and positive-number `minimum` constraints; + - leave the durable spell and NPC schemas unchanged. +- Replace `shared.UnitRef` in the private spell and NPC response DTOs with + integer candidates so zero and negative unit IDs survive decoding and mapping + into `source.SourceRef` for deterministic source validation. +- Update canonicalization and ordering helpers to operate on candidate integers + without repairing invalid values. Valid positive IDs retain current output, + ordering, and exact-deduplication behavior; invalid ranges remain available + to validators. +- Keep malformed JSON, missing/unknown fields, wrong JSON types, and + non-integer source IDs as LLM-boundary errors. +- Update LLM/module internals and the spell/NPC integration contracts to state + the structural-private/semantic-validator ownership boundary. ### Tests -- At the pipeline/checkpoint boundary, create a valid producer normalize - checkpoint while omitting or corrupting its extract and merge checkpoints. - Select a later step for recomputation and prove the producer hydrates, no - producer operation or validator runs, and the dependent receives the exact - canonical artifact. -- Cover missing, rejected-status, corrupt, non-canonical, wrong-codec-identity, - and wrong-content-digest normalize state. Assert failure and the recorded - stable decision before any consumer invocation. -- Prove forced producers execute instead of hydrating, while a reusable - predecessor and an unrelated lane remain reusable. -- Prove fresh and hydrated producer outputs create identical generated - provenance and downstream checkpoint fingerprints without exposing content - or paths. +- For each private schema, prove structurally valid candidates with blank + strings, empty required collections, and nonpositive unit IDs pass the + private schema, while missing fields, unknown fields, and wrong JSON types do + not. +- Through raw-JSON LLM fakes, prove semantic values survive decoding and mapping + without repair. +- Add representative assembled cases showing: + - blank or empty spell/NPC fields are rejected by the appropriate shape + validator; + - nonpositive or nonexistent unit IDs are rejected by the appropriate source + validator; and + - exhausted validation retries remain non-fatal rejected outputs. +- Preserve existing valid mapping, source-position ordering, deduplication, + catalog, identity, checkpoint-fingerprint, and durable codec tests. -### Completion Gate +### Completion Check -Run `go test ./internal/framework/checkpoint ./internal/framework/pipeline -./internal/modules/integration` and the corresponding race tests. Manually -inspect one failed test fixture to confirm the accepted artifact remains on -disk but its bytes do not appear in the manifest, decision detail, debug -summary, or error. +Run `go test ./internal/modules/dnd/extract/spells +./internal/modules/dnd/extract/npcs ./internal/modules/dnd/validate/spells/... +./internal/modules/dnd/validate/npcs/... ./internal/modules/integration` and +`git diff --check`. -## Stage 4: Complete Recompute CLI And Recovery Acceptance Coverage +## Final Verification -### Objective +Run: -Exercise the operator-visible recomputation contract through stable boundaries -and close the acceptance-test omissions from the original plan. - -### Changes - -1. Add one table-driven CLI contract test for `--recompute-step` covering: - - a valid explicit step and the implicit `default` step; - - repeated flags and an empty or unknown step ID; - - use without `--resume`; - - use when checkpoint recording is disabled; and - - combination with `--only`. - Assert exit classification and stable identifying fragments or reason codes, - not complete prose. -2. Add one filesystem-backed CLI execution test using deterministic fake - modules and a three-step generated dependency chain plus one unrelated lane: - - perform a fresh checkpointed run; - - remove or corrupt only the required producer's extract and merge state - inside `t.TempDir()`, leaving its normalize artifact valid; - - resume with the middle step selected; - - assert the selected lane and transitive dependents execute, the predecessor - hydrates without module calls, the unrelated lane reuses, and output and - decision ordering are deterministic; and - - invalidate the producer normalize state in a subcase and assert the command - fails before dependent execution with the bounded decision preserved. -3. Add or extend one generic two-step pipeline test proving handoff succeeds - both from fresh producer execution and from accepted normalized-artifact - hydration. Keep this at the pipeline boundary if the CLI execution test - already proves flag wiring; do not duplicate every CLI case end to end. -4. Review existing recomputation policy tests. Retain the pure closure test - because it protects transitive selection, but remove or consolidate any new - test that merely repeats the CLI or pipeline behavior above. -5. Make only production changes revealed as necessary by these contract tests; - do not add new flag semantics or broaden the feature roadmap. - -### Completion Gate - -Run `go test ./internal/cli ./internal/framework/pipeline -./internal/framework/checkpoint`, then run the CLI, pipeline, and checkpoint -packages under the race detector. Confirm no test asserts internal loader call -counts, exact decision detail, filesystem layout outside a temp workspace, or -the exact length of any prompt or prefix. - -## Stage 5: Current Documentation And Release Verification - -### Objective - -Reconcile all canonical documentation after the staged changes and verify the -complete feature against policy and roadmap. - -### Changes - -1. Re-read `docs/internal/pipeline.md`, `docs/internal/state.md`, - `docs/operations.md`, and `docs/cli.md` against the final code. Correct any - stale statements left by Stages 1 through 4 without duplicating their - canonical contracts. -2. Ensure internal documentation describes the coordinator, lane engine, stage - checkpoint flow, and accepted-artifact validation at the responsibility - level without listing volatile private helper names. -3. Confirm the operator-visible reason-code table has one canonical owner and - other documents link to it rather than maintaining parallel copies. -4. Remove stale implementation claims exposed by this work and ensure the - feature roadmap continues to describe target policy rather than task - sequencing. - -### Verification - -Run the repository-prescribed commands from `docs/development.md`: - -```sh +```text +git diff --check go test ./... go vet ./... go build ./cmd/notarius +go test -race ./internal/modules/dnd/... ./internal/framework/pipeline ./internal/cli ./internal/modules/integration ``` -Also run race tests for the pipeline, checkpoint, CLI state, D&D NPC registry, -and D&D integration packages. Validate maintained examples/configurations -through the production catalog. Review one fresh run, ordinary resumed run, -forced-recompute run, hydrated-predecessor run, and invalid-predecessor failure -for deterministic ordering, correct decisions, bounded provenance, and absence -of generated content, secrets, or local paths in manifests and diagnostics. - -### Completion Gate - -The follow-up is complete only when every finding summarized above is protected -by a stable behavioral test, the current documentation matches the corrected -implementation, the full verification suite passes, and the worktree contains -no temporary adapters or TODOs introduced by these stages. +Review current-behavior documentation for stale statements that private spell, +NPC, or combat schemas own semantic validation. Confirm the scene schema and +prompt remain unchanged: their enumerations are explicitly communicated and +scene-plan construction has a distinct structural mapping boundary. ## Open Questions -None. The feature roadmap and this plan fix the required product and -architecture choices. If implementation reveals that accepted normalized -artifacts cannot be validated safely without a persistent format change, stop -and amend this plan rather than weakening identity, codec, or content -validation. +None. The stages above define the validation ownership, default ordering, +compatibility policy, diagnostic behavior, and test boundaries required for +implementation. diff --git a/docs/roadmap/ordered-pipeline-steps.md b/docs/roadmap/ordered-pipeline-steps.md deleted file mode 100644 index dfb3462..0000000 --- a/docs/roadmap/ordered-pipeline-steps.md +++ /dev/null @@ -1,339 +0,0 @@ -# Scope: Ordered Pipeline Steps - -## Status - -Implemented. This document preserves the bounded feature policy, architecture -choices, acceptance criteria, and exclusions. Current behavior belongs in the -canonical [CLI](../cli.md), [Configuration](../config.md), -[Operations](../operations.md), and [internal pipeline](../internal/pipeline.md) -documentation rather than in this roadmap. - -## Policy Recommendation - -Treat ordered pipeline steps, generated artifact references, and -dependency-aware checkpoint reuse as one coherent platform capability. The D&D -proving workflow produces accepted normalized NPC output first and then supplies -it to spell extraction, combat-turn extraction, and combat-turn normalization. - -## Intended Outcome - -A configured pipeline may contain multiple ordered steps while retaining one -pipeline-wide input, chunk plan, output, worker budget, LLM scheduler, run -manifest, and failure boundary. Every artifact lane still follows the fixed -extract, validate, merge, validate, normalize, and validate lifecycle. Steps -add explicit barriers between groups of lanes; they do not create arbitrary -stage graphs. - -An accepted normalized artifact from an earlier step may be bound explicitly -to declared reference slots in a later step. The framework remains -domain-neutral, and generated references remain contextual material rather than -source evidence. - -## Fixed Product And Architecture Decisions - -### Pipeline shape - -- Input parsing and chunk planning remain pipeline-wide and execute once. -- A step contains one or more artifact lanes. Step order is configuration order. -- Lanes within a step remain independent and may use the existing bounded - concurrency model. -- A later step cannot begin until every lane in the current step is terminal - and every generated artifact it requires is accepted and available. -- Public artifact and failure ordering is step order followed by deterministic - lane and source-chunk order, never completion order. -- Output encoding occurs once, after every step succeeds. -- This is not an arbitrary DAG, a general workflow language, concurrent - cross-lane reconciliation, or permission for modules to invoke other modules. - -### Configuration model - -Existing single-step pipelines remain valid. A top-level `artifacts` map is -treated as an implicit step with stable ID `default`. A pipeline may configure -either `artifacts` or `steps`, but not both. Explicit steps must be non-empty -and have unique, trimmed, non-empty IDs. Artifact lane IDs must remain unique -across the entire pipeline so output paths, selectors, manifests, errors, and -checkpoint scopes remain unambiguous. - -The target configuration shape is: - -```yaml -pipelines: - dnd-session: - input: seriatim - chunk: generic - steps: - - id: identify-npcs - artifacts: - npcs: - extract: dnd/npcs - normalize: dnd/npcs - - id: grounded-events - references: - npcs: - artifact: - step: identify-npcs - lane: npcs - artifacts: - spells: - extract: dnd/spells - normalize: dnd/spells - combat: - extract: dnd/combat-turns - normalize: dnd/combat-turns - output: json -``` - -Existing scalar reference values continue to represent external file paths. -The structured `artifact` form identifies accepted normalized output from one -earlier step and lane. Generated artifact bindings are allowed at step scope or -at an individual module target; they are not inferred from module keys, lane -names, slot names, or domain knowledge. - -A step-scoped reference applies automatically to every selected target in that -step that declares the slot. In the example, one `npcs` binding reaches the -spell extractor plus the combat extractor and normalizer. A target-local -binding is used when only one module should consume the artifact. - -Pipeline-level external references remain defaults. Step-local external -references override pipeline-level external defaults, and target-local -external references retain their existing precedence. A generated reference -and an external reference may not resolve to the same effective target slot; -configuration or a runtime override that creates that conflict is invalid. -Likewise, a step-scoped and target-local generated binding cannot both target -the same effective slot. - -Each effective target slot accepts at most one producer. One producer may fan -out to multiple compatible slots in a later step. Aggregating several producer -artifacts into one slot is outside this scope. - -Reference-slot specs gain optional generated-artifact compatibility metadata. -A generated binding is allowed only when the consumer slot declares the -producer's artifact kind; the producer's registered codec supplies the exact -schema identity and media type used for the handoff. The D&D `npcs` consumer -slots declare the normalized NPC-list artifact kind. Existing external-file -slots and bindings retain their current behavior and do not acquire an artifact -kind merely because their bytes happen to decode as one. - -### Resolution and preparation - -Resolution validates the complete ordered structure before source processing. -It must reject duplicate identities, missing producers, same-step or forward -references, undeclared slots, reference conflicts, and incompatible artifact -kind, schema, media type, or cardinality constraints that are statically -discoverable. Size is checked when canonical producer bytes exist at handoff. -Ordered steps make cycles structurally impossible; resolution must not -introduce a general graph scheduler to rediscover their order. - -The resolved pipeline and its digest include step order, step IDs, lane -membership, generated-reference topology, producer identity, consumer targets, -and existing module and validator policy. Cloning, redaction, canonical JSON, -debug summaries, and manifests preserve the same structure without reference -content or secrets. - -All modules and validators are still selected, option-validated, and -constructed before source parsing. Generated content cannot be supplied during -construction because it does not exist yet. The framework therefore augments -the existing operation-request `References` at the step boundary. Consumers -that currently assume an NPC registry is construction-only must accept the -generated registry from their operation request without deferring general -module construction until after upstream work. - -Only validation that inherently depends on generated bytes may occur at the -handoff. A handoff validation failure is a contextual framework error and fails -the run before any consumer in that step begins. - -### Generated artifact handoff - -Only accepted normalized output may cross a step boundary. Raw extraction -responses, rejected artifacts, merge intermediates, and validator diagnostics -cannot be bound as references. - -The framework serializes the producer through its registered canonical artifact -codec and constructs one immutable reference item containing: - -- the declared target slot; -- canonical artifact bytes and media type; -- artifact kind and schema ID, name, version, and schema digest; -- canonical content digest and size; and -- producer pipeline, step, lane, and module provenance. - -A generated binding requires exactly one accepted normalized artifact from its -producer lane. No artifact is a missing dependency, while more than one is a -cardinality error; a typed collection such as an NPC list remains one artifact. -Combining several normalized outputs into one reference is aggregation and is -outside this scope. - -The existing slot contract remains authoritative for accepted media types, -maximum size, and cardinality. Generated content is cloned at ownership -boundaries and never exposed through a filesystem path. Manifests and debug -summaries record identities and bounded provenance, not artifact content. - -Configuring a generated binding makes that dependency required even when the -consumer module declares the underlying slot optional. An accepted artifact -whose domain collection is empty is still a valid artifact and may be handed -off. If the producer has no accepted normalized artifact, the entire run fails -with a deterministic dependency error and no later step begins. - -### Checkpoint reuse and selective recomputation - -Generated references participate in downstream checkpoint dependencies by -artifact kind, complete schema identity, media type, and canonical content -digest. The pipeline digest protects topology; stage dependency fingerprints -protect the exact upstream artifact consumed. The runner must never combine a -new or changed producer with stale dependent output. - -Ordinary resume may progressively decode compatible producer and consumer stage -checkpoints through the registered codec. Selective recomputation may hydrate a -required unselected producer directly from its accepted normalized artifact; -its extract and merge state are not prerequisites. Missing, rejected, corrupt, -incompatible, or changed accepted state stops the run before dependent -execution rather than implicitly rerunning the producer. Independent work -remains reusable. - -The operator control `--recompute-step ` has these semantics: - -- it requires checkpoint recording and `--resume`; -- the selected step and all transitive dependents execute rather than reuse - their checkpoints; -- valid required predecessors and unrelated work remain reusable; -- the recompute selection affects loader decisions, not the persistent - checkpoint identity of otherwise identical work; and -- the command fails before dependent execution if a required predecessor has - no reusable accepted artifact. - -Existing `--only` behavior remains unchanged for implicit single-step -pipelines. Combining `--only` with explicit multi-step pipelines is outside -this scope and should be rejected with actionable guidance rather than given -implicit dependency-expansion semantics. - -Checkpoint events, manifests, and diagnostics distinguish executed, reused, -forced-recomputed, and dependency-invalidated work. Invalidation reasons are -bounded, deterministic, and free of reference content, local paths, or secrets. -Old checkpoint state need not be migrated; it must produce a safe, explicit -cold miss rather than an error or unsafe reuse. - -### Failure, cancellation, and concurrency - -The existing run-wide worker and provider-call limits apply across every step. -Workers may be reused between steps, but concurrency cannot cross a step -barrier. A framework error cancels started work using the existing bounded -drain behavior and prevents later steps and output encoding. Rejections remain -recorded outcomes, but failure to produce a normalized artifact required by a -generated binding escalates to the run-level dependency error described above. - -The failed manifest retains completed upstream outcomes, step and lane -provenance, rejections, checkpoint events, and the dependency failure without -embedding generated artifact content. - -## D&D Proving Workflow - -The production acceptance workflow has two explicit steps: - -1. `identify-npcs` runs the NPC lane through normalization and its complete - validator policy. -2. `grounded-events` receives the canonical NPC artifact in its step-scoped - `npcs` reference and runs spell and combat-turn lanes. The binding reaches - spell extraction, combat-turn extraction, and combat-turn normalization. - -Spell and combat-turn lanes may execute concurrently after the handoff. NPC -content may ground names and identities but cannot establish a spell cast or -combat event; source units remain the only event evidence. - -The maintained workflow uses one ordered-pipeline example instead of a manual -two-run NPC-to-spell or NPC-to-combat handoff. Existing module keys, artifact -contracts, reference slot names, prompt IDs, and D&D evidence policy remain -unchanged. - -## Included Work - -- Configuration parsing, validation, cloning, defaults, redaction, and - documentation for explicit steps and structured generated references. -- Domain-neutral resolved step, dependency, producer, and consumer identities. -- Generated-artifact compatibility metadata on reference-slot contracts, - including D&D NPC-list declarations for every `npcs` consumer. -- Step-aware preparation metadata and runner orchestration. -- Canonical codec handoff into existing reference request contracts. -- Required-dependency failure and bounded provenance behavior. -- Dependency-aware checkpoint reuse, invalidation, events, and selective step - recomputation. -- D&D NPC-first production composition for spell and combat-turn consumers. -- Refactoring the affected D&D consumers so generated NPC references are - available at operation time while retaining early static construction. -- Maintained examples and current architecture, configuration, CLI, operations, - internal, integration, and testing documentation. -- An ADR recording the bounded ordered-step extension to the fixed pipeline - architecture and its explicit rejection of a general DAG. - -## Explicitly Out Of Scope - -- D&D item extraction or any other new artifact lane. -- Cross-artifact NPC ID fields or artifact-schema migration machinery. -- Arbitrary DAGs, conditional branches, loops, joins, dynamic step creation, or - module-controlled scheduling. -- Multiple source inputs, per-step input adapters, per-step chunk plans, or - per-step output encoders. -- Aggregating multiple generated artifacts into one reference slot. -- Optional or best-effort generated dependencies; a configured dependency is - required in this scope. -- Prior-run or cross-pipeline generated references. -- `--only` dependency closure for explicit multi-step pipelines. -- Cross-lane reconciliation or domain concepts in the generic framework. -- Live-provider tests or model-quality changes to D&D prompts. - -## Acceptance Criteria - -The scope is complete when: - -- all existing single-step configurations retain their current behavior; -- explicit step order and dependency topology resolve deterministically and - affect pipeline identity; -- invalid producer, consumer, conflict, ordering, type, schema, media, and - cardinality configurations fail before source processing when statically - discoverable, while content-size violations fail at handoff; -- no consumer step begins before all required generated artifacts are accepted, - canonicalized, and validated for its target slots; -- one producer artifact fans out safely to every compatible target selected by - a step-scoped binding; -- missing required producer output fails the complete run before dependent work; -- changing NPC output invalidates spell and combat-turn checkpoints while - leaving compatible independent work reusable; -- selective step recomputation executes exactly the selected dependency closure - and reports why work was executed, reused, or invalidated; -- the D&D ordered workflow supplies NPC content to spell extraction, combat-turn - extraction, and combat-turn normalization without treating it as evidence; -- completion timing cannot change public ordering, failure selection, or - dependency behavior; -- output, manifests, checkpoints, and debug artifacts contain the required - identities and provenance without leaking generated reference content; and -- repository-wide tests, vet, build, maintained-example checks, and - documentation validation pass. - -## Testing Strategy - -Tests should protect behavior and invariants rather than the implementation's -internal scheduler shape. - -- Configuration contract tests own legacy shorthand, explicit step parsing, - source-form discrimination, conflicts, and redaction. -- Resolution tests own ordering, global lane uniqueness, dependency validation, - slot compatibility, fan-out, cloning, canonical JSON, and digest changes. -- Runner tests own step barriers, within-step bounded concurrency, stable - ordering, cancellation, required-producer failure, and immutable handoff. -- Checkpoint tests own producer decoding, exact dependency matching, transitive - invalidation, forced recomputation, cold misses, and bounded decisions. -- One CLI contract test should cover the recompute control and its invalid - combinations. -- One D&D integration test with offline fake LLM responses should prove the - complete NPC-to-spell-and-combat handoff, including combat normalization. -- Maintained configuration examples should be parsed and resolved through the - production catalog. - -Do not add scheduler choreography tests, exact goroutine-count assertions, -complete manifest snapshots, exact diagnostic strings, or duplicated tests for -every invalid configuration at every layer. No test may require credentials or -a live model provider. - -## Open Questions - -None required to define this scope. Exact internal type names and implementation -decomposition are intentionally not feature-policy decisions.