From 3cf2ac577fe0f96eb60c16abdb4738609952f5fe Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Fri, 3 Jul 2026 10:19:57 -0500 Subject: [PATCH] Add a staged implementation plan for the pipeline stages --- ...ipeline-stages-chunking-merge-normalize.md | 50 +- docs/roadmap/implementation.md | 1254 +++++++++++------ 2 files changed, 805 insertions(+), 499 deletions(-) diff --git a/docs/roadmap/3-pipeline-stages-chunking-merge-normalize.md b/docs/roadmap/3-pipeline-stages-chunking-merge-normalize.md index 55becfa..78c05f7 100644 --- a/docs/roadmap/3-pipeline-stages-chunking-merge-normalize.md +++ b/docs/roadmap/3-pipeline-stages-chunking-merge-normalize.md @@ -81,17 +81,11 @@ The repository should also contain a resolved pipeline model that represents: The runner should orchestrate fake implementations through chunk, extract, merge, normalize, and approval/validation behavior in tests. -The checkpoint should include one fixture-driven integration test that starts -from fixture input bytes and ends at encoded output bytes. The fixture should -use an in-memory pipeline profile with fake input adapter, deterministic -chunker, trivial extractor, fake structured LLM client, generic merger, no-op -normalizer, and fake or minimal JSON output encoder. This is a contract -exercise, not a useful user-facing workflow. - -The walking skeleton should live in `internal/framework/pipeline`, with -fixtures under that package's `testdata/`. If the CLI has an extract command by -then, the same fixture may also be exercised through CLI wiring; if not, CLI -coverage remains deferred. +The checkpoint should include a fixture-driven walking skeleton that starts from +fixture input bytes and ends at encoded output bytes. The walking skeleton should +exercise the stage contracts, resolved pipeline model, module metadata, +capability validation, defaults, lane selection, and fake LLM client wiring. It +is contract coverage, not useful user-facing behavior. ## Design Intent @@ -99,7 +93,7 @@ Chunking is a core application concern because many source documents, especially transcripts, will be too large for a single LLM extraction pass. Chunk processing may be serial or parallel depending on extractor needs. The -architecture should support both, but the first implementation can execute +architecture should support both, while checkpoint 3 may execute deterministically in series until a later checkpoint introduces concurrency. Merge and normalize are separate stages: @@ -120,8 +114,9 @@ The architecture should leave room for extractor-level processing modes: - serial chunk processing; - parallel chunk processing. -The first implementation may model these modes without implementing parallel -execution. It should not bake in a single-pass assumption. +Checkpoint 3 may execute chunks serially for deterministic behavior. +The contracts should not bake in a single-pass assumption or prevent later +parallel execution. ## Generic Merge Behavior @@ -143,29 +138,16 @@ Domain-specific normalizers may later: - enforce chronological or source-reference consistency; - attach normalization warnings. -## Walking Skeleton Fixture +## Walking Skeleton The fixture-driven skeleton should prove the staged architecture continuously as -new contracts are added. It should be deliberately small: +new contracts are added. It should remain deliberately small and use fake modules +only. It should validate module keys and flat capability requirements before +execution, using registry metadata rather than constructing modules. Capability +values should remain simple strings. -- a fixture source document or raw source input with two or three source units; -- a fake input adapter that parses the fixture into `SourceDocument`; -- a deterministic chunker that produces multiple chunks; -- a trivial extractor that calls the fake structured LLM client and emits one - artifact candidate per chunk; -- a generic append-in-chunk-order merger; -- a no-op normalizer; -- a fake or minimal JSON output encoder; -- an expected output fixture checked byte-for-byte or by unmarshaled structural - equality. - -The fake LLM client should be part of the test setup so the contract is -exercised without introducing provider code, prompt assets, or response schema -assets. - -The walking skeleton should validate module keys and flat capability -requirements before execution. This should use registry metadata rather than -constructing modules. Keep capability values as simple strings. +Implementation staging for the walking skeleton belongs in +[`implementation.md`](implementation.md). ## Done Criteria diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md index b4e32cb..5038306 100644 --- a/docs/roadmap/implementation.md +++ b/docs/roadmap/implementation.md @@ -1,15 +1,15 @@ -# Implementation Plan: Checkpoint 2 Framework Composition +# Implementation Plan: Checkpoint 3 Pipeline Stages ## Status This is a staged implementation plan for -[`2-framework-composition.md`](2-framework-composition.md). It is intended for -an LLM coding agent to follow stage by stage. +[`3-pipeline-stages-chunking-merge-normalize.md`](3-pipeline-stages-chunking-merge-normalize.md). +It is intended for an LLM coding agent to follow stage by stage. -This plan implements only checkpoint 2. Do not implement real input modules, -real extract modules, source chunking, LLM provider clients, prompt assets, -response schema registries, diagnostics run directories, production config -loading, or D&D artifact schemas in this pass. +This plan implements only checkpoint 3. Do not add real Seriatim parsing, real +D&D extraction, production config file loading, CLI command behavior, LLM +provider clients, prompt assets, response schema assets, diagnostics run +directories, or durable output writing. ## Policy Context @@ -20,489 +20,576 @@ Follow: Required boundaries: -- framework packages must stay source-agnostic and domain-agnostic; -- input adapter registry code must not import concrete input module packages; -- extractor registry and runner code must not import concrete extract module - packages; -- runner code should operate on `SourceDocument`, not transcript-specific - structures; -- validators should be independently testable and composable; -- planned behavior outside this checkpoint must remain in roadmap docs. +- framework code must stay source-agnostic and domain-agnostic; +- concrete business logic remains under `internal/modules//...`, but this + checkpoint should use fake modules only; +- framework plumbing should remain consolidated under `internal/framework/pipeline` + unless a boundary proves itself; +- capabilities are flat strings, not a type system; +- pipeline profiles are fixed-shape templates, not arbitrary step lists; +- no structural module wiring through ad hoc CLI flags. ## Global Implementation Decisions -- Add no third-party dependencies in checkpoint 2. -- Do not change public core source/artifact/contract types unless a stage cannot - compile without a small compatibility adjustment. -- Use constructor-based registries. Constructors can close over future - dependencies without changing registry APIs. -- Registries normalize keys with `strings.TrimSpace`. -- Registries reject empty keys, duplicate keys, nil constructors, nil built - instances, and key/name mismatches. -- The minimal runner receives an already parsed `*source.SourceDocument`. -- The minimal runner does not use input adapters yet; input adapter registry is - tested directly. -- The runner assigns global candidate indices in extractor execution order. -- If a candidate supplies non-empty extractor metadata that conflicts with its - owning extractor, the runner returns an error. -- If a candidate leaves extractor metadata empty, the runner fills - `ExtractorKey`, `ArtifactType`, and `SchemaVersion` from the owning extractor. -- Validator chains run in the order returned by `Extractor.Validators()`. -- A validator result must use the same `ValidatorName` as `Validator.Name()`. -- Each validator must return exactly one decision for each currently eligible - candidate. -- If an extractor has no validators, its candidates are approved. -- On setup, extraction, or validation errors, return the current partial - `RunOutput` with a wrapped error. +- Add no third-party dependencies. +- Keep new checkpoint 3 framework code in `internal/framework/pipeline`. +- Keep shared request/result contracts in `internal/framework/contracts` when + stage modules will implement or consume them. +- Do not create new framework packages for chunking, merge, normalize, output, + warnings, response schemas, or structured output. +- Use in-memory pipeline profile structs only. Do not implement YAML/TOML + parsing in this checkpoint. +- Keep one shared chunker per resolved pipeline. Per-lane chunker overrides are + deferred. +- Execute chunks serially in checkpoint 3. Contracts must not prevent later + parallel execution. +- Sort artifact lane IDs during resolution for deterministic output and digest + behavior. +- Use `sha256:` for resolved pipeline digests. +- Preserve existing validator decision-cardinality behavior. - Run `gofmt` on all touched Go files before validation. -## Stage 1: Input Adapter Registry +## Stage 1: Stage Contracts And Source Chunks ### Goal -Add a constructor-based registry for `contracts.InputAdapter`. - -### Files To Add - -- `internal/framework/pipeline/input_registry.go` -- `internal/framework/pipeline/input_registry_test.go` - -### Required API - -Extend package `pipeline`. - -Define: - -```go -type InputAdapterConstructor func() (contracts.InputAdapter, error) - -type InputAdapterRegistry struct { - // unexported fields -} - -func NewInputAdapterRegistry() *InputAdapterRegistry -func (r *InputAdapterRegistry) Register(key string, constructor InputAdapterConstructor) error -func (r *InputAdapterRegistry) Build(key string) (contracts.InputAdapter, error) -func (r *InputAdapterRegistry) RegisteredKeys() []string -``` - -### Required Behavior - -`Register` must: - -- return an error if the registry is nil; -- trim `key`; -- reject empty keys; -- reject nil constructors; -- reject duplicate keys; -- store constructors by normalized key. - -`Build` must: - -- return an error if the registry is nil; -- trim `key`; -- reject empty keys; -- return a clear unknown-key error for unregistered keys; -- call the constructor; -- wrap constructor errors with key context; -- reject nil adapters; -- reject adapters whose `Key()` does not match the normalized key. - -`RegisteredKeys` must: - -- return nil for a nil registry; -- return registered keys in sorted order; -- return a copy that callers cannot mutate to affect registry state. - -### Required Tests - -Cover: - -- successful registration and build; -- key trimming; -- empty key rejection; -- duplicate key rejection; -- nil constructor rejection; -- unknown key build error; -- constructor error wrapping; -- nil adapter rejection; -- adapter key mismatch rejection; -- sorted `RegisteredKeys`; -- nil registry behavior. - -Use fake adapters only. Do not add concrete input module packages. - -### Validation - -Run: - -```sh -gofmt -w internal/framework/pipeline -go test ./internal/framework/pipeline -go test ./... -``` - -## Stage 2: Extractor Registry - -### Goal - -Add a constructor-based registry for `contracts.Extractor`. - -### Files To Add - -- `internal/framework/pipeline/extractor_registry.go` -- `internal/framework/pipeline/extractor_registry_test.go` - -### Required API - -Extend package `pipeline`. - -Define: - -```go -type ExtractorConstructor func() (contracts.Extractor, error) - -type ExtractorRegistry struct { - // unexported fields -} - -func NewExtractorRegistry() *ExtractorRegistry -func (r *ExtractorRegistry) Register(key string, constructor ExtractorConstructor) error -func (r *ExtractorRegistry) Build(key string) (contracts.Extractor, error) -func (r *ExtractorRegistry) RegisteredKeys() []string -``` - -### Required Behavior - -Mirror input adapter registry behavior, but for `contracts.Extractor`. - -`Build` must reject extractors whose `Key()` does not match the normalized key. -It should not validate artifact type, schema version, or validator chain. Those -are extractor/runtime concerns. - -### Required Tests - -Cover the same cases as the input adapter registry: - -- successful registration and build; -- key trimming; -- empty key rejection; -- duplicate key rejection; -- nil constructor rejection; -- unknown key build error; -- constructor error wrapping; -- nil extractor rejection; -- extractor key mismatch rejection; -- sorted `RegisteredKeys`; -- nil registry behavior. - -Use fake extractors only. Do not add concrete extract module packages. - -### Validation - -Run: - -```sh -gofmt -w internal/framework/pipeline -go test ./internal/framework/pipeline -go test ./... -``` - -## Stage 3: Validator Runtime Helpers - -### Goal - -Add shared validator decision helpers and decision-cardinality enforcement. - -### Files To Add - -- `internal/framework/validate/validate.go` -- `internal/framework/validate/validate_test.go` - -### Required API - -Create package `validate`. - -Define reason constants: - -```go -const ( - ReasonApproved = "approved" -) -``` - -Define helpers: - -```go -func Approved(candidateIndex int) contracts.ValidationDecision -func Rejected(candidateIndex int, reasonCode string, message string) contracts.ValidationDecision -func EnforceDecisionCardinality(candidates []artifacts.ArtifactCandidate, decisions []contracts.ValidationDecision) error -``` - -### Required Behavior - -`Approved` returns an approved decision with: - -- `CandidateIndex` set to the input; -- `Approved` set to true; -- `ReasonCode` set to `ReasonApproved`; -- `Message` set to `approved`. - -`Rejected` returns a rejected decision with: - -- `CandidateIndex` set to the input; -- `Approved` set to false; -- `ReasonCode` set to the trimmed reason code; -- `Message` set to the trimmed message. - -`EnforceDecisionCardinality` must: - -- require exactly one decision per candidate index; -- reject decisions for unknown candidate indices; -- reject duplicate decisions for the same candidate index; -- reject missing decisions; -- work with candidate indices, not slice positions; -- allow an empty candidate slice only when decisions are also empty. - -Artifact candidate indices are expected to be unique by the time validators run. -If the candidate slice itself contains duplicate indices, return an error. - -### Required Tests - -Cover: - -- approved helper shape; -- rejected helper trims reason/message; -- cardinality success with non-zero candidate indices; -- empty candidates and empty decisions success; -- unknown decision index error; -- duplicate decision index error; -- missing decision index error; -- duplicate candidate index error. - -### Validation - -Run: - -```sh -gofmt -w internal/framework/validate -go test ./internal/framework/validate -go test ./... -``` - -## Stage 4: Minimal Runner Types And Constructor - -### Goal - -Add pipeline runner types and constructor without implementing the full run loop -yet. - -### Files To Add - -- `internal/framework/pipeline/runner.go` -- `internal/framework/pipeline/runner_test.go` - -### Required API - -Create or extend package `pipeline`. - -Define: - -```go -type ExtractorFactory interface { - Build(key string) (contracts.Extractor, error) -} - -type Runner struct { - // unexported fields -} - -func New(extractors ExtractorFactory) *Runner -``` - -Define input/output structs: - -```go -type RunInput struct { - Source *source.SourceDocument - ExtractorKeys []string - LLMClient contracts.StructuredLLMClient - Metadata map[string]any -} - -type RunOutput struct { - Approved []artifacts.Artifact `json:"approved,omitempty"` - Rejected []artifacts.RejectedArtifact `json:"rejected,omitempty"` - Warnings []contracts.Warning `json:"warnings,omitempty"` -} -``` - -Define package errors as ordinary returned errors. Do not introduce custom error -types in this checkpoint unless tests require behavior that ordinary errors -cannot express. - -### Required Behavior - -At this stage, `New` should only store the extractor factory. - -If you add `Run` in this stage as a stub, it must return an explicit -`runner is not implemented` error and must be completed in stage 5. Prefer -adding the real `Run` method only in stage 5. - -### Required Tests - -Cover: - -- `New` accepts a fake extractor factory and returns a non-nil runner; -- `RunInput` and `RunOutput` can be constructed with the expected fields. - -Do not add registry imports to runner tests unless needed for interface -assertions. Runner should depend only on framework contracts and core packages. - -### Validation - -Run: - -```sh -gofmt -w internal/framework/pipeline -go test ./internal/framework/pipeline -go test ./... -``` - -## Stage 5: Minimal Runner Execution - -### Goal - -Implement the minimal runner flow from `SourceDocument` to approved/rejected -artifacts. +Define the source chunk model and contracts for chunk, merge, normalize, and +output stages. ### Files To Update -- `internal/framework/pipeline/runner.go` -- `internal/framework/pipeline/runner_test.go` +- `internal/framework/contracts/contracts.go` +- `internal/framework/contracts/contracts_test.go` +- `internal/framework/contracts/composition_test.go` ### Required API Add: ```go -func (r *Runner) Run(ctx context.Context, input RunInput) (RunOutput, error) +type SourceChunk struct { + ID string `json:"id"` + SourceID string `json:"source_id"` + Index int `json:"index"` + Units []source.SourceUnit `json:"units"` + Metadata map[string]any `json:"metadata,omitempty"` +} + +type ChunkRequest struct { + Source *source.SourceDocument `json:"-"` + Metadata map[string]any `json:"metadata,omitempty"` +} + +type ChunkResult struct { + Chunks []SourceChunk `json:"chunks"` + Warnings []Warning `json:"warnings,omitempty"` +} + +type Chunker interface { + Key() string + Chunk(ctx context.Context, req ChunkRequest) (ChunkResult, error) +} +``` + +Extend `ExtractionRequest`: + +```go +type ExtractionRequest struct { + Source *source.SourceDocument `json:"-"` + Chunk *SourceChunk `json:"chunk,omitempty"` + AmbientContext map[string]any `json:"ambient_context,omitempty"` + LLMClient StructuredLLMClient `json:"-"` + Metadata map[string]any `json:"metadata,omitempty"` +} +``` + +Add: + +```go +type ChunkArtifacts struct { + Chunk SourceChunk `json:"chunk"` + Candidates []artifacts.ArtifactCandidate `json:"candidates"` +} + +type MergeRequest struct { + Source *source.SourceDocument `json:"-"` + LaneID string `json:"lane_id"` + ChunkArtifacts []ChunkArtifacts `json:"chunk_artifacts"` + Metadata map[string]any `json:"metadata,omitempty"` +} + +type MergeResult struct { + Candidates []artifacts.ArtifactCandidate `json:"candidates"` + Warnings []Warning `json:"warnings,omitempty"` +} + +type Merger interface { + Key() string + Merge(ctx context.Context, req MergeRequest) (MergeResult, error) +} + +type NormalizeRequest struct { + Source *source.SourceDocument `json:"-"` + LaneID string `json:"lane_id"` + Candidates []artifacts.ArtifactCandidate `json:"candidates"` + Metadata map[string]any `json:"metadata,omitempty"` +} + +type NormalizeResult struct { + Candidates []artifacts.ArtifactCandidate `json:"candidates"` + Warnings []Warning `json:"warnings,omitempty"` +} + +type Normalizer interface { + Key() string + Normalize(ctx context.Context, req NormalizeRequest) (NormalizeResult, error) +} + +type OutputRequest struct { + Manifest artifacts.RunManifest `json:"manifest"` + Approved []artifacts.Artifact `json:"approved,omitempty"` + Rejected []artifacts.RejectedArtifact `json:"rejected,omitempty"` + Warnings []Warning `json:"warnings,omitempty"` + Metadata map[string]any `json:"metadata,omitempty"` +} + +type OutputResult struct { + Bytes []byte `json:"-"` + ContentType string `json:"content_type,omitempty"` + Warnings []Warning `json:"warnings,omitempty"` +} + +type OutputEncoder interface { + Key() string + Encode(ctx context.Context, req OutputRequest) (OutputResult, error) +} +``` + +If `gofmt` aligns fields differently, keep the formatted output. + +### Required Behavior + +- Existing fake extractor tests must still compile with `ExtractionRequest{Source: doc}`. +- `SourceChunk` is a framework contract type, not a core source type. +- `SourceChunk.Units` should be copied by implementations where mutation risk + exists; this stage only defines the contract. + +### Required Tests + +Update fake implementations in existing contract tests to prove: + +- fake chunker satisfies `contracts.Chunker`; +- fake merger satisfies `contracts.Merger`; +- fake normalizer satisfies `contracts.Normalizer`; +- fake output encoder satisfies `contracts.OutputEncoder`; +- fake extractor can receive a `SourceChunk` and ambient context. + +### Validation + +Run: + +```sh +gofmt -w internal/framework/contracts +go test ./internal/framework/contracts +go test ./... +``` + +## Stage 2: Module Metadata And Stage Registries + +### Goal + +Teach pipeline registries about module metadata and add registries for chunk, +merge, normalize, configured validators, and output modules. + +### Files To Update + +- `internal/framework/pipeline/input_registry.go` +- `internal/framework/pipeline/input_registry_test.go` +- `internal/framework/pipeline/extractor_registry.go` +- `internal/framework/pipeline/extractor_registry_test.go` + +### Files To Add + +- `internal/framework/pipeline/module.go` +- `internal/framework/pipeline/chunker_registry.go` +- `internal/framework/pipeline/chunker_registry_test.go` +- `internal/framework/pipeline/merger_registry.go` +- `internal/framework/pipeline/merger_registry_test.go` +- `internal/framework/pipeline/normalizer_registry.go` +- `internal/framework/pipeline/normalizer_registry_test.go` +- `internal/framework/pipeline/validator_registry.go` +- `internal/framework/pipeline/validator_registry_test.go` +- `internal/framework/pipeline/output_registry.go` +- `internal/framework/pipeline/output_registry_test.go` + +### Required API + +Add: + +```go +type ModuleStage string + +const ( + StageInput ModuleStage = "input" + StageChunk ModuleStage = "chunk" + StageExtract ModuleStage = "extract" + StageMerge ModuleStage = "merge" + StageNormalize ModuleStage = "normalize" + StageValidate ModuleStage = "validate" + StageOutput ModuleStage = "output" +) + +type ModuleSpec struct { + Key string + Stage ModuleStage + Provides []string + Requires []string +} +``` + +Add metadata registration methods to existing registries: + +```go +func (r *InputAdapterRegistry) RegisterWithSpec(spec ModuleSpec, constructor InputAdapterConstructor) error +func (r *InputAdapterRegistry) Spec(key string) (ModuleSpec, bool) + +func (r *ExtractorRegistry) RegisterWithSpec(spec ModuleSpec, constructor ExtractorConstructor) error +func (r *ExtractorRegistry) Spec(key string) (ModuleSpec, bool) +``` + +Keep existing `Register` methods. They should call `RegisterWithSpec` with a +default spec using the provided key and correct stage. + +Add typed registries for other stages: + +```go +type ChunkerConstructor func() (contracts.Chunker, error) +type ChunkerRegistry struct { /* unexported fields */ } +func NewChunkerRegistry() *ChunkerRegistry +func (r *ChunkerRegistry) Register(key string, constructor ChunkerConstructor) error +func (r *ChunkerRegistry) RegisterWithSpec(spec ModuleSpec, constructor ChunkerConstructor) error +func (r *ChunkerRegistry) Build(key string) (contracts.Chunker, error) +func (r *ChunkerRegistry) Spec(key string) (ModuleSpec, bool) +func (r *ChunkerRegistry) RegisteredKeys() []string +``` + +Repeat the same pattern for `MergerRegistry`, `NormalizerRegistry`, +`ValidatorRegistry`, and `OutputEncoderRegistry`. + +`ValidatorRegistry` should use: + +```go +type ValidatorConstructor func() (contracts.Validator, error) +``` + +Because `contracts.Validator` uses `Name()` rather than `Key()`, +`ValidatorRegistry.Build` must require `validator.Name()` to match the +registered key. + +### Required Behavior + +- Registry keys are trimmed. +- Specs are normalized by trimming `Key`, `Provides`, and `Requires`. +- Empty capability strings are ignored. +- `Provides` and `Requires` are deduplicated and sorted for deterministic + behavior. +- `RegisterWithSpec` rejects an empty key, wrong stage, nil constructor, and + duplicate key. +- `Spec` returns false for unknown keys and nil registries. +- `Build` rejects unknown keys, constructor errors, nil modules, and key + mismatches. +- Existing registry tests continue to pass. + +### Required Tests + +For each new registry, cover the same behavior as existing input/extractor +registries: + +- successful registration and build; +- metadata registration and lookup; +- capability normalization; +- wrong stage rejection; +- key trimming; +- empty key rejection; +- duplicate key rejection; +- nil constructor rejection; +- unknown key build error; +- constructor error wrapping; +- nil module rejection; +- key mismatch rejection; +- sorted `RegisteredKeys`; +- nil registry behavior. + +For `ValidatorRegistry`, key mismatch checks should compare the registry key to +`Validator.Name()`. + +For existing input/extractor registries, add tests for metadata registration, +default specs from `Register`, capability normalization, and wrong stage +rejection. + +### Validation + +Run: + +```sh +gofmt -w internal/framework/pipeline +go test ./internal/framework/pipeline +go test ./... +``` + +## Stage 3: Resolved Pipeline Model + +### Goal + +Add fixed-shape pipeline profile structs, default resolution, lane selection, +capability validation, and resolved pipeline digesting. + +### Files To Add + +- `internal/framework/pipeline/profile.go` +- `internal/framework/pipeline/profile_test.go` + +### Files To Update + +- `internal/core/artifacts/artifacts.go` +- `internal/core/artifacts/artifacts_test.go` + +### Required API + +Update `artifacts.RunManifest` to include: + +```go +PipelineID string `json:"pipeline_id,omitempty"` +PipelineDigest string `json:"pipeline_digest,omitempty"` +``` + +Add lane-level manifest details rather than relying on the existing shared +`Extractors`, `Merger`, or `Normalizer` fields for multi-lane runs: + +```go +type ArtifactLaneManifest struct { + ID string `json:"id"` + Extractor string `json:"extractor"` + Merger string `json:"merger"` + Normalizer string `json:"normalizer"` + Validators []string `json:"validators,omitempty"` +} +``` + +Then add: + +```go +ArtifactLanes []ArtifactLaneManifest `json:"artifact_lanes,omitempty"` +``` + +Do not remove the existing singular fields in this checkpoint unless doing so is +required to make the package compile. Treat `ArtifactLanes` as the authoritative +manifest representation for checkpoint 3's multi-lane pipeline model. + +Add pipeline model types: + +```go +type ModuleBinding struct { + Module string `json:"module"` + LLMProfile string `json:"llm_profile,omitempty"` + Options map[string]any `json:"options,omitempty"` +} + +type ArtifactLaneProfile struct { + Extract ModuleBinding `json:"extract"` + Merge ModuleBinding `json:"merge,omitempty"` + Normalize ModuleBinding `json:"normalize,omitempty"` + Validators []ModuleBinding `json:"validators,omitempty"` +} + +type PipelineProfile struct { + ID string `json:"id"` + Input ModuleBinding `json:"input"` + Chunk ModuleBinding `json:"chunk,omitempty"` + Artifacts map[string]ArtifactLaneProfile `json:"artifacts"` + Output ModuleBinding `json:"output,omitempty"` +} + +type ResolveOptions struct { + Only []string +} + +type ResolvedArtifactLane struct { + ID string + Extract ModuleBinding + Merge ModuleBinding + Normalize ModuleBinding + Validators []ModuleBinding +} + +type ResolvedPipeline struct { + ID string + Digest string + Input ModuleBinding + Chunk ModuleBinding + ArtifactLanes []ResolvedArtifactLane + Output ModuleBinding +} +``` + +Add defaults: + +```go +const ( + DefaultChunkModule = "generic" + DefaultMergeModule = "appendorder" + DefaultNormalizeModule = "noop" + DefaultOutputModule = "json" + DefaultLLMProfile = "default" +) +``` + +Add: + +```go +type ModuleCatalog struct { + Inputs *InputAdapterRegistry + Chunkers *ChunkerRegistry + Extractors *ExtractorRegistry + Mergers *MergerRegistry + Normalizers *NormalizerRegistry + Validators *ValidatorRegistry + Outputs *OutputEncoderRegistry +} + +func Binding(module string) ModuleBinding +func ResolvePipeline(profile PipelineProfile, options ResolveOptions, catalog ModuleCatalog) (ResolvedPipeline, error) ``` ### Required Behavior -`Run` must: +`ResolvePipeline` must: -1. reject nil runner or nil extractor factory; -2. validate `input.Source` with `source.ValidateDocument`; -3. reject an empty `ExtractorKeys` list; -4. iterate extractor keys in input order; -5. build each extractor through the factory; -6. call `Extractor.Extract` with the source, LLM client, and metadata; -7. append extractor warnings to `RunOutput.Warnings`; -8. normalize candidate metadata for each returned candidate; -9. run validators in `Extractor.Validators()` order; -10. approve candidates that survive all validators; -11. reject candidates denied by validators; -12. return approved/rejected artifacts in deterministic order. +- trim `PipelineProfile.ID`; +- reject empty pipeline IDs; +- require non-empty input module; +- require at least one artifact lane before and after `Only` filtering; +- apply defaults for missing chunk, merge, normalize, output, and binding + `llm_profile` values; +- trim all module keys; +- trim lane names in `ResolveOptions.Only`; +- reject empty or unknown lane names in `ResolveOptions.Only`; +- deduplicate repeated `Only` lane names; +- sort selected lane IDs alphabetically; +- reject unknown module keys using registry metadata, not constructors; +- validate flat capability requirements before execution; +- compute `ResolvedPipeline.Digest` from canonical JSON of the resolved + pipeline excluding the digest field itself. -Artifact candidate normalization must: +Capability validation algorithm: -- assign `ArtifactCandidate.Index` using a global monotonically increasing counter; -- fill empty `ExtractorKey`, `ArtifactType`, and `SchemaVersion` from the - owning extractor; -- return an error if a candidate's non-empty `ExtractorKey` differs from - `Extractor.Key()`; -- return an error if a candidate's non-empty `ArtifactType` differs from - `Extractor.ArtifactType()`; -- return an error if a candidate's non-empty `SchemaVersion` differs from - `Extractor.SchemaVersion()`. +1. Start with capabilities provided by the input module. +2. Validate chunker requirements against current capabilities, then add chunker + provided capabilities. +3. For each selected lane, start with input plus chunk capabilities. +4. Validate extractor requirements, then add extractor provided capabilities. +5. Validate merger requirements, then add merger provided capabilities. +6. Validate normalizer requirements, then add normalizer provided capabilities. +7. Validate configured validator requirements in lane order, then add validator + provided capabilities. If no validators are configured for a lane, do not + attempt capability validation for extractor-owned default validators. +8. Validate output requirements against the union of pipeline-level capabilities + and all selected lane capabilities. -Validation behavior: +Error messages must name the pipeline ID, lane ID when applicable, stage, module +key, and missing capability when applicable. -- if an extractor has no validators, all normalized candidates are approved; -- each validator receives only currently eligible candidates; -- each validator result must have `ValidatorName == Validator.Name()`; -- call `validate.EnforceDecisionCardinality` for every validator result; -- approved decisions keep candidates eligible for the next validator; -- rejected decisions create `artifacts.RejectedArtifact` records using the - candidate, validator name, reason code, and message; -- candidates rejected by one validator are not passed to later validators; -- append validator warnings to `RunOutput.Warnings`. - -Error behavior: - -- return current partial `RunOutput` plus an error for setup, extraction, - validation, cardinality, or metadata mismatch failures; -- wrap errors with enough context to identify the extractor key or validator - name. - -Do not validate source references inside the runner in checkpoint 2. Source -reference validation will be a deterministic validator in a later checkpoint. +`Binding(module)` should trim the module key and leave `LLMProfile` empty and +`Options` nil before resolution. Resolution should fill empty `LLMProfile` +values with `DefaultLLMProfile` on all resolved bindings. ### Required Tests Cover: -- nil runner/factory error; -- invalid source document error; -- empty extractor list error; -- extractor keys run in configured order; -- runner assigns global candidate indices across extractors; -- runner fills empty candidate extractor metadata; -- candidate extractor-key mismatch errors; -- candidate artifact-type mismatch errors; -- candidate schema-version mismatch errors; -- no validators approves all candidates; -- validator approval produces approved artifacts; -- validator rejection produces rejected artifacts and removes candidate from - later validators; -- validator name mismatch errors; -- validator cardinality error is surfaced; -- extractor warnings and validator warnings are collected; -- partial output is returned when a later extractor or validator fails. +- successful resolution with explicit modules; +- default chunk, merge, normalize, output, and `llm_profile`; +- lane selection with `Only`; +- `Only` trimming and deduplication; +- unknown `Only` lane rejection; +- empty `Only` lane rejection; +- empty artifact set rejection; +- empty pipeline ID rejection; +- missing input rejection; +- unknown module key rejection for each stage; +- missing capability rejection for input/chunk/extract/merge/normalize/validate/output + interactions; +- deterministic lane ordering; +- deterministic digest for equivalent input maps; +- digest changes when a module binding changes; +- manifest JSON still omits empty optional fields. -Use fake extractors, validators, and factories only. +Use fake registries and fake specs only. Do not add config file parsing. ### Validation Run: ```sh -gofmt -w internal/framework/pipeline +gofmt -w internal/core/artifacts internal/framework/pipeline +go test ./internal/core/artifacts go test ./internal/framework/pipeline go test ./... ``` -## Stage 6: Registry And Runner Integration Tests +## Stage 4: Generic Merge And Normalize Helpers ### Goal -Prove the extractor registry and runner compose without adding real extract -modules. +Add generic append-in-chunk-order merge and no-op normalization behavior inside +`internal/framework/pipeline`. ### Files To Add -- `internal/framework/pipeline/registry_integration_test.go` +- `internal/framework/pipeline/generic_stages.go` +- `internal/framework/pipeline/generic_stages_test.go` -### Required Test Scenario +### Required API -Use `NewExtractorRegistry()` to register two fake extractor constructors. +Add: -Run the runner with: +```go +type AppendOrderMerger struct{} +func (m AppendOrderMerger) Key() string +func (m AppendOrderMerger) Merge(ctx context.Context, req contracts.MergeRequest) (contracts.MergeResult, error) -- a valid `SourceDocument`; -- extractor keys in a deliberate order; -- fake extractors that each return one candidate; -- fake validators that approve or reject candidates. +type NoopNormalizer struct{} +func (n NoopNormalizer) Key() string +func (n NoopNormalizer) Normalize(ctx context.Context, req contracts.NormalizeRequest) (contracts.NormalizeResult, error) +``` -Assertions: +### Required Behavior -- registered fake extractors are built by key; -- extractor execution follows configured key order; -- approved artifacts are in deterministic order; -- rejected artifacts are in deterministic order; -- no concrete input module or extract module packages are imported. +`AppendOrderMerger` must: + +- return key `appendorder`; +- concatenate candidates by `ChunkArtifacts` slice order and candidate order; +- copy candidate slices so caller mutation does not affect the result; +- preserve source refs and metadata exactly; +- append no warnings. + +`NoopNormalizer` must: + +- return key `noop`; +- copy candidates so caller mutation does not affect the result; +- preserve ordering and source refs exactly; +- append no warnings. + +### Required Tests + +Cover: + +- key values; +- append order across multiple chunks; +- stable candidate order within chunks; +- no-op normalize preserves order; +- result slices are caller-mutation safe; +- nil/empty input returns empty candidates and no error. ### Validation @@ -512,56 +599,293 @@ Run: gofmt -w internal/framework/pipeline go test ./internal/framework/pipeline go test ./... -go build ./cmd/notarius ``` -## Stage 7: Final Checkpoint 2 Review Pass +## Stage 5: Pipeline Runner Refactor ### Goal -Clean up naming, formatting, and accidental scope creep before checkpoint 2 is +Refactor the existing runner from extractor-only execution to resolved pipeline +execution across input, chunk, extract, merge, normalize, validate, and output. + +### Files To Update + +- `internal/framework/pipeline/runner.go` +- `internal/framework/pipeline/runner_test.go` +- `internal/framework/pipeline/registry_integration_test.go` + +### Required API + +Replace the extractor-only runner constructor with: + +```go +type Registries struct { + Inputs *InputAdapterRegistry + Chunkers *ChunkerRegistry + Extractors *ExtractorRegistry + Mergers *MergerRegistry + Normalizers *NormalizerRegistry + Validators *ValidatorRegistry + Outputs *OutputEncoderRegistry +} + +type Runner struct { + // unexported fields +} + +func New(registries Registries) *Runner +``` + +Replace `RunInput` with: + +```go +type RunInput struct { + Pipeline ResolvedPipeline + SourceID string + Path string + RawInput []byte + LLMClient contracts.StructuredLLMClient + Metadata map[string]any +} +``` + +Replace or extend `RunOutput` with: + +```go +type RunOutput struct { + Manifest artifacts.RunManifest `json:"manifest"` + Approved []artifacts.Artifact `json:"approved,omitempty"` + Rejected []artifacts.RejectedArtifact `json:"rejected,omitempty"` + Warnings []contracts.Warning `json:"warnings,omitempty"` + EncodedOutput []byte `json:"-"` + ContentType string `json:"content_type,omitempty"` +} +``` + +Keep the existing candidate metadata normalization behavior, but it should now +run per extractor and across all selected lanes. + +### Required Behavior + +`Run` must: + +1. reject nil runner; +2. reject missing registries for stages used by the pipeline; + `Validators` is required only when at least one selected lane declares + validator bindings; +3. reject empty resolved pipeline ID or digest; +4. build the input adapter and parse `RawInput`; +5. validate the resulting `SourceDocument`; +6. build the chunker and produce chunks; +7. reject empty chunk results; +8. for each selected artifact lane in resolved order: + - build extractor, merger, and normalizer; + - extract from every chunk serially; + - pass `Source`, active `Chunk`, `LLMClient`, and metadata to the extractor; + - normalize candidate metadata with global monotonically increasing indices; + - merge per-chunk candidates; + - normalize merged candidates; + - run configured lane validators against normalized candidates when the lane + declares validators; + - otherwise run the extractor-owned default validator chain; + - append approved artifacts and rejected artifacts to output; +9. build the output encoder and encode the final output bundle; +10. populate `RunManifest` with pipeline ID, pipeline digest, input module, + chunker, lane-level extractor/merger/normalizer/validator keys, output + encoder, source digest, and validation status; +11. return partial output plus wrapped errors on failure. + +Validation status: + +- `approved` when all candidates from all lanes survive validation and no lane + returns rejected artifacts; +- `rejected` when at least one candidate is rejected and execution otherwise + succeeds; +- `failed` when returning an error after a manifest has been initialized. + +Warnings: + +- collect warnings from input, chunk, extract, merge, normalize, validator, and + output stages where those stages expose warnings; +- existing input adapters do not expose warnings, so no input warnings are + required in this checkpoint. + +Do not validate source references inside the runner in checkpoint 3. Source +reference validation remains a later deterministic validator. + +### Required Tests + +Refactor existing runner tests and add coverage for: + +- nil runner error; +- missing registry errors; +- invalid resolved pipeline error; +- input adapter build and parse errors; +- invalid source document error; +- chunker build and chunk errors; +- empty chunk result error; +- extractor runs once per chunk; +- extractor receives active chunk and LLM client; +- merge receives per-chunk candidates; +- normalize receives merged candidates; +- validator approvals and rejections still work; +- configured validators are built and run in lane order; +- extractor-owned default validators run when a lane does not configure + validators; +- configured validators replace extractor-owned default validators when present; +- global candidate indices across lanes and chunks; +- warnings from chunk, extract, merge, normalize, validator, and output stages; +- output encoder receives manifest and artifacts; +- manifest pipeline ID, digest, and artifact lane details are populated; +- partial output is returned when a later lane fails. + +Use fake modules only. + +### Validation + +Run: + +```sh +gofmt -w internal/framework/pipeline +go test ./internal/framework/pipeline +go test ./... +``` + +## Stage 6: Fixture-Driven Walking Skeleton + +### Goal + +Add an end-to-end fixture-driven integration test that exercises the full staged +path with fake modules and a fake LLM client. + +### Files To Add + +- `internal/framework/pipeline/testdata/walking_skeleton_input.json` +- `internal/framework/pipeline/testdata/walking_skeleton_output.json` +- `internal/framework/pipeline/walking_skeleton_test.go` + +### Required Fixture Shape + +Use a small input fixture with two or three source units in a simple fake JSON +format. Do not use Seriatim. + +Example input shape: + +```json +{ + "id": "fixture-source", + "units": [ + {"id": "u1", "text": "First event."}, + {"id": "u2", "text": "Second event."}, + {"id": "u3", "text": "Third event."} + ] +} +``` + +The expected output fixture should contain: + +- manifest with `pipeline_id` and `pipeline_digest`; +- approved artifacts emitted from multiple chunks; +- no rejected artifacts for the happy path; +- warnings only if fake stages deliberately emit them. + +Compare expected output by unmarshaled structural equality, not raw bytes, so +formatting differences do not fail the test. + +### Required Test Setup + +Register fake modules with metadata: + +- fake input adapter key `fake/input`, provides `plain_text`; +- fake chunker key `fake/chunk`, requires `plain_text`, provides `chunks`; +- fake extractor key `fake/extract`, requires `chunks`, provides + `fake_artifacts`; +- append-order merger key `appendorder`, requires `fake_artifacts`; +- no-op normalizer key `noop`; +- fake output encoder key `json`. + +Use an in-memory `PipelineProfile`: + +```go +PipelineProfile{ + ID: "walking-skeleton", + Input: Binding("fake/input"), + Chunk: Binding("fake/chunk"), + Artifacts: map[string]ArtifactLaneProfile{ + "events": {Extract: Binding("fake/extract")}, + }, + Output: Binding("json"), +} +``` + +Resolve the profile, run the runner with fixture bytes and fake LLM client, then +assert the encoded output matches the expected fixture structurally. + +### Required Behavior + +- The fake input adapter parses raw fixture bytes into `SourceDocument`. +- The fake chunker produces at least two chunks. +- The fake extractor calls the fake structured LLM client once per chunk. +- The fake extractor returns one artifact candidate per chunk. +- The append-order merger preserves chunk order. +- The no-op normalizer preserves candidates. +- The fake output encoder produces deterministic JSON bytes. +- The test fails before execution if required capabilities are missing. + +### Additional Tests + +Add focused tests for the walking skeleton failure path: + +- missing required capability fails during resolution; +- `Only` with an unknown lane fails during resolution; +- fake LLM call count equals chunk count in the happy path. + +### Validation + +Run: + +```sh +gofmt -w internal/framework/pipeline +go test ./internal/framework/pipeline +go test ./... +``` + +## Stage 7: Final Checkpoint 3 Review Pass + +### Goal + +Clean up naming, formatting, and accidental scope creep before checkpoint 3 is considered complete. ### Required Review Check: -- no package names mention transcripts, Seriatim, D&D, spells, NPCs, items, or - combat; -- no concrete input module or extract module package exists; +- no Seriatim parser exists; +- no D&D module exists; +- no production config file loader exists; +- no CLI behavior was added; - no LLM provider code exists; -- no prompt, response schema, diagnostics, config, or source chunking package - was added; -- no third-party dependency was added; -- runner operates on `SourceDocument`; -- input adapter registry is not wired into runner yet; -- roadmap docs remain the only place describing future behavior. +- no prompt, response schema, diagnostics, or durable output package exists; +- no new framework package was created beyond `contracts`, `pipeline`, and + `validate`; +- stage contracts remain source/domain agnostic; +- capabilities remain flat strings; +- resolved pipeline profiles remain fixed-shape templates; +- walking skeleton uses fake modules only. ### Required Validation Run: ```sh -gofmt -w internal/framework +gofmt -w internal/core internal/framework go test ./... go build ./cmd/notarius -git status --short ``` -Remove the root `./notarius` binary produced by `go build ./cmd/notarius` before -finishing the implementation turn. - -The implementation response for this checkpoint should summarize: - -- files added; -- tests run; -- any deviations from this plan and why. - ## Open Questions -No blocking open questions remain for checkpoint 2. - -The plan intentionally keeps raw input parsing outside the runner. The input -adapter registry is added and tested now because it is part of framework -composition, but concrete parsing and adapter-runner wiring remain deferred -until the Seriatim adapter checkpoint. +None. The deferred choices identified while drafting this plan are intentionally +out of scope for checkpoint 3 and should remain deferred until later roadmap +items.