Add a staged implementation plan for the pipeline stages
This commit is contained in:
@@ -81,17 +81,11 @@ The repository should also contain a resolved pipeline model that represents:
|
||||
The runner should orchestrate fake implementations through chunk, extract,
|
||||
merge, normalize, and approval/validation behavior in tests.
|
||||
|
||||
The checkpoint should include one fixture-driven integration test that starts
|
||||
from fixture input bytes and ends at encoded output bytes. The fixture should
|
||||
use an in-memory pipeline profile with fake input adapter, deterministic
|
||||
chunker, trivial extractor, fake structured LLM client, generic merger, no-op
|
||||
normalizer, and fake or minimal JSON output encoder. This is a contract
|
||||
exercise, not a useful user-facing workflow.
|
||||
|
||||
The walking skeleton should live in `internal/framework/pipeline`, with
|
||||
fixtures under that package's `testdata/`. If the CLI has an extract command by
|
||||
then, the same fixture may also be exercised through CLI wiring; if not, CLI
|
||||
coverage remains deferred.
|
||||
The checkpoint should include a fixture-driven walking skeleton that starts from
|
||||
fixture input bytes and ends at encoded output bytes. The walking skeleton should
|
||||
exercise the stage contracts, resolved pipeline model, module metadata,
|
||||
capability validation, defaults, lane selection, and fake LLM client wiring. It
|
||||
is contract coverage, not useful user-facing behavior.
|
||||
|
||||
## Design Intent
|
||||
|
||||
@@ -99,7 +93,7 @@ Chunking is a core application concern because many source documents, especially
|
||||
transcripts, will be too large for a single LLM extraction pass.
|
||||
|
||||
Chunk processing may be serial or parallel depending on extractor needs. The
|
||||
architecture should support both, but the first implementation can execute
|
||||
architecture should support both, while checkpoint 3 may execute
|
||||
deterministically in series until a later checkpoint introduces concurrency.
|
||||
|
||||
Merge and normalize are separate stages:
|
||||
@@ -120,8 +114,9 @@ The architecture should leave room for extractor-level processing modes:
|
||||
- serial chunk processing;
|
||||
- parallel chunk processing.
|
||||
|
||||
The first implementation may model these modes without implementing parallel
|
||||
execution. It should not bake in a single-pass assumption.
|
||||
Checkpoint 3 may execute chunks serially for deterministic behavior.
|
||||
The contracts should not bake in a single-pass assumption or prevent later
|
||||
parallel execution.
|
||||
|
||||
## Generic Merge Behavior
|
||||
|
||||
@@ -143,29 +138,16 @@ Domain-specific normalizers may later:
|
||||
- enforce chronological or source-reference consistency;
|
||||
- attach normalization warnings.
|
||||
|
||||
## Walking Skeleton Fixture
|
||||
## Walking Skeleton
|
||||
|
||||
The fixture-driven skeleton should prove the staged architecture continuously as
|
||||
new contracts are added. It should be deliberately small:
|
||||
new contracts are added. It should remain deliberately small and use fake modules
|
||||
only. It should validate module keys and flat capability requirements before
|
||||
execution, using registry metadata rather than constructing modules. Capability
|
||||
values should remain simple strings.
|
||||
|
||||
- a fixture source document or raw source input with two or three source units;
|
||||
- a fake input adapter that parses the fixture into `SourceDocument`;
|
||||
- a deterministic chunker that produces multiple chunks;
|
||||
- a trivial extractor that calls the fake structured LLM client and emits one
|
||||
artifact candidate per chunk;
|
||||
- a generic append-in-chunk-order merger;
|
||||
- a no-op normalizer;
|
||||
- a fake or minimal JSON output encoder;
|
||||
- an expected output fixture checked byte-for-byte or by unmarshaled structural
|
||||
equality.
|
||||
|
||||
The fake LLM client should be part of the test setup so the contract is
|
||||
exercised without introducing provider code, prompt assets, or response schema
|
||||
assets.
|
||||
|
||||
The walking skeleton should validate module keys and flat capability
|
||||
requirements before execution. This should use registry metadata rather than
|
||||
constructing modules. Keep capability values as simple strings.
|
||||
Implementation staging for the walking skeleton belongs in
|
||||
[`implementation.md`](implementation.md).
|
||||
|
||||
## Done Criteria
|
||||
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user