724 lines
32 KiB
Markdown
724 lines
32 KiB
Markdown
# Domain-Typed Pipeline Implementation Roadmap
|
|
|
|
## Status
|
|
|
|
The domain-typed pipeline and the first five remediation stages were completed
|
|
on 2026-07-17. A subsequent review identified three additional implementation
|
|
issues. Stages 1 through 5 are retained as the completed implementation record;
|
|
Stages 6 through 8 are the decision-complete plan for the remaining work.
|
|
|
|
Implement the pending stages in order. Keep each stage independently reviewable
|
|
and leave the repository passing its full validation suite before beginning
|
|
the next stage. Unless a stage explicitly says otherwise, preserve public CLI,
|
|
configuration, module-key, validator-key, durable output, and checkpoint file
|
|
contracts.
|
|
|
|
Current architectural policy is defined by
|
|
[Architecture](../policy/architecture.md). The design context remains in the
|
|
[feature roadmap](domain.md) and
|
|
[ADR-0002](../adr/0002-linear-pipes-and-filters-pipeline.md),
|
|
[ADR-0003](../adr/0003-typed-interfaces-with-two-zone-data-model.md), and
|
|
[ADR-0004](../adr/0004-package-modules-by-domain.md).
|
|
|
|
## Delivered Baseline
|
|
|
|
The existing implementation already provides:
|
|
|
|
- domain-first production extensions and package-family registrars;
|
|
- engine-owned source provenance using `source.SourceRef` and workspace schema
|
|
v2;
|
|
- one domain-owned Go type through extraction, merge, normalization, and typed
|
|
validation;
|
|
- schema-aware artifact codecs at checkpoint, debug, and output boundaries;
|
|
- pre-execution typed resolution, option validation, and preparation;
|
|
- one shared scheduled LLM client; and
|
|
- bounded chunk-first, lane-second extraction with deterministic public
|
|
outcomes.
|
|
|
|
Stages 1 through 5 corrected checkpoint identity, completed merge and normalize
|
|
retry debugging, removed kind-ambiguous registry lookup, strengthened the
|
|
initial architectural guard, and removed the obsolete sequential extraction
|
|
path. Stages 6 through 8 complete candidate-artifact handling, make attempt
|
|
debugging comprehensive across all stages, and close the remaining import-guard
|
|
loopholes.
|
|
|
|
## Implementation Rules
|
|
|
|
Apply these rules to every stage:
|
|
|
|
1. Read the task-specific documents listed in
|
|
[Development](../development.md) before changing the affected subsystem.
|
|
2. Add focused regression tests that fail against the pre-stage code and pass
|
|
after the change.
|
|
3. Preserve deterministic behavior. Do not base digests, public ordering,
|
|
reference resolution, or reported errors on Go map iteration or goroutine
|
|
completion order.
|
|
4. Keep framework-owned type erasure private. Do not reintroduce module-facing
|
|
`any`, raw JSON handoffs, or a generic processing interface.
|
|
5. Update current-behavior documentation in the same stage when externally
|
|
observable or documented internal behavior changes. Follow
|
|
[Documentation Policy](../policy/documentation.md); do not describe a later
|
|
stage as already implemented.
|
|
6. Run focused tests while iterating. At the end of every stage, run:
|
|
|
|
```sh
|
|
go test ./...
|
|
go test -race ./...
|
|
go vet ./...
|
|
go build ./cmd/notarius
|
|
git diff --check
|
|
```
|
|
|
|
## Completed Remediation Record
|
|
|
|
The following five stages have been implemented and are retained to document
|
|
the decisions and completion criteria that produced the current baseline.
|
|
|
|
## Stage 1: Include Resolved Validator Policy In Pipeline Identity
|
|
|
|
### Goal
|
|
|
|
Ensure that every effective validator-chain change alters the resolved
|
|
pipeline digest and therefore prevents reuse of checkpoints created under a
|
|
different validation policy.
|
|
|
|
### Required Changes
|
|
|
|
1. Extend `resolvedPipelineDigest` in
|
|
`internal/framework/pipeline/profile.go` to include the complete resolved
|
|
validator-chain collection. Include, for every chain:
|
|
|
|
- stage, lane ID, and owning module key;
|
|
- validator order;
|
|
- each complete resolved binding, including module key, LLM profile,
|
|
retries, options, and references;
|
|
- validator execution class;
|
|
- validator target; and
|
|
- artifact kind.
|
|
|
|
2. Hash the already canonical `ResolvedPipeline.ValidatorChains` order produced
|
|
by resolution. Do not independently sort validators or otherwise weaken
|
|
configured order. Continue excluding only the digest field itself.
|
|
3. Use the same deterministic JSON-and-SHA-256 mechanism as the existing
|
|
pipeline digest. Go map keys encoded inside bindings must retain the
|
|
deterministic ordering supplied by `encoding/json`.
|
|
4. Do not add validator fingerprints separately to individual stage
|
|
dependencies. Pipeline/workspace identity is the authoritative invalidation
|
|
boundary for validation-policy changes.
|
|
5. Do not advance the workspace schema version. The corrected digest naturally
|
|
selects a different checkpoint directory; existing checkpoints remain
|
|
intact and become reuse misses for the changed pipeline identity.
|
|
|
|
### Tests
|
|
|
|
Add focused tests under `internal/framework/pipeline` and, where useful,
|
|
`internal/core/workspace` proving that:
|
|
|
|
- registering a different default chain changes the resolved pipeline digest;
|
|
- adding, removing, or reordering a default validator changes the digest;
|
|
- changing a resolved validator binding option or LLM profile changes the
|
|
digest;
|
|
- changing execution class, target, or artifact kind changes the digest;
|
|
- resolving the same pipeline and catalog repeatedly produces the same digest;
|
|
- an explicit empty override produces an empty resolved chain and differs from
|
|
a non-empty inherited default; and
|
|
- a changed validator-chain digest produces a different checkpoint identity,
|
|
while an unchanged chain preserves it.
|
|
|
|
Retain the existing tests proving that the digest field itself is excluded and
|
|
that artifact schema identity participates in the digest.
|
|
|
|
### Documentation
|
|
|
|
Update the checkpoint invalidation description in `docs/operations.md` and the
|
|
identity description in `docs/internal/pipeline.md` to state concisely that the
|
|
effective resolved validator policy participates in pipeline identity.
|
|
|
|
### Completion Gate
|
|
|
|
A checkpoint accepted under one resolved default or explicit validator chain
|
|
must not be reusable after that chain changes.
|
|
|
|
## Stage 2: Complete Attempt-Scoped Debugging For Merge And Normalize
|
|
|
|
### Goal
|
|
|
|
Give merge and normalize retries the same attempt-level observability and
|
|
nested LLM-call association already provided for chunk and extract attempts.
|
|
|
|
### Required Changes
|
|
|
|
1. In `internal/framework/pipeline/runner_typed.go`, create an attempt-specific
|
|
debug context before invoking each merge or normalize module. Pass that
|
|
context to both the module operation and its validation chain.
|
|
2. Use these stable paths:
|
|
|
|
```text
|
|
merge/<lane-id>/attempt-<NN>.json
|
|
merge/<lane-id>/attempt-<NN>/prompt-<NNNN>.json
|
|
merge/<lane-id>/attempt-<NN>/response-<NNNN>.json
|
|
merge/<lane-id>/attempt-<NN>/response-content-<NNNN>.<ext>
|
|
|
|
normalize/<lane-id>/attempt-<NN>.json
|
|
normalize/<lane-id>/attempt-<NN>/prompt-<NNNN>.json
|
|
normalize/<lane-id>/attempt-<NN>/response-<NNNN>.json
|
|
normalize/<lane-id>/attempt-<NN>/response-content-<NNNN>.<ext>
|
|
```
|
|
|
|
Continue using two-digit retry attempt numbers and the existing debug path
|
|
sanitization and LLM call numbering behavior.
|
|
3. Write one module-attempt envelope for every attempted merge and normalize
|
|
operation:
|
|
|
|
- on success, record the codec-backed candidate artifact and warnings that
|
|
will be promoted if the attempt is accepted;
|
|
- on validator rejection, record the rejection without promoting discarded
|
|
warnings;
|
|
- on module, validation, serialization, or debug failure, record the error;
|
|
and
|
|
- in all cases, attach the LLM calls recorded in the module attempt scope.
|
|
|
|
4. Keep validator-specific debug scopes under the existing `validate/...`
|
|
hierarchy. A validator's LLM calls remain linked to its validator attempt;
|
|
the merge or normalize module envelope links only calls made by that module
|
|
attempt.
|
|
5. Preserve the existing stage-level `input.json` and `output.json` artifacts.
|
|
Checkpoint reuse should continue to emit stage-level artifacts but should
|
|
not synthesize retry attempts that did not execute.
|
|
6. Factor common attempt-envelope behavior into a small private helper where it
|
|
prevents chunk, extract, merge, and normalize instrumentation from drifting.
|
|
Do not introduce a new public runner abstraction solely for debugging.
|
|
7. Treat any debug write failure as a framework error, consistent with current
|
|
debug policy.
|
|
|
|
### Tests
|
|
|
|
Add tests with instrumented LLM-backed fake mergers and normalizers proving
|
|
that:
|
|
|
|
- first-attempt success writes the expected attempt and nested LLM artifacts;
|
|
- a failed first attempt followed by success writes two distinct attempt
|
|
envelopes and associates each call with the correct attempt;
|
|
- module errors, validator errors, and final rejection are represented in the
|
|
corresponding attempt envelope;
|
|
- validator LLM calls remain under validator paths rather than being attributed
|
|
to the module scope;
|
|
- discarded-attempt warnings are not promoted;
|
|
- checkpoint reuse produces no module-attempt files; and
|
|
- no merge or normalize call falls back to an unscoped stage-name debug path.
|
|
|
|
Extend the CLI debug integration test only as needed to verify the public debug
|
|
directory layout. Keep most behavioral coverage in the pipeline package.
|
|
|
|
### Documentation
|
|
|
|
Update `docs/operations.md` if necessary to show the stable merge and normalize
|
|
attempt paths. Update `docs/internal/pipeline.md` only where its implementation
|
|
description needs clarification; its existing attempt-scoping guarantee should
|
|
become fully true rather than be weakened.
|
|
|
|
### Completion Gate
|
|
|
|
Every executed merge and normalize retry must have a distinct debug envelope,
|
|
and every LLM call made by that module attempt must be nested under and linked
|
|
from that attempt.
|
|
|
|
## Stage 3: Make Typed Variant Spec Lookup Kind-Specific
|
|
|
|
### Goal
|
|
|
|
Ensure that CLI reference-target discovery and other behavior-sensitive lookup
|
|
select the merger or normalizer specification for the lane's actual artifact
|
|
kind, never an arbitrary map entry.
|
|
|
|
### Required Changes
|
|
|
|
1. Add explicit kind-specific lookup methods to `MergerRegistry` and
|
|
`NormalizerRegistry`, named:
|
|
|
|
```go
|
|
SpecForArtifact(key string, kind contracts.ArtifactKind) (ModuleSpec, bool)
|
|
```
|
|
|
|
Normalize the key and artifact kind in the same way as typed registration
|
|
and return a cloned spec.
|
|
2. Change CLI reference-target discovery in `internal/cli/run.go` to retain the
|
|
selected extractor's declared artifact kind and use it for merger and
|
|
normalizer spec lookup in that lane.
|
|
3. A missing typed variant must produce a deterministic error naming the
|
|
pipeline, lane, stage, module key, requested artifact kind, and sorted
|
|
registered kinds, matching the quality of full pipeline resolution errors.
|
|
4. Keep `Spec(key)` for kind-neutral catalog inspection and compatibility with
|
|
existing callers, but remove its map-order dependence. Select the first
|
|
registered artifact kind in sorted order before cloning its spec. Add a
|
|
comment making clear that behavior-sensitive code must use
|
|
`SpecForArtifact`.
|
|
5. Do not require typed variants under one reusable key to expose identical
|
|
reference slots or capabilities. Their kind-specific specs are allowed to
|
|
differ, and resolution must consistently choose the matching variant.
|
|
6. Audit all merger and normalizer `Spec` callers. Convert any caller making a
|
|
lane-specific decision to `SpecForArtifact`; leave only catalog or display
|
|
callers on the kind-neutral method.
|
|
|
|
### Tests
|
|
|
|
Register at least two artifact-kind variants under the same merger key and the
|
|
same normalizer key with intentionally different reference slots. Prove that:
|
|
|
|
- `SpecForArtifact` returns the correct cloned variant;
|
|
- lookup is stable regardless of registration order;
|
|
- CLI qualified and unqualified reference discovery uses the selected lane's
|
|
variant;
|
|
- a reference accepted by one variant is not incorrectly accepted for another;
|
|
- a missing variant reports sorted available kinds; and
|
|
- repeated `Spec(key)` calls return the same deterministic catalog result.
|
|
|
|
Retain coverage for the existing single-variant D&D catalog behavior.
|
|
|
|
### Documentation
|
|
|
|
Update `docs/internal/pipeline.md` or `docs/internal/modules.md` only if either
|
|
currently describes registry lookup mechanics. No CLI or configuration syntax
|
|
change is intended.
|
|
|
|
### Completion Gate
|
|
|
|
No behavior-sensitive merger or normalizer lookup may depend on map iteration,
|
|
and CLI reference resolution must use the artifact kind selected by the lane's
|
|
extractor.
|
|
|
|
## Stage 4: Enforce Domain Import Boundaries Generically
|
|
|
|
### Goal
|
|
|
|
Make the repository guard enforce ADR-0004 for present and future module
|
|
families without a hardcoded list of domain names or domain pairs.
|
|
|
|
### Required Changes
|
|
|
|
1. Refactor `internal/modules/import_boundaries_test.go` so that a module family
|
|
is derived from the first path segment under `internal/modules/`, rather than
|
|
recognized by a fixed `isDomain` list.
|
|
2. Treat `generic` as the domain-neutral reusable extension family. Treat every
|
|
other production family, including `dnd`, `seriatim`, and future families,
|
|
as concrete for import-boundary purposes.
|
|
3. Enforce these rules:
|
|
|
|
- a family root package must not import its own child implementation or
|
|
registrar packages;
|
|
- child packages within the same concrete family may import the family root,
|
|
shared helpers, or sibling implementations when needed;
|
|
- a concrete family must not import another concrete family;
|
|
- generic packages must not import concrete families;
|
|
- concrete implementation packages must not import generic implementation
|
|
packages directly;
|
|
- `internal/modules/<family>/register` may import its own family packages and
|
|
generic packages to compose typed strategies;
|
|
- the generic registrar may import generic child packages; and
|
|
- the application composition root and designated black-box integration
|
|
tests may compose multiple families.
|
|
|
|
4. Exclude `internal/modules/integration` from production-family discovery.
|
|
Preserve its exemption only for `_test.go` black-box composition files; do
|
|
not create a blanket exemption for production Go files.
|
|
5. Apply production import rules to white-box tests located in concrete module
|
|
packages. If an existing test composes concrete and generic implementations,
|
|
move that cross-family coverage to `internal/modules/integration` or replace
|
|
the foreign implementation with a package-local test double. Do not exempt
|
|
arbitrary `_test.go` files merely because they are tests.
|
|
6. Keep the test based on parsed Go imports. Do not add a new build tool or
|
|
external dependency for this guard.
|
|
|
|
### Tests
|
|
|
|
Expand the table-driven import tests to cover:
|
|
|
|
- a hypothetical future concrete family, demonstrating that no code change is
|
|
needed to enforce its boundaries;
|
|
- generic-to-concrete rejection for both current and hypothetical families;
|
|
- concrete-to-peer-concrete rejection;
|
|
- concrete implementation-to-generic rejection;
|
|
- concrete registrar-to-generic acceptance;
|
|
- family-root-to-child rejection;
|
|
- child-to-family-root and same-family sibling acceptance;
|
|
- application composition-root acceptance; and
|
|
- black-box integration-test acceptance without exempting non-test files.
|
|
|
|
Run the guard against the complete current repository and confirm that no
|
|
production package must be moved to satisfy it.
|
|
|
|
### Documentation
|
|
|
|
No ADR change is required. Update `docs/internal/modules.md` only if its package
|
|
boundary description needs to name the registrar-only generic composition
|
|
rule more clearly.
|
|
|
|
### Completion Gate
|
|
|
|
Adding a new directory under `internal/modules/<new-family>` must automatically
|
|
receive the same cross-family and registrar enforcement as current production
|
|
families.
|
|
|
|
## Stage 5: Remove The Obsolete Sequential Extraction Path
|
|
|
|
### Goal
|
|
|
|
Make the runner's structure match the implemented concurrent architecture:
|
|
the coordinator owns extraction, and a lane continuation begins from finalized
|
|
extract results and performs only merge and normalize work.
|
|
|
|
### Required Changes
|
|
|
|
1. Replace `runTypedLane` with a continuation-oriented private function whose
|
|
input explicitly contains the finalized extraction state needed by merge and
|
|
normalize:
|
|
|
|
- accepted typed extract artifacts;
|
|
- serialized checkpoint artifacts;
|
|
- accepted warnings;
|
|
- rejected outputs; and
|
|
- the original extract checkpoint decision.
|
|
|
|
Use a private struct if it keeps the call boundary clear and avoids a long
|
|
positional parameter list.
|
|
2. Change `continueLane` in `runner_concurrent.go` to pass the finalized state
|
|
directly. Do not make completed extraction look like checkpoint reuse.
|
|
3. Remove:
|
|
|
|
- `completedExtractLoader`;
|
|
- the `RunInput.extractDecision` private override;
|
|
- the sequential extraction branch formerly contained in `runTypedLane`;
|
|
and
|
|
- duplicate extraction retry, validation, checkpoint, and debug logic made
|
|
unreachable by concurrent coordination.
|
|
|
|
4. Keep extraction checkpoint loading and validation in `prepareLaneExtract`,
|
|
extract execution in `runExtractJob`, and deterministic final aggregation in
|
|
`finalizeLaneExtract`.
|
|
5. Preserve existing behavior exactly:
|
|
|
|
- extract checkpoint events report the real loader decision;
|
|
- reused and freshly computed extract results enter continuation through the
|
|
same typed state;
|
|
- accepted artifacts reach merge in chunk-index order;
|
|
- warnings and rejections retain stable ordering and promotion semantics;
|
|
- a lane with no accepted extracts follows the current merge behavior;
|
|
- stage-level extract input/output debug artifacts remain unchanged; and
|
|
- failure classification and cancellation retain their deterministic scope.
|
|
|
|
6. Keep merge and normalize serial within a lane and keep cross-lane
|
|
continuations bounded by the existing worker policy. Do not introduce a
|
|
goroutine per lane or chunk.
|
|
|
|
### Tests
|
|
|
|
Add or adjust focused runner tests proving parity for:
|
|
|
|
- fresh extraction, fully reused extraction, and a mixture of accepted and
|
|
rejected chunk results;
|
|
- deterministic extract ordering under reverse completion;
|
|
- warning promotion across retries;
|
|
- real checkpoint decision reporting for reused and recomputed extracts;
|
|
- extract input/output and attempt debug paths;
|
|
- framework cancellation and deterministic error selection; and
|
|
- unchanged bounded worker and global LLM concurrency behavior.
|
|
|
|
Use source search or a package-local compile-time assertion where practical to
|
|
confirm there is only one extraction execution path and no remaining
|
|
`completedExtractLoader` or `extractDecision` compatibility shim.
|
|
|
|
### Documentation
|
|
|
|
Update `docs/internal/pipeline.md` if its execution-flow description names the
|
|
old continuation mechanism. This stage is an internal refactor and must not
|
|
change operator or integration contracts.
|
|
|
|
### Completion Gate
|
|
|
|
The concurrent coordinator must be the only code path that executes extraction,
|
|
and lane continuation must consume finalized extraction state without routing
|
|
it through a synthetic checkpoint loader.
|
|
|
|
## Pending Remediation Plan
|
|
|
|
Implement Stages 6 through 8 in order. Do not perform final roadmap closeout
|
|
until all three stages and their documentation changes have landed.
|
|
|
|
## Stage 6: Validate Merge And Normalize Candidates Before Final Encoding
|
|
|
|
### Goal
|
|
|
|
Preserve the two-zone artifact model at merge and normalize boundaries: a
|
|
module result remains a candidate artifact until typed validation accepts it,
|
|
and only an accepted artifact is encoded with the final artifact codec and
|
|
eligible for checkpointing or downstream use.
|
|
|
|
### Required Changes
|
|
|
|
1. Refactor the merge and normalize attempt paths in
|
|
`internal/framework/pipeline/runner_typed.go` so each attempt proceeds in
|
|
this order:
|
|
|
|
1. execute the typed module and collect its warnings;
|
|
2. serialize the returned value with `ArtifactCodec.EncodeCandidate` for
|
|
attempt-debug capture;
|
|
3. execute the resolved typed validator chain against the in-memory value;
|
|
4. on validator rejection, record the rejected candidate and finish the
|
|
attempt without invoking `ArtifactCodec.Encode` or creating a
|
|
checkpoint artifact; and
|
|
5. only after validator acceptance, invoke `ArtifactCodec.Encode` through
|
|
the normal checkpoint-artifact path.
|
|
|
|
2. Use one private candidate-serialization helper for merge, normalize, and
|
|
any equivalent pre-validation debug path. The helper must preserve the
|
|
codec's media type and schema digest in the debug artifact, but its result
|
|
must never be used as a final checkpoint or downstream artifact.
|
|
3. Treat candidate-encoding and final-encoding failures as framework errors,
|
|
not validator rejections. Record the applicable attempt envelope before
|
|
returning the error using the existing attempt-debug behavior; Stage 7 will
|
|
consolidate that behavior across every retrying stage.
|
|
4. Preserve retry behavior:
|
|
|
|
- a validator rejection may be retried according to the stage retry policy;
|
|
- warnings from rejected attempts remain attempt-local unless existing
|
|
warning-promotion policy says otherwise;
|
|
- a final accepted value and its serialized checkpoint artifact are
|
|
published only after the attempt has completed successfully; and
|
|
- checkpoint identity, checkpoint event reporting, and durable artifact
|
|
formats remain unchanged.
|
|
|
|
5. Do not weaken a domain codec so that its final `Encode` accepts invalid
|
|
domain values. In particular, keep the D&D spell codec's semantic checks at
|
|
the final-artifact boundary and use `EncodeCandidate` for its intentionally
|
|
incomplete pre-validation representation.
|
|
|
|
### Tests
|
|
|
|
Add focused merge and normalize tests using an instrumented codec for which
|
|
`EncodeCandidate` accepts an invalid value but `Encode` rejects it. Prove that:
|
|
|
|
- the validator, rather than final encoding, determines that the candidate is
|
|
rejected;
|
|
- rejected candidates never call final `Encode` and never produce checkpoint
|
|
artifacts;
|
|
- an accepted candidate calls final `Encode` exactly once and produces the
|
|
existing checkpoint representation;
|
|
- candidate-encoding and post-acceptance final-encoding failures are reported
|
|
as framework errors and have attempt debug records; and
|
|
- retry warnings, rejection promotion, and checkpoint event ordering remain
|
|
deterministic.
|
|
|
|
Retain or add a D&D spell regression test demonstrating that an incomplete
|
|
candidate can reach validation without making the final spell codec permissive.
|
|
|
|
### Documentation
|
|
|
|
Update `docs/internal/pipeline.md` to state explicitly that merge and normalize
|
|
debug payloads may contain candidate encodings, while checkpoints and stage
|
|
outputs always contain validator-approved final encodings. Update
|
|
`docs/internal/artifacts.md` if it currently implies that every serialized
|
|
debug artifact is a final Domain Artifact Zone representation.
|
|
|
|
### Completion Gate
|
|
|
|
No merge or normalize code path may invoke final artifact encoding before typed
|
|
validation acceptance, and a validator-rejected candidate must remain
|
|
observable as a rejection even when the codec would refuse to encode it as a
|
|
final artifact.
|
|
|
|
## Stage 7: Record Every Executed Attempt And Its Terminal Outcome
|
|
|
|
### Goal
|
|
|
|
Make attempt debugging complete and uniform across chunk, extract, merge, and
|
|
normalize so every executed retry has one terminal attempt envelope, including
|
|
validator rejection, module error, validator error, and serialization error.
|
|
|
|
### Required Changes
|
|
|
|
1. Introduce or consolidate a private attempt-terminal recorder used by all
|
|
four retrying stages. It must write exactly one envelope per executed
|
|
attempt and support these terminal outcomes:
|
|
|
|
- accepted success;
|
|
- validator rejection;
|
|
- module execution error;
|
|
- validator execution error;
|
|
- candidate-serialization error; and
|
|
- final-serialization error where final encoding occurs within the attempt.
|
|
|
|
2. Preserve the existing debug directory layout and envelope schema. Populate
|
|
the envelope consistently with the attempt number, warnings accumulated by
|
|
that attempt, any available candidate payload or rejection details, and the
|
|
terminal error text when an error occurred. A validator rejection without
|
|
an execution error remains a rejection and must not acquire a synthetic
|
|
error string.
|
|
3. Complete the extract path in
|
|
`internal/framework/pipeline/runner_concurrent.go`:
|
|
|
|
- write an attempt envelope before returning a validator rejection or
|
|
validator error;
|
|
- write an attempt envelope before returning a final-serialization error;
|
|
and
|
|
- stop discarding errors returned while writing module-error attempt data.
|
|
|
|
4. Complete the chunk path in `internal/framework/pipeline/runner.go` by
|
|
recording validator execution errors in the envelope's error field and by
|
|
propagating attempt-write failures on every terminal path.
|
|
5. Route the existing merge and normalize attempt handling through the same
|
|
terminal-recording behavior without changing their retry or validation
|
|
semantics. Coordinate this work with Stage 6 so rejected candidates use
|
|
candidate encoding and accepted values use final encoding.
|
|
6. Keep LLM-call ownership scoped to the operation that made the call:
|
|
|
|
- module calls belong to the module attempt envelope;
|
|
- validator calls remain in their validator-specific debug scope; and
|
|
- attaching a call to an attempt must not duplicate it in another module
|
|
attempt or orphan it from the attempt that initiated it.
|
|
|
|
7. Never silently discard a debug write failure. If another error already
|
|
exists, return an `errors.Join` result that preserves both the primary error
|
|
and the contextualized debug error. If no primary error exists, return the
|
|
contextualized debug error. Do not replace a validator rejection with a
|
|
framework error unless persisting its required debug record fails.
|
|
8. Keep debug failures subject to the retry boundary already surrounding the
|
|
relevant attempt; do not add a second retry loop specifically for debug
|
|
persistence.
|
|
|
|
### Tests
|
|
|
|
Add table-driven attempt-debug tests covering each terminal outcome for chunk
|
|
and extract, and retain equivalent merge and normalize coverage. At minimum,
|
|
prove that:
|
|
|
|
- a first-attempt extract rejection followed by success writes both envelopes;
|
|
- a terminal extract rejection, validator error, module error, candidate-codec
|
|
error, and final-codec error each write an envelope with the correct fields;
|
|
- a chunk validator execution error appears in its attempt envelope;
|
|
- debug write failures are returned, and are joined with the primary error
|
|
when both occur;
|
|
- module LLM calls are attached to the correct attempt while validator LLM
|
|
calls remain isolated; and
|
|
- success, rejection, warning, and retry ordering remains deterministic.
|
|
|
|
Use a shared assertion helper to verify the invariant that the number and
|
|
indices of attempt envelopes equal the attempts actually executed.
|
|
|
|
### Documentation
|
|
|
|
Update `docs/internal/pipeline.md` and `docs/operations.md` so their attempt
|
|
debug guarantees name all terminal outcomes and explain the separation between
|
|
module-attempt and validator-call scopes. Do not promise that a payload exists
|
|
when failure occurred before a candidate value was available.
|
|
|
|
### Completion Gate
|
|
|
|
For chunk, extract, merge, and normalize, every entered attempt must leave
|
|
exactly one terminal envelope or return an error that explicitly reports why
|
|
that envelope could not be persisted. No attempt-related debug write error may
|
|
be ignored.
|
|
|
|
## Stage 8: Close Production Import-Guard Loopholes
|
|
|
|
### Goal
|
|
|
|
Make the automated import guard enforce the complete domain-first dependency
|
|
policy for production code, including the framework, the application
|
|
composition root, and the integration-test-only package.
|
|
|
|
### Required Changes
|
|
|
|
1. Extend `internal/modules/import_boundaries_test.go` so validation considers
|
|
both the importing file's repository-relative path and whether it is a test
|
|
file. Do not return early merely because the importer is outside
|
|
`internal/modules`.
|
|
2. Treat `internal/modules/integration` as test infrastructure, not as a module
|
|
family or a production dependency target:
|
|
|
|
- reject every import of `internal/modules/integration` from a non-test Go
|
|
file; and
|
|
- continue allowing designated black-box tests under
|
|
`internal/modules/integration` to import concrete families for composition
|
|
coverage.
|
|
|
|
3. Reject imports of any `internal/modules/**` package from non-test files
|
|
under `internal/framework/**` or `internal/core/**`. These packages define
|
|
inward framework and core layers and must remain independent of all module
|
|
implementations, including `generic`.
|
|
4. For non-test files under the production CLI composition root
|
|
`internal/cli/**`, allow module imports only when the target is the exact
|
|
family registrar package `internal/modules/<family>/register`. Reject direct
|
|
imports of family roots, domain leaves, adapters, and generic implementation
|
|
leaves. Apply the same rule automatically to newly added families.
|
|
5. Preserve test-only assembly allowances needed for black-box and
|
|
compatibility coverage. In particular, `_test.go` files in the CLI,
|
|
framework, and core trees may import module packages, and designated
|
|
integration black-box tests may compose concrete families. These allowances
|
|
must be based on the importing file being a test file, not on a directory
|
|
exemption that production files could inherit.
|
|
6. Retain all Stage 4 within-module rules: concrete peer isolation,
|
|
registrar-only concrete-to-generic composition, family-root direction, and
|
|
same-family child relationships. Keep path parsing structural so new module
|
|
families receive the rules without a hard-coded family list.
|
|
7. Improve failure messages to name the importing file, import target, and the
|
|
applicable boundary rule.
|
|
|
|
### Tests
|
|
|
|
Add table-driven fixtures proving that the guard:
|
|
|
|
- rejects production imports of `internal/modules/integration` from module and
|
|
non-module packages;
|
|
- rejects production framework and core imports of concrete and generic module
|
|
packages;
|
|
- accepts an exact registrar import from production CLI code;
|
|
- rejects production CLI imports of a family root, concrete leaf, generic
|
|
leaf, or other non-registrar module package;
|
|
- accepts equivalent direct imports from `_test.go` compatibility tests;
|
|
- accepts designated black-box integration-test composition while rejecting a
|
|
production `.go` file in the same directory; and
|
|
- retains every cross-family, registrar, and family-root case covered by Stage
|
|
4.
|
|
|
|
Run the guard against the full repository and ensure all current production
|
|
imports comply without adding path-specific exemptions.
|
|
|
|
### Documentation
|
|
|
|
Update `docs/internal/modules.md` to distinguish production composition through
|
|
family registrars from test-only direct composition, and to state that
|
|
`internal/modules/integration` is not a production dependency target. Update
|
|
other current-behavior documentation only if it describes broader composition
|
|
permissions.
|
|
|
|
### Completion Gate
|
|
|
|
A new production file cannot bypass the domain-first dependency rules by being
|
|
placed outside `internal/modules`, under `internal/modules/integration`, or in
|
|
the CLI composition tree. Test-only allowances must remain explicit and
|
|
file-scoped.
|
|
|
|
## Final Verification And Closeout
|
|
|
|
After Stages 6 through 8:
|
|
|
|
1. Run the full validation commands from this document on a clean worktree.
|
|
2. Exercise the maintained D&D production example with both a fresh workspace
|
|
and checkpoint resume.
|
|
3. Verify that a merge or normalize candidate rejected by typed validation is
|
|
not passed to final encoding or checkpoint creation.
|
|
4. Verify that every executed chunk, extract, merge, and normalize retry has a
|
|
terminal attempt envelope with correctly scoped LLM-call data.
|
|
5. Confirm current production imports satisfy the strengthened framework,
|
|
registrar-only CLI, integration-target, and module-family boundary rules.
|
|
6. Re-read current-behavior documentation for statements made true or obsolete
|
|
by these stages.
|
|
7. Replace this roadmap's status with a concise completion record only after all
|
|
stages and documentation updates have landed.
|
|
|
|
## Open Questions
|
|
|
|
None. The implementation choices required for Stages 6 through 8 are specified
|
|
above. Final roadmap closeout is intentionally deferred until those stages are
|
|
complete.
|