Finalize bounded typed pipeline implementation

This commit is contained in:
2026-07-17 08:49:08 +00:00
parent adfd3bd052
commit 3013ee044d
11 changed files with 369 additions and 857 deletions

View File

@@ -142,7 +142,8 @@ The runner:
source document;
3. obtains or executes the chunk result;
4. validates and canonicalizes chunks;
5. executes each resolved artifact lane in order;
5. dispatches extract jobs in source-chunk then resolved-lane order, starting a
bounded lane continuation when all extracts for that lane are terminal;
6. invokes the prepared output encoder and validates its logical file results;
7. returns the assembled manifest, outcomes, warnings, and files.
@@ -161,6 +162,13 @@ and validators while performing these transitions:
Module-provided warnings and payload warnings are promoted only from attempts
whose results are accepted and used.
The extract job channel has the same capacity as the effective extract worker
count, so dispatch applies backpressure. A fixed continuation executor prevents
ready or checkpoint-reused lanes from creating one goroutine each. Workers and
continuations publish lane-local results; the coordinator is the only writer of
aggregate output and merges those results in resolved lane and source-chunk
order.
## Chunk Canonicalization
Before lane execution, generic validation requires unique chunk IDs, matching
@@ -234,6 +242,12 @@ time. Successful status reflects whether any result was rejected. The
durable manifest and logical file schemas are defined in the
[JSON output contract](../integrations/json-output.md).
On a framework failure, the runner cancels its derived context, stops submitting
new extract work, drains started tasks, and skips the output encoder. Parent
cancellation takes precedence. Otherwise context-cancellation fallout is
discarded when a substantive error exists, and the primary error is selected by
stage, resolved lane, and source chunk rather than completion time.
## Tests To Inspect
- `internal/core/config/effective_config_test.go`: config-to-resolution boundary.
@@ -244,6 +258,9 @@ durable manifest and logical file schemas are defined in the
- `internal/framework/pipeline/typed_resolution_test.go`: heterogeneous typed
lane resolution and preparation, target-specific validators,
incompatibilities, ordering, and schema-sensitive pipeline identity.
- `internal/framework/pipeline/runner_concurrency_test.go`: bounded dispatch and
continuations, reverse completion, stable errors, rejection, cancellation,
retries, and independent provider-call limits.
- `internal/framework/pipeline/preparation_test.go`: option validation,
construction order, dependency failures, and the before-source-work boundary.
- `internal/framework/pipeline/references_test.go`: target resolution and