Document accepted chunk map export

This commit is contained in:
2026-07-23 15:06:33 +00:00
parent 16a998055c
commit 06148074a2
13 changed files with 228 additions and 15 deletions

View File

@@ -391,6 +391,12 @@ safe logical names, pretty-prints JSON payloads, and assembles the logical index
manifest, rejected-result, warning, and lane files. Invalid JSON, unsupported
media types, unsafe names, and sanitized-name collisions are errors.
Its strict `include_chunk_map` option is disabled by default. When enabled, it
validates the framework-supplied accepted chunk map through its codec and adds
the pipeline-wide `chunk-map.json` plus its index descriptor; it does not treat
the map as a lane payload. The external shape is owned by the
[Accepted Chunk Map contract](../integrations/chunk-map.md).
The encoder returns logical files only. The CLI places them on disk, and the
[JSON output contract](../integrations/json-output.md) defines their external
paths and schemas.

View File

@@ -50,6 +50,7 @@ required normalized artifacts have crossed the typed handoff.
| `internal/framework/promptfs` | Builds module prompt filesystems from module-owned and caller-provided shared prompt assets. |
| `internal/framework/checkpoint` | Root-based checkpoint loading, recording, identity, and payload serialization. |
| `internal/framework/chunkplan` | Source-addressed chunk-plan filesystem storage, envelope validation, and atomic publication. |
| `internal/framework/chunkmap` | Strict durable accepted chunk-map construction, schema, validation, cloning, and serialization. |
| `internal/framework/debug` | Root-based framework and LLM debug recording. |
Framework contracts provide typed artifact, provenance-wrapper, chunk-validator,
@@ -109,7 +110,7 @@ Configuration. The implemented module packages are:
| `internal/modules/generic/normalize/noop` | Preserves accepted merged output. |
| `internal/modules/dnd/normalize/spells` | Canonicalizes catalog-backed spell names and exact source references, conservatively collapses duplicate casts, and reports deterministic warnings and independently scoped catalog checkpoint identity. |
| `internal/modules/dnd/normalize/npcs` | Consolidates NPC records deterministically by canonical name, unions exact evidence, and reports bounded warnings. |
| `internal/modules/generic/output/json` | Encodes manifests, lane payloads, warnings, and rejections as logical JSON files. |
| `internal/modules/generic/output/json` | Encodes manifests, lane payloads, warnings, rejections, and an explicitly enabled accepted chunk map as logical JSON files. |
`internal/modules/dnd/shared` owns reusable D&D prompt fragments,
reference declarations, prompt input assembly, source-unit reference helpers,

View File

@@ -208,15 +208,18 @@ The runner:
3. selects a stored plan or executes the configured chunker's `Plan` operation;
4. canonicalizes and materializes the plan, then validates the resulting
chunks;
5. executes each resolved step in configuration order. For one step, it
5. builds the framework-owned accepted chunk map from the accepted source,
logical plan, and exact materialized chunks, then supplies it to the output
request independently of output-module options;
6. executes each resolved step in configuration order. For one step, it
dispatches extract jobs in source-chunk then resolved-lane order, starts a
bounded lane continuation when all extracts for that lane are terminal, and
waits for every lane to become terminal;
6. encodes and validates each accepted normalized producer artifact, then
7. encodes and validates each accepted normalized producer artifact, then
builds the immutable generated reference sets for the next step;
7. invokes the prepared output encoder only after every step succeeds and
8. invokes the prepared output encoder only after every step succeeds and
validates its logical file results;
8. returns the assembled manifest, outcomes, warnings, and files.
9. returns the assembled manifest, outcomes, warnings, and files.
Within each artifact lane, it reuses the prepared extractor, merger, normalizer,
and validators while performing these transitions:
@@ -265,6 +268,14 @@ currently requested chunker and the effective plan producer. Cache state and
paths are configured and operated outside the runner; see
[Configuration](../config.md#state-surfaces) and [Operations](../operations.md).
For an accepted plan, the runner also constructs the strict framework-owned
[Accepted Chunk Map](../integrations/chunk-map.md) before lane execution. It
uses the current resolved chunker as `requested_chunker` and the stored or
generated record as `producer`, preserving that distinction on reuse. Chunk
rejection supplies no map; later lane rejection does not discard it. Output
encoders receive a defensively owned serialized value and may explicitly
ignore it.
The extract job channel has the same capacity as the effective extract worker
count, so dispatch applies backpressure. A fixed continuation executor prevents
ready or checkpoint-reused lanes from creating one goroutine each. Workers and

View File

@@ -17,6 +17,11 @@ debug roots.
The pipeline runner returns logical output files. After validating every
logical name, the CLI exclusively creates the run directory beneath the
selected output root and performs confined, atomic file writes within it.
The runner supplies an accepted chunk map as an optional, defensively owned
output-request artifact. The JSON encoder alone decides whether its explicit
option writes the map and optional index descriptor; neither the map payload
nor its annotations are copied into the run manifest. The durable fields are
owned by the [Accepted Chunk Map contract](../integrations/chunk-map.md).
`internal/framework/chunkplan` owns source-addressed plan storage, validation,
and atomic publication. Its store is constructed only when the selected mode is