82 lines
3.6 KiB
Markdown
82 lines
3.6 KiB
Markdown
# Accepted Chunk Map
|
|
|
|
This document defines the optional durable `chunk-map.json` artifact in a
|
|
[published JSON bundle](json-output.md). It describes the accepted,
|
|
materialized chunk plan used by one run. It is not a lane payload and is never
|
|
an input to a later pipeline step.
|
|
|
|
## Contract Identity
|
|
|
|
| Property | Value |
|
|
| --- | --- |
|
|
| Artifact kind | `source/chunk-map` |
|
|
| Logical file | `chunk-map.json` |
|
|
| Media type | `application/json` |
|
|
| Schema ID | `notarius.source.chunk_map` |
|
|
| Schema name | `notarius_source_chunk_map_v1` |
|
|
| Schema version | `v1` |
|
|
|
|
The optional `chunk_map` descriptor in `index.json` identifies this artifact.
|
|
Export is controlled by the JSON output binding described in
|
|
[Configuration](../config.md#module-bindings-and-validators).
|
|
|
|
## Wire Shape
|
|
|
|
Every payload has these required fields:
|
|
|
|
| Field | Meaning |
|
|
| --- | --- |
|
|
| `source_id` | Accepted source-document identity. |
|
|
| `source_digest` | Lower-case `sha256:` digest of that source document. |
|
|
| `plan_digest` | Lower-case `sha256:` digest of the logical chunk plan. |
|
|
| `requested_chunker` | Chunk module selected by the resolved pipeline. |
|
|
| `producer` | Original accepted-plan producer. `input_module` and `chunk_module` are required; `llm_profile` is optional. |
|
|
| `plan_annotations` | Plan-level annotation namespace map; `{}` when none are present. |
|
|
| `chunks` | Non-empty execution-order chunk collection. |
|
|
|
|
Each `chunks` entry contains non-empty `id`, zero-based `index`, `source_ref`,
|
|
positive `unit_count`, and an explicit `annotations` map. `source_ref` contains
|
|
the same `source_id` as the top-level value plus positive inclusive
|
|
`start_unit_id` and `end_unit_id` values. Endpoints identify source units; their
|
|
numeric values do not by themselves establish source-document order.
|
|
|
|
Annotation namespaces are non-empty trimmed strings. Their values are arbitrary
|
|
valid JSON and are retained without interpreting a module-specific namespace.
|
|
|
|
## Ordering And Validation
|
|
|
|
`chunks` are in execution order. Their indexes are contiguous, start at zero,
|
|
and equal their array positions; chunk IDs are unique. The emitted map is built
|
|
only after the selected plan has been accepted and materialized against the
|
|
source document, so its ranges, unit counts, annotations, and digests describe
|
|
that exact plan.
|
|
|
|
The codec rejects malformed JSON, trailing content, unknown fixed-object
|
|
fields, invalid identities or digests, invalid annotations, duplicate chunk
|
|
IDs, non-contiguous indexes, and a `plan_digest` that does not match the
|
|
reconstructed logical plan. The checked-in
|
|
[schema](../../internal/framework/chunkmap/assets/schemas/source_chunk_map.v1.json)
|
|
defines the strict JSON shape.
|
|
|
|
## Valid Example
|
|
|
|
The compact
|
|
[source chunk-map fixture](../../internal/framework/chunkmap/testdata/source_chunk_map.v1.json)
|
|
is decoded by the production codec and demonstrates an accepted map with
|
|
annotations, producer identity, and ordered chunks.
|
|
|
|
## Publication And Compatibility
|
|
|
|
The map is present only when a chunk plan was accepted and its export is
|
|
enabled. It remains publishable if a later lane is rejected, but is absent when
|
|
chunk-plan validation rejects the plan. `requested_chunker` identifies the
|
|
current pipeline selection, while `producer` identifies the component that
|
|
originally produced the accepted plan; they may differ when an accepted plan is
|
|
reused.
|
|
|
|
The map contains structure rather than source content: it excludes transcript
|
|
bytes, source-unit metadata, chunk text, private model output, reference
|
|
content, debug data, and filesystem paths. Treat the exported map with the
|
|
same care as other published output. Publication location and retention are
|
|
defined in [Operations](../operations.md#output-bundles).
|