Harmonize foundational integration contracts
This commit is contained in:
@@ -1,201 +1,111 @@
|
||||
# JSON Output
|
||||
# Published JSON Output
|
||||
|
||||
This document is the durable JSON output file-format contract produced by the
|
||||
production JSON encoder and written by the CLI. Selectable output-encoder keys
|
||||
are cataloged in
|
||||
[Configuration](../config.md#implemented-production-modules).
|
||||
This document defines the logical JSON bundle emitted by the production JSON
|
||||
output encoder. The bundle’s physical destination, atomic publication, and
|
||||
retention are operational concerns; see [Operations](../operations.md#output-bundles).
|
||||
Output configuration, including chunk-map export, belongs in
|
||||
[Configuration](../config.md#module-bindings-and-validators).
|
||||
|
||||
The output module produces the logical bundle described here. The CLI's
|
||||
physical placement and lifecycle for that bundle are defined in
|
||||
[Operations](../operations.md#output-directory).
|
||||
## Bundle Layout
|
||||
|
||||
## Files
|
||||
All paths below are logical, relative, slash-separated bundle paths. The
|
||||
encoder always emits the first four JSON files below and adds lane or chunk-map
|
||||
files when their corresponding artifacts are available:
|
||||
|
||||
The encoder writes:
|
||||
| Path | Purpose |
|
||||
| --- | --- |
|
||||
| `index.json` | Entry point that names the other published files and lane payloads. |
|
||||
| `manifest.json` | Run provenance and result summaries. |
|
||||
| `rejected.json` | Rejected pipeline outputs. |
|
||||
| `warnings.json` | Accepted-output and run warnings. |
|
||||
| `lanes/<safe-lane-id>.json` | One normalized artifact payload for each lane. |
|
||||
| `chunk-map.json` | Optional accepted chunk map, when its export is enabled and available. |
|
||||
|
||||
- `index.json`
|
||||
- `manifest.json`
|
||||
- `lanes/<lane-id>.json`, one file per normalized serialized artifact
|
||||
- `rejected.json`
|
||||
- `warnings.json`
|
||||
- `chunk-map.json`, only when the JSON output binding enables
|
||||
`include_chunk_map` and the run has an accepted chunk map
|
||||
|
||||
Files are pretty-printed JSON with a trailing newline when the payload is JSON.
|
||||
Logical file paths are relative, slash-separated, and may not contain `..`.
|
||||
JSON files are pretty-printed with a trailing newline. Lane payloads are
|
||||
accepted only when their media type is `application/json`.
|
||||
|
||||
## `index.json`
|
||||
|
||||
Shape:
|
||||
`index.json` is the bundle’s discovery document. An approved run with no
|
||||
normalized lanes has this valid minimal index:
|
||||
|
||||
```json
|
||||
{
|
||||
"manifest_file": "manifest.json",
|
||||
"output_files": [
|
||||
{
|
||||
"lane_id": "spells",
|
||||
"media_type": "application/json",
|
||||
"file": "lanes/spells.json",
|
||||
"module_key": "noop",
|
||||
"schema_id": "notarius.dnd.spells",
|
||||
"schema_name": "notarius_dnd_spells_v1",
|
||||
"schema_version": "v1"
|
||||
}
|
||||
],
|
||||
"output_files": [],
|
||||
"rejected_file": "rejected.json",
|
||||
"warnings_file": "warnings.json"
|
||||
}
|
||||
```
|
||||
|
||||
`output_files` is sorted by lane ID. Output file names are produced by
|
||||
sanitizing the lane ID:
|
||||
| Field | Required | Meaning |
|
||||
| --- | --- | --- |
|
||||
| `manifest_file` | Yes | Always `manifest.json`. |
|
||||
| `output_files` | Yes | Lane descriptors sorted by `lane_id`. |
|
||||
| `rejected_file` | Yes | Always `rejected.json`. |
|
||||
| `warnings_file` | Yes | Always `warnings.json`. |
|
||||
| `chunk_map` | No | Descriptor for the pipeline-wide `chunk-map.json`; never a lane descriptor. |
|
||||
|
||||
- characters outside `A-Z`, `a-z`, `0-9`, `.`, `_`, and `-` become `_`;
|
||||
- repeated `..` sequences are replaced;
|
||||
- leading and trailing `.`, `_`, and `-` are trimmed;
|
||||
- empty sanitized names are rejected;
|
||||
- two lanes that sanitize to the same output file are rejected.
|
||||
|
||||
`manifest_file`, `rejected_file`, and `warnings_file` contain the fixed paths
|
||||
shown above. Each `output_files` entry requires `lane_id` and `file`. It also
|
||||
contains the normalized payload `media_type`, normalizer `module_key`, and
|
||||
response `schema_id`, `schema_name`, and `schema_version` when those values are
|
||||
available.
|
||||
|
||||
When present, the top-level optional `chunk_map` descriptor contains exactly
|
||||
Each lane descriptor has required `lane_id` and `file`. It may also include
|
||||
`media_type`, `module_key`, `schema_id`, `schema_name`, and `schema_version`
|
||||
when supplied by the normalized artifact. A `chunk_map` descriptor contains
|
||||
`artifact_kind`, `file`, `media_type`, `schema_id`, `schema_name`, and
|
||||
`schema_version`. It identifies the pipeline-wide `chunk-map.json`; it is not
|
||||
a lane output and never appears in `output_files`. The descriptor and file are
|
||||
both absent when export is disabled or no chunk plan was accepted. Its payload
|
||||
contract is defined by [Accepted Chunk Map](chunk-map.md).
|
||||
`schema_version`; its payload is defined by the
|
||||
[Accepted Chunk Map contract](chunk-map.md).
|
||||
|
||||
The lane path is derived from its lane ID. Characters outside letters, digits,
|
||||
periods, underscores, and hyphens become underscores; `..` sequences are
|
||||
neutralized; leading and trailing periods and underscores are removed. A lane
|
||||
that produces an empty name, or two lanes that produce the same path, makes
|
||||
output encoding fail.
|
||||
|
||||
## Lane Payloads
|
||||
|
||||
Each `lanes/<safe-lane-id>.json` file is the codec-owned normalized JSON for
|
||||
that lane. Consumers should use the index descriptor’s schema identity rather
|
||||
than infer a lane schema from its name. The current D&D payload contracts are
|
||||
[spells](dnd-spell-artifacts.md), [NPCs](dnd-npc-artifacts.md),
|
||||
[NPC interactions](dnd-npc-interaction-artifacts.md),
|
||||
[combat turns](dnd-combat-turn-artifacts.md),
|
||||
[item events](dnd-item-event-artifacts.md), and
|
||||
[scene descriptions](dnd-scene-description-artifacts.md).
|
||||
|
||||
## `manifest.json`
|
||||
|
||||
`manifest.json` contains a run manifest. This abridged example shows its core
|
||||
structure:
|
||||
`manifest.json` is published provenance, not a copy of lane payloads or a
|
||||
checkpoint store. Fields without a value may be omitted. Its top-level fields
|
||||
group into the following externally observable summaries:
|
||||
|
||||
```json
|
||||
{
|
||||
"run_id": "run-123",
|
||||
"pipeline_id": "dnd-session",
|
||||
"artifact_lanes": [
|
||||
{
|
||||
"id": "spells",
|
||||
"extractor": "dnd/spells",
|
||||
"merger": "appendorder",
|
||||
"normalizer": "noop"
|
||||
}
|
||||
],
|
||||
"validation_status": "approved",
|
||||
"started_at": "2026-01-01T00:00:00Z",
|
||||
"completed_at": "2026-01-01T00:00:01Z"
|
||||
}
|
||||
```
|
||||
| Group | Fields |
|
||||
| --- | --- |
|
||||
| Run identity and result | `run_id`, `pipeline_id`, `pipeline_digest`, `schema_version`, `validation_status`, `started_at`, `completed_at` |
|
||||
| Resolved components | `input_module`, `chunker`, `extractors`, `merger`, `normalizer`, `output_encoder`, `artifact_lanes`, `validator_chains`, `module_metadata` |
|
||||
| Source and references | `source_digests`, `references` |
|
||||
| Published result summaries | `normalized_outputs`, `rejected_outputs` |
|
||||
| Execution summaries | `chunk_plan`, `checkpoint_decisions`, `llm_profiles`, `metadata` |
|
||||
|
||||
Fields with empty values may be omitted by JSON encoding.
|
||||
`references` records provenance such as the target, slot, origin, digest,
|
||||
media type, size, and generated-artifact identity. It does not contain
|
||||
reference content. `normalized_outputs` and `rejected_outputs` likewise
|
||||
summarize results without embedding lane payload bytes. A chunk-plan summary is
|
||||
provenance for the plan used by this run; cache records, debug artifacts, and
|
||||
other operational state are not published as bundle files.
|
||||
|
||||
The manifest fields are:
|
||||
## Rejections And Warnings
|
||||
|
||||
- `run_id`, `pipeline_id`, and `pipeline_digest`: run and resolved-pipeline
|
||||
identity;
|
||||
- `input_module`, `chunker`, `extractors`, `merger`, `normalizer`, and
|
||||
`output_encoder`: resolved module keys;
|
||||
- `chunk_plan`: payload-free provenance for the effective chunk plan. `mode`
|
||||
is the effective cache mode; `action` is `reused`, `generated`,
|
||||
`refreshed`, or `bypassed` when a plan was materialized. `requested_module`
|
||||
is the current pipeline chunker, while `producer_input_module`,
|
||||
`producer_module`, `producer_llm_profile`, `producer_references`,
|
||||
`producer_metadata`, `source_digest`, `plan_digest`, `plan_schema_version`,
|
||||
and `created_at` describe the stored or generated producer when available.
|
||||
A cached plan can therefore identify a producer different from the requested
|
||||
module. This object never embeds ranges, units, annotations, prompts,
|
||||
responses, or reference content;
|
||||
- `module_metadata` and `artifact_lanes`: module and per-lane provenance,
|
||||
including prompt and response-schema provenance when provided;
|
||||
- `validator_chains`: resolved validation points and validators;
|
||||
- `source_digests` and `references`: source and reference provenance;
|
||||
- `normalized_outputs` and `rejected_outputs`: payload-free result summaries;
|
||||
- `llm_profiles`: selected profile IDs and provider or model names when
|
||||
available;
|
||||
- `metadata`: the effective prompt `session_id`;
|
||||
- `validation_status`: `approved` or `rejected`;
|
||||
- `started_at` and `completed_at`: UTC run timestamps.
|
||||
`rejected.json` is always an object with a `rejected` array. Each entry has
|
||||
required `stage` and `message`; `step_id`, `lane_id`, `module_key`, `chunk_id`,
|
||||
`chunk_index`, `validator_name`, `reason_code`, `attempt_count`, and
|
||||
`diagnostic_artifact_path` are present only when applicable.
|
||||
|
||||
`source_digests` contains source document digests only. Bound references are
|
||||
recorded separately under `references`, which contains provenance only: target
|
||||
stage, lane ID when present, slot name, origin type and URI, digest, media
|
||||
type, byte size, and binding source. Reference content is not written to
|
||||
durable output.
|
||||
`warnings.json` is always an object with a `warnings` array. Each warning has
|
||||
`reason_code` and `message`; `scope` is optional. Both arrays are empty when
|
||||
there is nothing to report.
|
||||
|
||||
Reference `stage` is `chunk`, `extract`, `merge`, or `normalize`. `lane_id` is
|
||||
omitted for chunk references and present for extract, merge, and normalize
|
||||
references.
|
||||
## Compatibility
|
||||
|
||||
`validation_status` is `approved` when no outputs were rejected and `rejected`
|
||||
when one or more outputs were rejected.
|
||||
|
||||
Producer warnings and the current run's chunk-validation warnings remain in
|
||||
`warnings.json`. The manifest records only provenance and decision summaries;
|
||||
empty producer-only values are omitted for compatibility with existing readers.
|
||||
|
||||
`validator_chains` records the resolved validator chain for each validation
|
||||
point. Entries include stage, lane ID when applicable, module key, and validators
|
||||
with key and execution class. Empty chains are recorded with an empty
|
||||
`validators` array, including chains resolved from explicit empty config
|
||||
overrides.
|
||||
|
||||
`normalized_outputs` summarizes each normalized lane output without embedding
|
||||
payload bytes. Entries include lane ID, normalizer module key, source ID, media
|
||||
type, and response schema provenance where available.
|
||||
|
||||
`rejected_outputs` summarizes rejected module outputs without embedding raw
|
||||
payload bytes. Entries include stage, lane, module, chunk, validator or reason,
|
||||
message, attempt count, and optional diagnostic artifact path.
|
||||
|
||||
## Output Payload Files
|
||||
|
||||
Each normalized serialized artifact is written to
|
||||
`lanes/<sanitized-lane-id>.json`. The JSON output encoder is domain-neutral and
|
||||
accepts only artifacts whose codec media type is `application/json`. The file
|
||||
contains the codec-owned JSON bytes pretty-printed.
|
||||
|
||||
The schema of each lane payload is owned by that artifact contract. For the
|
||||
current D&D lanes, see [D&D Spell Artifact](dnd-spell-artifacts.md),
|
||||
[D&D NPC Artifact](dnd-npc-artifacts.md), and
|
||||
[D&D Combat-Turn Artifact](dnd-combat-turn-artifacts.md), and
|
||||
[D&D Scene Description Artifact](dnd-scene-description-artifacts.md).
|
||||
|
||||
## `rejected.json`
|
||||
|
||||
Shape:
|
||||
|
||||
```json
|
||||
{
|
||||
"rejected": []
|
||||
}
|
||||
```
|
||||
|
||||
When output validation rejects an output, each entry contains `stage` and
|
||||
`message`. It includes `lane_id`, `module_key`, `chunk_id`, `chunk_index`,
|
||||
`validator_name`, `reason_code`, `attempt_count`, and
|
||||
`diagnostic_artifact_path` when applicable.
|
||||
|
||||
## `warnings.json`
|
||||
|
||||
Shape:
|
||||
|
||||
```json
|
||||
{
|
||||
"warnings": [
|
||||
{
|
||||
"scope": "extract",
|
||||
"reason_code": "example",
|
||||
"message": "human-readable warning"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
`warnings` is an empty array when no warnings are reported.
|
||||
Each warning requires `reason_code` and `message`; `scope` is omitted when it is
|
||||
empty.
|
||||
The index is the authoritative map from a logical lane to its published
|
||||
payload. Consumers must tolerate omitted optional manifest and descriptor
|
||||
fields, and should rely on the linked artifact contract for each lane’s JSON
|
||||
shape. This contract describes the published logical bundle only; it does not
|
||||
promise a filesystem layout or expose internal state formats.
|
||||
|
||||
Reference in New Issue
Block a user