Harmonize foundational integration contracts
This commit is contained in:
@@ -1,83 +1,81 @@
|
||||
# Accepted Chunk Map
|
||||
|
||||
This document defines the durable accepted chunk-map artifact that the JSON
|
||||
output encoder can write as `chunk-map.json`. It describes the exact accepted,
|
||||
materialized chunks used by a run; it is not a lane artifact and is never an
|
||||
input to later pipeline steps. Enable it with the JSON output option described
|
||||
in [Configuration](../config.md#json-output-options).
|
||||
This document defines the optional durable `chunk-map.json` artifact in a
|
||||
[published JSON bundle](json-output.md). It describes the accepted,
|
||||
materialized chunk plan used by one run. It is not a lane payload and is never
|
||||
an input to a later pipeline step.
|
||||
|
||||
## Identity
|
||||
## Contract Identity
|
||||
|
||||
- Artifact kind: `source/chunk-map`
|
||||
- Logical file: `chunk-map.json`
|
||||
- Schema ID: `notarius.source.chunk_map`
|
||||
- Schema name: `notarius_source_chunk_map_v1`
|
||||
- Schema version: `v1`
|
||||
- Media type: `application/json`
|
||||
| Property | Value |
|
||||
| --- | --- |
|
||||
| Artifact kind | `source/chunk-map` |
|
||||
| Logical file | `chunk-map.json` |
|
||||
| Media type | `application/json` |
|
||||
| Schema ID | `notarius.source.chunk_map` |
|
||||
| Schema name | `notarius_source_chunk_map_v1` |
|
||||
| Schema version | `v1` |
|
||||
|
||||
The checked-in [JSON Schema](../../internal/framework/chunkmap/assets/schemas/source_chunk_map.v1.json)
|
||||
defines the strict wire shape. `chunk-map.json` is listed by the optional
|
||||
`chunk_map` descriptor in [the JSON output index](json-output.md#indexjson),
|
||||
not by the lane-oriented `output_files` collection.
|
||||
The optional `chunk_map` descriptor in `index.json` identifies this artifact.
|
||||
Export is controlled by the JSON output binding described in
|
||||
[Configuration](../config.md#module-bindings-and-validators).
|
||||
|
||||
## Payload
|
||||
## Wire Shape
|
||||
|
||||
The payload has these required fields:
|
||||
Every payload has these required fields:
|
||||
|
||||
- `source_id`: accepted source-document identity.
|
||||
- `source_digest`: canonical lower-case `sha256:` digest of that document.
|
||||
- `plan_digest`: canonical lower-case `sha256:` digest of the accepted logical
|
||||
plan.
|
||||
- `requested_chunker`: chunk module selected by the current resolved pipeline.
|
||||
- `producer`: the original accepted-plan producer, with required
|
||||
`input_module` and `chunk_module`; `llm_profile` is present only for an
|
||||
LLM-backed producer.
|
||||
- `plan_annotations`: accepted plan-level annotation namespace map. It is
|
||||
`{}` when no namespaces are present.
|
||||
- `chunks`: non-empty execution-order collection of accepted chunks.
|
||||
| Field | Meaning |
|
||||
| --- | --- |
|
||||
| `source_id` | Accepted source-document identity. |
|
||||
| `source_digest` | Lower-case `sha256:` digest of that source document. |
|
||||
| `plan_digest` | Lower-case `sha256:` digest of the logical chunk plan. |
|
||||
| `requested_chunker` | Chunk module selected by the resolved pipeline. |
|
||||
| `producer` | Original accepted-plan producer. `input_module` and `chunk_module` are required; `llm_profile` is optional. |
|
||||
| `plan_annotations` | Plan-level annotation namespace map; `{}` when none are present. |
|
||||
| `chunks` | Non-empty execution-order chunk collection. |
|
||||
|
||||
Each chunk has `id`, zero-based `index`, `source_ref`, positive `unit_count`,
|
||||
and an explicit `annotations` namespace map. A source reference has the source
|
||||
ID and inclusive positive `start_unit_id` and `end_unit_id` endpoints.
|
||||
Annotation values are arbitrary valid JSON under non-empty namespaces. They
|
||||
are preserved as canonical JSON without interpreting any module-specific
|
||||
namespace.
|
||||
Each `chunks` entry contains non-empty `id`, zero-based `index`, `source_ref`,
|
||||
positive `unit_count`, and an explicit `annotations` map. `source_ref` contains
|
||||
the same `source_id` as the top-level value plus positive inclusive
|
||||
`start_unit_id` and `end_unit_id` values. Endpoints identify source units; their
|
||||
numeric values do not by themselves establish source-document order.
|
||||
|
||||
## Invariants
|
||||
Annotation namespaces are non-empty trimmed strings. Their values are arbitrary
|
||||
valid JSON and are retained without interpreting a module-specific namespace.
|
||||
|
||||
The framework constructs this artifact only after materializing the selected
|
||||
logical plan and accepting it through the configured chunk validator chain.
|
||||
Construction proves the source and plan digests, source-document range order,
|
||||
materialized chunk IDs and indexes, source references, unit membership and
|
||||
counts, and plan/range annotations agree exactly. Chunk IDs are unique and
|
||||
indexes are contiguous and agree with array order.
|
||||
## Ordering And Validation
|
||||
|
||||
The codec rejects unknown fixed-object fields, malformed identities or
|
||||
digests, invalid annotation JSON, trailing JSON content, and any payload whose
|
||||
reconstructed logical plan does not reproduce `plan_digest`. It makes
|
||||
defensive copies at serialization and decoding boundaries.
|
||||
`chunks` are in execution order. Their indexes are contiguous, start at zero,
|
||||
and equal their array positions; chunk IDs are unique. The emitted map is built
|
||||
only after the selected plan has been accepted and materialized against the
|
||||
source document, so its ranges, unit counts, annotations, and digests describe
|
||||
that exact plan.
|
||||
|
||||
## Acceptance And Provenance
|
||||
The codec rejects malformed JSON, trailing content, unknown fixed-object
|
||||
fields, invalid identities or digests, invalid annotations, duplicate chunk
|
||||
IDs, non-contiguous indexes, and a `plan_digest` that does not match the
|
||||
reconstructed logical plan. The checked-in
|
||||
[schema](../../internal/framework/chunkmap/assets/schemas/source_chunk_map.v1.json)
|
||||
defines the strict JSON shape.
|
||||
|
||||
The artifact is available only when the chunk plan was accepted. It remains
|
||||
available when a later extraction, merge, or normalization result is rejected;
|
||||
it is absent when chunk validation rejects the candidate plan.
|
||||
## Valid Example
|
||||
|
||||
`requested_chunker` describes the current pipeline selection. `producer`
|
||||
describes who originally produced the accepted plan. On a cache hit these can
|
||||
differ: the accepted ranges, annotations, digests, and stable materialized IDs
|
||||
are reused, while the producer remains the stored producer. Cache paths,
|
||||
actions, references, metadata, warnings, timestamps, and detailed provenance
|
||||
remain in the run manifest rather than this payload.
|
||||
The compact
|
||||
[source chunk-map fixture](../../internal/framework/chunkmap/testdata/source_chunk_map.v1.json)
|
||||
is decoded by the production codec and demonstrates an accepted map with
|
||||
annotations, producer identity, and ordered chunks.
|
||||
|
||||
## Data Handling
|
||||
## Publication And Compatibility
|
||||
|
||||
The map contains structure, not source content. It excludes transcript bytes,
|
||||
materialized units, source-unit metadata, chunk content, private model
|
||||
responses, rejected proposals, debug data, external-reference content, and
|
||||
filesystem paths.
|
||||
The map is present only when a chunk plan was accepted and its export is
|
||||
enabled. It remains publishable if a later lane is rejected, but is absent when
|
||||
chunk-plan validation rejects the plan. `requested_chunker` identifies the
|
||||
current pipeline selection, while `producer` identifies the component that
|
||||
originally produced the accepted plan; they may differ when an accepted plan is
|
||||
reused.
|
||||
|
||||
Annotations can nevertheless be source- or model-derived. Treat an enabled
|
||||
`chunk-map.json` with the same sensitivity and retention expectations as lane
|
||||
output. Physical placement, confined atomic writing, and permissions follow
|
||||
the ordinary [output operation](../operations.md#output).
|
||||
The map contains structure rather than source content: it excludes transcript
|
||||
bytes, source-unit metadata, chunk text, private model output, reference
|
||||
content, debug data, and filesystem paths. Treat the exported map with the
|
||||
same care as other published output. Publication location and retention are
|
||||
defined in [Operations](../operations.md#output-bundles).
|
||||
|
||||
@@ -1,19 +1,27 @@
|
||||
# D&D Spell-Catalog Overlay Contract
|
||||
# D&D Spell-Catalog Overlays
|
||||
|
||||
This document defines the JSON format accepted by the D&D spell catalog
|
||||
resolver. An overlay supplies campaign-specific spell names and aliases for
|
||||
recognition. It does not supply spell rules, levels, classes, effects, or
|
||||
source evidence.
|
||||
This document defines the optional JSON overlay consumed by the D&D spell
|
||||
extractor. An overlay contributes campaign spell names and aliases for
|
||||
recognition. It does not define spell rules, effects, levels, classes, or
|
||||
transcript evidence. Bind the optional `spell_catalog` reference as described
|
||||
in [Configuration](../config.md#references-and-ordered-handoffs).
|
||||
|
||||
The `dnd/spells` extractor accepts one optional UTF-8 `application/json` overlay
|
||||
bundle through its `spell_catalog` reference slot. The framework materializes
|
||||
that file relative to the configuration or command-line binding, enforces the
|
||||
1 MiB slot limit, and records its origin and raw digest separately from the
|
||||
effective catalog digest.
|
||||
## Contract Identity
|
||||
|
||||
## Shape
|
||||
| Property | Value |
|
||||
| --- | --- |
|
||||
| Consumer | D&D spell extraction and normalization |
|
||||
| Reference slot | `spell_catalog` |
|
||||
| Media type | `application/json` |
|
||||
| Required schema version | `notarius.dnd.spell-catalog-overlay.v1` |
|
||||
| Base catalog | Embedded D&D 5e 2014 SRD catalog |
|
||||
|
||||
An overlay bundle has this shape:
|
||||
At most one overlay document may be bound. The maintained example is
|
||||
[dnd-spell-catalog.json](../../examples/dnd-spell-catalog.json).
|
||||
|
||||
## Wire Shape
|
||||
|
||||
This is a minimal valid overlay:
|
||||
|
||||
```json
|
||||
{
|
||||
@@ -22,49 +30,43 @@ An overlay bundle has this shape:
|
||||
{
|
||||
"id": "campaign.example",
|
||||
"ruleset": "dnd-5e-2014",
|
||||
"source": {
|
||||
"title": "Example campaign spells",
|
||||
"version": "1",
|
||||
"url": "",
|
||||
"license": ""
|
||||
},
|
||||
"spells": [
|
||||
{
|
||||
"name": "Aegis of Emberfall",
|
||||
"aliases": ["Emberfall Aegis"]
|
||||
}
|
||||
]
|
||||
"source": {"title": "Example campaign spells"},
|
||||
"spells": [{"name": "Aegis of Emberfall"}]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
The top-level `schema_version` and `catalogs` fields are required. The schema
|
||||
version must be exactly `notarius.dnd.spell-catalog-overlay.v1`, and at least
|
||||
one catalog is required. Catalogs require a unique, non-empty, trimmed `id`,
|
||||
the exact `dnd-5e-2014` `ruleset`, a `source`, and a non-empty `spells` array.
|
||||
| Field | Required | Meaning and constraints |
|
||||
| --- | --- | --- |
|
||||
| `schema_version` | Yes | Exactly `notarius.dnd.spell-catalog-overlay.v1`. |
|
||||
| `catalogs` | Yes | Non-empty array of catalog objects with unique IDs. |
|
||||
| `catalogs[].id` | Yes | Non-empty trimmed string. |
|
||||
| `catalogs[].ruleset` | Yes | Exactly `dnd-5e-2014`. |
|
||||
| `catalogs[].source.title` | Yes | Non-empty trimmed string. |
|
||||
| `catalogs[].source.version` | No | String when present. |
|
||||
| `catalogs[].source.url` | No | String when present. |
|
||||
| `catalogs[].source.license` | No | String when present. |
|
||||
| `catalogs[].spells` | Yes | Non-empty array of spell objects. |
|
||||
| `catalogs[].spells[].name` | Yes | Non-empty trimmed string. |
|
||||
| `catalogs[].spells[].aliases` | No | Array of non-empty trimmed strings when present. |
|
||||
|
||||
`source.title` is required and must be non-empty and trimmed. `source.version`,
|
||||
`source.url`, and `source.license` are optional strings and may be empty.
|
||||
Each spell requires a non-empty, trimmed `name`. `aliases` may be omitted or
|
||||
may be an array of trimmed, non-empty strings; JSON `null` is not an alias
|
||||
array. Overlay objects contain no other supported spell fields.
|
||||
Unknown fields are rejected at every object level. The document must contain
|
||||
one JSON value; `null` is not accepted for optional strings or aliases.
|
||||
|
||||
Decoding is strict: unknown fields, malformed JSON, trailing JSON values, and
|
||||
non-string optional source fields are rejected.
|
||||
## Composition And Compatibility
|
||||
|
||||
## Composition
|
||||
Notarius starts with the embedded base catalog, then applies overlay catalogs
|
||||
in ascending catalog-ID order. A new canonical spell name adds a recognition
|
||||
entry. If an overlay names an existing canonical spell, it augments that spell
|
||||
with aliases while retaining the established display spelling.
|
||||
|
||||
The resolver always starts with the embedded D&D 5e 2014 SRD catalog. Overlay
|
||||
catalogs are sorted by `id` before composition, so the input order does not
|
||||
affect the result. A new canonical name adds a recognition entry. A canonical
|
||||
name matching an existing canonical name augments that spell and keeps the
|
||||
established canonical display spelling. Repeated aliases for the same spell
|
||||
are idempotent.
|
||||
Repeated aliases for the same spell are accepted. A canonical-name, canonical-
|
||||
to-alias, or alias-to-alias collision between different spells is rejected,
|
||||
including a collision with the embedded catalog. Matching uses the catalog’s
|
||||
case, whitespace, and apostrophe normalization, so authors should avoid names
|
||||
or aliases that normalize to another spell.
|
||||
|
||||
Canonical-name display conflicts and canonical/alias or alias/alias collisions
|
||||
between different spells are errors, including collisions with the embedded
|
||||
catalog. Canonical names and aliases use the catalog's case, whitespace, and
|
||||
common-apostrophe normalization rules. The effective catalog returns canonical
|
||||
names in sorted order and produces a semantic SHA-256 digest that is stable
|
||||
under JSON formatting, object-key, catalog, spell, and alias reordering.
|
||||
The overlay is a recognition aid only. The durable spell-artifact schema and
|
||||
source-evidence rules are defined by the
|
||||
[D&D spell artifact contract](dnd-spell-artifacts.md).
|
||||
|
||||
@@ -1,201 +1,111 @@
|
||||
# JSON Output
|
||||
# Published JSON Output
|
||||
|
||||
This document is the durable JSON output file-format contract produced by the
|
||||
production JSON encoder and written by the CLI. Selectable output-encoder keys
|
||||
are cataloged in
|
||||
[Configuration](../config.md#implemented-production-modules).
|
||||
This document defines the logical JSON bundle emitted by the production JSON
|
||||
output encoder. The bundle’s physical destination, atomic publication, and
|
||||
retention are operational concerns; see [Operations](../operations.md#output-bundles).
|
||||
Output configuration, including chunk-map export, belongs in
|
||||
[Configuration](../config.md#module-bindings-and-validators).
|
||||
|
||||
The output module produces the logical bundle described here. The CLI's
|
||||
physical placement and lifecycle for that bundle are defined in
|
||||
[Operations](../operations.md#output-directory).
|
||||
## Bundle Layout
|
||||
|
||||
## Files
|
||||
All paths below are logical, relative, slash-separated bundle paths. The
|
||||
encoder always emits the first four JSON files below and adds lane or chunk-map
|
||||
files when their corresponding artifacts are available:
|
||||
|
||||
The encoder writes:
|
||||
| Path | Purpose |
|
||||
| --- | --- |
|
||||
| `index.json` | Entry point that names the other published files and lane payloads. |
|
||||
| `manifest.json` | Run provenance and result summaries. |
|
||||
| `rejected.json` | Rejected pipeline outputs. |
|
||||
| `warnings.json` | Accepted-output and run warnings. |
|
||||
| `lanes/<safe-lane-id>.json` | One normalized artifact payload for each lane. |
|
||||
| `chunk-map.json` | Optional accepted chunk map, when its export is enabled and available. |
|
||||
|
||||
- `index.json`
|
||||
- `manifest.json`
|
||||
- `lanes/<lane-id>.json`, one file per normalized serialized artifact
|
||||
- `rejected.json`
|
||||
- `warnings.json`
|
||||
- `chunk-map.json`, only when the JSON output binding enables
|
||||
`include_chunk_map` and the run has an accepted chunk map
|
||||
|
||||
Files are pretty-printed JSON with a trailing newline when the payload is JSON.
|
||||
Logical file paths are relative, slash-separated, and may not contain `..`.
|
||||
JSON files are pretty-printed with a trailing newline. Lane payloads are
|
||||
accepted only when their media type is `application/json`.
|
||||
|
||||
## `index.json`
|
||||
|
||||
Shape:
|
||||
`index.json` is the bundle’s discovery document. An approved run with no
|
||||
normalized lanes has this valid minimal index:
|
||||
|
||||
```json
|
||||
{
|
||||
"manifest_file": "manifest.json",
|
||||
"output_files": [
|
||||
{
|
||||
"lane_id": "spells",
|
||||
"media_type": "application/json",
|
||||
"file": "lanes/spells.json",
|
||||
"module_key": "noop",
|
||||
"schema_id": "notarius.dnd.spells",
|
||||
"schema_name": "notarius_dnd_spells_v1",
|
||||
"schema_version": "v1"
|
||||
}
|
||||
],
|
||||
"output_files": [],
|
||||
"rejected_file": "rejected.json",
|
||||
"warnings_file": "warnings.json"
|
||||
}
|
||||
```
|
||||
|
||||
`output_files` is sorted by lane ID. Output file names are produced by
|
||||
sanitizing the lane ID:
|
||||
| Field | Required | Meaning |
|
||||
| --- | --- | --- |
|
||||
| `manifest_file` | Yes | Always `manifest.json`. |
|
||||
| `output_files` | Yes | Lane descriptors sorted by `lane_id`. |
|
||||
| `rejected_file` | Yes | Always `rejected.json`. |
|
||||
| `warnings_file` | Yes | Always `warnings.json`. |
|
||||
| `chunk_map` | No | Descriptor for the pipeline-wide `chunk-map.json`; never a lane descriptor. |
|
||||
|
||||
- characters outside `A-Z`, `a-z`, `0-9`, `.`, `_`, and `-` become `_`;
|
||||
- repeated `..` sequences are replaced;
|
||||
- leading and trailing `.`, `_`, and `-` are trimmed;
|
||||
- empty sanitized names are rejected;
|
||||
- two lanes that sanitize to the same output file are rejected.
|
||||
|
||||
`manifest_file`, `rejected_file`, and `warnings_file` contain the fixed paths
|
||||
shown above. Each `output_files` entry requires `lane_id` and `file`. It also
|
||||
contains the normalized payload `media_type`, normalizer `module_key`, and
|
||||
response `schema_id`, `schema_name`, and `schema_version` when those values are
|
||||
available.
|
||||
|
||||
When present, the top-level optional `chunk_map` descriptor contains exactly
|
||||
Each lane descriptor has required `lane_id` and `file`. It may also include
|
||||
`media_type`, `module_key`, `schema_id`, `schema_name`, and `schema_version`
|
||||
when supplied by the normalized artifact. A `chunk_map` descriptor contains
|
||||
`artifact_kind`, `file`, `media_type`, `schema_id`, `schema_name`, and
|
||||
`schema_version`. It identifies the pipeline-wide `chunk-map.json`; it is not
|
||||
a lane output and never appears in `output_files`. The descriptor and file are
|
||||
both absent when export is disabled or no chunk plan was accepted. Its payload
|
||||
contract is defined by [Accepted Chunk Map](chunk-map.md).
|
||||
`schema_version`; its payload is defined by the
|
||||
[Accepted Chunk Map contract](chunk-map.md).
|
||||
|
||||
The lane path is derived from its lane ID. Characters outside letters, digits,
|
||||
periods, underscores, and hyphens become underscores; `..` sequences are
|
||||
neutralized; leading and trailing periods and underscores are removed. A lane
|
||||
that produces an empty name, or two lanes that produce the same path, makes
|
||||
output encoding fail.
|
||||
|
||||
## Lane Payloads
|
||||
|
||||
Each `lanes/<safe-lane-id>.json` file is the codec-owned normalized JSON for
|
||||
that lane. Consumers should use the index descriptor’s schema identity rather
|
||||
than infer a lane schema from its name. The current D&D payload contracts are
|
||||
[spells](dnd-spell-artifacts.md), [NPCs](dnd-npc-artifacts.md),
|
||||
[NPC interactions](dnd-npc-interaction-artifacts.md),
|
||||
[combat turns](dnd-combat-turn-artifacts.md),
|
||||
[item events](dnd-item-event-artifacts.md), and
|
||||
[scene descriptions](dnd-scene-description-artifacts.md).
|
||||
|
||||
## `manifest.json`
|
||||
|
||||
`manifest.json` contains a run manifest. This abridged example shows its core
|
||||
structure:
|
||||
`manifest.json` is published provenance, not a copy of lane payloads or a
|
||||
checkpoint store. Fields without a value may be omitted. Its top-level fields
|
||||
group into the following externally observable summaries:
|
||||
|
||||
```json
|
||||
{
|
||||
"run_id": "run-123",
|
||||
"pipeline_id": "dnd-session",
|
||||
"artifact_lanes": [
|
||||
{
|
||||
"id": "spells",
|
||||
"extractor": "dnd/spells",
|
||||
"merger": "appendorder",
|
||||
"normalizer": "noop"
|
||||
}
|
||||
],
|
||||
"validation_status": "approved",
|
||||
"started_at": "2026-01-01T00:00:00Z",
|
||||
"completed_at": "2026-01-01T00:00:01Z"
|
||||
}
|
||||
```
|
||||
| Group | Fields |
|
||||
| --- | --- |
|
||||
| Run identity and result | `run_id`, `pipeline_id`, `pipeline_digest`, `schema_version`, `validation_status`, `started_at`, `completed_at` |
|
||||
| Resolved components | `input_module`, `chunker`, `extractors`, `merger`, `normalizer`, `output_encoder`, `artifact_lanes`, `validator_chains`, `module_metadata` |
|
||||
| Source and references | `source_digests`, `references` |
|
||||
| Published result summaries | `normalized_outputs`, `rejected_outputs` |
|
||||
| Execution summaries | `chunk_plan`, `checkpoint_decisions`, `llm_profiles`, `metadata` |
|
||||
|
||||
Fields with empty values may be omitted by JSON encoding.
|
||||
`references` records provenance such as the target, slot, origin, digest,
|
||||
media type, size, and generated-artifact identity. It does not contain
|
||||
reference content. `normalized_outputs` and `rejected_outputs` likewise
|
||||
summarize results without embedding lane payload bytes. A chunk-plan summary is
|
||||
provenance for the plan used by this run; cache records, debug artifacts, and
|
||||
other operational state are not published as bundle files.
|
||||
|
||||
The manifest fields are:
|
||||
## Rejections And Warnings
|
||||
|
||||
- `run_id`, `pipeline_id`, and `pipeline_digest`: run and resolved-pipeline
|
||||
identity;
|
||||
- `input_module`, `chunker`, `extractors`, `merger`, `normalizer`, and
|
||||
`output_encoder`: resolved module keys;
|
||||
- `chunk_plan`: payload-free provenance for the effective chunk plan. `mode`
|
||||
is the effective cache mode; `action` is `reused`, `generated`,
|
||||
`refreshed`, or `bypassed` when a plan was materialized. `requested_module`
|
||||
is the current pipeline chunker, while `producer_input_module`,
|
||||
`producer_module`, `producer_llm_profile`, `producer_references`,
|
||||
`producer_metadata`, `source_digest`, `plan_digest`, `plan_schema_version`,
|
||||
and `created_at` describe the stored or generated producer when available.
|
||||
A cached plan can therefore identify a producer different from the requested
|
||||
module. This object never embeds ranges, units, annotations, prompts,
|
||||
responses, or reference content;
|
||||
- `module_metadata` and `artifact_lanes`: module and per-lane provenance,
|
||||
including prompt and response-schema provenance when provided;
|
||||
- `validator_chains`: resolved validation points and validators;
|
||||
- `source_digests` and `references`: source and reference provenance;
|
||||
- `normalized_outputs` and `rejected_outputs`: payload-free result summaries;
|
||||
- `llm_profiles`: selected profile IDs and provider or model names when
|
||||
available;
|
||||
- `metadata`: the effective prompt `session_id`;
|
||||
- `validation_status`: `approved` or `rejected`;
|
||||
- `started_at` and `completed_at`: UTC run timestamps.
|
||||
`rejected.json` is always an object with a `rejected` array. Each entry has
|
||||
required `stage` and `message`; `step_id`, `lane_id`, `module_key`, `chunk_id`,
|
||||
`chunk_index`, `validator_name`, `reason_code`, `attempt_count`, and
|
||||
`diagnostic_artifact_path` are present only when applicable.
|
||||
|
||||
`source_digests` contains source document digests only. Bound references are
|
||||
recorded separately under `references`, which contains provenance only: target
|
||||
stage, lane ID when present, slot name, origin type and URI, digest, media
|
||||
type, byte size, and binding source. Reference content is not written to
|
||||
durable output.
|
||||
`warnings.json` is always an object with a `warnings` array. Each warning has
|
||||
`reason_code` and `message`; `scope` is optional. Both arrays are empty when
|
||||
there is nothing to report.
|
||||
|
||||
Reference `stage` is `chunk`, `extract`, `merge`, or `normalize`. `lane_id` is
|
||||
omitted for chunk references and present for extract, merge, and normalize
|
||||
references.
|
||||
## Compatibility
|
||||
|
||||
`validation_status` is `approved` when no outputs were rejected and `rejected`
|
||||
when one or more outputs were rejected.
|
||||
|
||||
Producer warnings and the current run's chunk-validation warnings remain in
|
||||
`warnings.json`. The manifest records only provenance and decision summaries;
|
||||
empty producer-only values are omitted for compatibility with existing readers.
|
||||
|
||||
`validator_chains` records the resolved validator chain for each validation
|
||||
point. Entries include stage, lane ID when applicable, module key, and validators
|
||||
with key and execution class. Empty chains are recorded with an empty
|
||||
`validators` array, including chains resolved from explicit empty config
|
||||
overrides.
|
||||
|
||||
`normalized_outputs` summarizes each normalized lane output without embedding
|
||||
payload bytes. Entries include lane ID, normalizer module key, source ID, media
|
||||
type, and response schema provenance where available.
|
||||
|
||||
`rejected_outputs` summarizes rejected module outputs without embedding raw
|
||||
payload bytes. Entries include stage, lane, module, chunk, validator or reason,
|
||||
message, attempt count, and optional diagnostic artifact path.
|
||||
|
||||
## Output Payload Files
|
||||
|
||||
Each normalized serialized artifact is written to
|
||||
`lanes/<sanitized-lane-id>.json`. The JSON output encoder is domain-neutral and
|
||||
accepts only artifacts whose codec media type is `application/json`. The file
|
||||
contains the codec-owned JSON bytes pretty-printed.
|
||||
|
||||
The schema of each lane payload is owned by that artifact contract. For the
|
||||
current D&D lanes, see [D&D Spell Artifact](dnd-spell-artifacts.md),
|
||||
[D&D NPC Artifact](dnd-npc-artifacts.md), and
|
||||
[D&D Combat-Turn Artifact](dnd-combat-turn-artifacts.md), and
|
||||
[D&D Scene Description Artifact](dnd-scene-description-artifacts.md).
|
||||
|
||||
## `rejected.json`
|
||||
|
||||
Shape:
|
||||
|
||||
```json
|
||||
{
|
||||
"rejected": []
|
||||
}
|
||||
```
|
||||
|
||||
When output validation rejects an output, each entry contains `stage` and
|
||||
`message`. It includes `lane_id`, `module_key`, `chunk_id`, `chunk_index`,
|
||||
`validator_name`, `reason_code`, `attempt_count`, and
|
||||
`diagnostic_artifact_path` when applicable.
|
||||
|
||||
## `warnings.json`
|
||||
|
||||
Shape:
|
||||
|
||||
```json
|
||||
{
|
||||
"warnings": [
|
||||
{
|
||||
"scope": "extract",
|
||||
"reason_code": "example",
|
||||
"message": "human-readable warning"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
`warnings` is an empty array when no warnings are reported.
|
||||
Each warning requires `reason_code` and `message`; `scope` is omitted when it is
|
||||
empty.
|
||||
The index is the authoritative map from a logical lane to its published
|
||||
payload. Consumers must tolerate omitted optional manifest and descriptor
|
||||
fields, and should rely on the linked artifact contract for each lane’s JSON
|
||||
shape. This contract describes the published logical bundle only; it does not
|
||||
promise a filesystem layout or expose internal state formats.
|
||||
|
||||
@@ -1,69 +1,73 @@
|
||||
# Seriatim Transcript JSON
|
||||
# Seriatim Transcript Input
|
||||
|
||||
This document is the external input contract consumed by the production
|
||||
Seriatim input adapter. Selectable input-adapter keys are cataloged in
|
||||
[Configuration](../config.md#implemented-production-modules).
|
||||
This document defines the JSON transcript accepted by the production Seriatim
|
||||
input adapter. It is a source input, not a durable lane artifact. Configure the
|
||||
input adapter through [Configuration](../config.md#production-module-keys).
|
||||
|
||||
## Adapter
|
||||
## Contract Identity
|
||||
|
||||
- Source format: `application/vnd.seriatim+json`
|
||||
| Property | Value |
|
||||
| --- | --- |
|
||||
| Consumer | Seriatim input adapter |
|
||||
| Media type | `application/vnd.seriatim+json` |
|
||||
| Source document kind | `transcript` |
|
||||
| Source-unit kind | `transcript_segment` |
|
||||
|
||||
## Accepted Shape
|
||||
|
||||
The input must be one JSON object with top-level `metadata` and `segments`
|
||||
fields. This covers the maintained minimal fixture and Seriatim intermediate
|
||||
output that provides the same required segment fields.
|
||||
The input is one JSON object containing `metadata` and a non-empty `segments`
|
||||
array. This minimal document is valid:
|
||||
|
||||
The maintained example is
|
||||
[examples/seriatim-minimal-transcript.json](../../examples/seriatim-minimal-transcript.json).
|
||||
```json
|
||||
{
|
||||
"metadata": {"id": "session-alpha"},
|
||||
"segments": [
|
||||
{
|
||||
"id": 1,
|
||||
"start": 0,
|
||||
"end": 4,
|
||||
"speaker": "Aria",
|
||||
"text": "Aria casts Cure Wounds."
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Required top-level fields:
|
||||
The maintained two-segment input is
|
||||
[seriatim-minimal-transcript.json](../../examples/seriatim-minimal-transcript.json).
|
||||
|
||||
- `metadata`: an object. Its entries are accepted as source metadata.
|
||||
- `segments`: a non-empty array of segment objects.
|
||||
| Field | Required | Meaning and constraints |
|
||||
| --- | --- | --- |
|
||||
| `metadata` | Yes | JSON object. Its entries become source metadata; no particular metadata key is otherwise required. |
|
||||
| `segments` | Yes | Non-empty array of segment objects, kept in input order. |
|
||||
| `segments[].id` | Yes | Positive canonical decimal integer, supplied as a JSON number or string. IDs must be unique. |
|
||||
| `segments[].start` | Yes | Finite, non-negative numeric value, supplied as a JSON number or string. |
|
||||
| `segments[].end` | Yes | Finite, non-negative numeric value that is not earlier than `start`. |
|
||||
| `segments[].speaker` | Yes | String that is non-empty after trimming. |
|
||||
| `segments[].text` | Yes | String that is non-empty after trimming. Its original text is retained. |
|
||||
|
||||
Required segment fields:
|
||||
Additional top-level and segment fields are ignored. A missing required field,
|
||||
`null` in place of an object or array, malformed JSON, or more than one
|
||||
top-level JSON value is rejected.
|
||||
|
||||
- `id`: a positive integer JSON number or canonical decimal string without
|
||||
leading zeros or surrounding whitespace;
|
||||
- `start`: a finite, non-negative JSON number or numeric string;
|
||||
- `end`: a finite, non-negative JSON number or numeric string that is not less
|
||||
than `start`;
|
||||
- `speaker`: a non-empty string;
|
||||
- `text`: a non-empty string.
|
||||
## Source Identity And References
|
||||
|
||||
Other top-level and segment fields, such as `categories`, are ignored.
|
||||
The adapter chooses the source ID in this order:
|
||||
|
||||
Multiple top-level JSON values are rejected.
|
||||
1. a non-empty source ID supplied by the calling request;
|
||||
2. non-empty string `metadata.id`;
|
||||
3. non-empty string `metadata.source_id`;
|
||||
4. `seriatim:` followed by the first 16 hexadecimal characters of the raw
|
||||
input’s SHA-256 digest.
|
||||
|
||||
## Validation
|
||||
Each accepted segment becomes one source unit whose unit ID is `segments[].id`.
|
||||
Its self-reference uses the derived source ID and the same segment ID for both
|
||||
range endpoints. Artifact contracts use those segment IDs when they cite
|
||||
transcript evidence.
|
||||
|
||||
The adapter rejects empty input, malformed JSON, multiple top-level JSON values,
|
||||
non-object segment values, duplicate segment IDs, and any violation of the
|
||||
shape or field constraints above.
|
||||
## Compatibility
|
||||
|
||||
Segment text is preserved as provided, but it must not be empty after trimming.
|
||||
|
||||
## Derived Identity
|
||||
|
||||
Notarius identifies the parsed source in this order:
|
||||
|
||||
1. `metadata.id`, when it is a non-empty string after trimming;
|
||||
2. `metadata.source_id`, when it is a non-empty string after trimming;
|
||||
3. `seriatim:<first-16-hex-chars-of-raw-sha256>`.
|
||||
|
||||
The exact raw input SHA-256 remains the basis of the fallback source ID. The
|
||||
source digest recorded in output provenance is instead the SHA-256 of the
|
||||
canonical generic source document, excluding the digest field itself. It covers
|
||||
the derived source identity, document kind and format, ordered units and their
|
||||
self-references, and accepted metadata. Segment IDs become the unit IDs used by
|
||||
artifact source references; each produced unit carries a self-reference whose
|
||||
source ID is the derived document ID and whose start and end IDs both equal the
|
||||
segment ID.
|
||||
|
||||
## Compatibility Limit
|
||||
|
||||
This contract covers only Seriatim transcript JSON with the top-level
|
||||
`metadata` object and `segments` array described here. Broader Seriatim output
|
||||
schemas are compatible only when they provide these required fields with the
|
||||
accepted types.
|
||||
This adapter accepts only the shape described here. A broader Seriatim export
|
||||
is usable only when it supplies this object, metadata, and segment shape with
|
||||
the stated types and constraints. Unknown additional fields do not add
|
||||
Notarius behavior.
|
||||
|
||||
Reference in New Issue
Block a user