Compare commits

...

7 Commits

78 changed files with 3494 additions and 3254 deletions

View File

@@ -49,22 +49,22 @@ Flags:
use one Scriptorium profile ID. use one Scriptorium profile ID.
- `--session-id id`: pass a stable prompt session identifier through LLM-backed - `--session-id id`: pass a stable prompt session identifier through LLM-backed
module calls. module calls.
- `--reference selector=path`: bind a reference path to a chunk, extractor, or - `--reference selector=path`: bind a reference path to a chunk, extractor,
normalizer reference slot. Repeatable. merger, or normalizer reference slot. Repeatable.
- `--without-reference selector`: remove a configured optional reference binding. - `--without-reference selector`: remove a configured optional reference binding.
Repeatable. It accepts the same selector forms as `--reference`, without Repeatable. It accepts the same selector forms as `--reference`, without
`=path`. `=path`.
On success, the command prints the completed pipeline ID, approved and rejected On success, the command prints the completed pipeline ID, normalized output and
artifact counts, and the output directory. If the run completes with warnings, rejected output counts, and the output directory. If the run completes with warnings,
the warning count is printed to stderr. the warning count is printed to stderr.
Reference flags are resolved against selected chunk, extractor, and normalizer Reference flags are resolved against selected chunk, extractor, merger, and normalizer
targets before the run starts. Flat slot names are accepted only when exactly targets before the run starts. Flat slot names are accepted only when exactly
one selected target declares that slot. Bound reference files are read before one selected target declares that slot. Bound reference files are read before
pipeline work starts, validated as UTF-8 text, and recorded as provenance for pipeline work starts, validated as UTF-8 text, and recorded as provenance for
the target that declares the slot. Runtime reference content is passed to the the target that declares the slot. Runtime reference content is passed to the
chunker, extractor, or normalizer target that declares the slot. Notarius infers chunker, extractor, merger, or normalizer target that declares the slot. Notarius infers
reference media types from file extensions for provenance and for optional slot reference media types from file extensions for provenance and for optional slot
checks. Reference content is not written to diagnostics, logs, errors, or checks. Reference content is not written to diagnostics, logs, errors, or
manifests. manifests.
@@ -81,9 +81,11 @@ Reference binding precedence is:
- `slot=path`: valid when exactly one selected target declares `slot`; - `slot=path`: valid when exactly one selected target declares `slot`;
- `chunk.slot=path`: target the chunker; - `chunk.slot=path`: target the chunker;
- `lane.slot=path`: valid when exactly one selected extractor or normalizer in - `merge.slot=path`: valid when exactly one selected merger declares `slot`;
that lane declares `slot`; - `lane.slot=path`: valid when exactly one selected extractor, merger, or
normalizer in that lane declares `slot`;
- `lane.extract.slot=path`: target a lane extractor; - `lane.extract.slot=path`: target a lane extractor;
- `lane.merge.slot=path`: target a lane merger;
- `lane.normalize.slot=path`: target a lane normalizer. - `lane.normalize.slot=path`: target a lane normalizer.
Use `slot=path` when the selected targets declare the slot unambiguously: Use `slot=path` when the selected targets declare the slot unambiguously:
@@ -105,7 +107,7 @@ go run ./cmd/notarius run dnd-session \
--reference spells.extract.glossary=./campaign-glossary.txt --reference spells.extract.glossary=./campaign-glossary.txt
``` ```
The same grammar can target chunk and normalize slots when the configured The same grammar can target chunk, merge, and normalize slots when the configured
modules declare them: modules declare them:
```sh ```sh
@@ -113,6 +115,7 @@ go run ./cmd/notarius run dnd-session \
--config path/to/config.yml \ --config path/to/config.yml \
--input examples/seriatim-minimal-transcript.json \ --input examples/seriatim-minimal-transcript.json \
--reference chunk.scene_guide=./campaign-scenes.txt \ --reference chunk.scene_guide=./campaign-scenes.txt \
--reference spells.merge.merge_notes=./merge-notes.txt \
--reference spells.normalize.normalization_notes=./normalization-notes.txt --reference spells.normalize.normalization_notes=./normalization-notes.txt
``` ```

View File

@@ -141,13 +141,13 @@ against the production module catalog and fail fast for unknown or incompatible
module keys. module keys.
Reference bindings are validated against reference slots declared by eligible Reference bindings are validated against reference slots declared by eligible
chunk, extract, and normalize targets during pipeline resolution. Required slots chunk, extract, merge, and normalize targets during pipeline resolution. Required slots
must be bound after config defaults, target-local references, lane-level must be bound after config defaults, target-local references, lane-level
compatibility bindings, and run-time `--reference` or `--without-reference` compatibility bindings, and run-time `--reference` or `--without-reference`
overrides are applied. Config-relative paths are resolved relative to the config overrides are applied. Config-relative paths are resolved relative to the config
file; CLI reference paths are resolved relative to the current working file; CLI reference paths are resolved relative to the current working
directory. Materialized bound files must be UTF-8 text. Materialized reference directory. Materialized bound files must be UTF-8 text. Materialized reference
provenance is recorded for chunk, extractor, and normalizer targets, and runtime provenance is recorded for chunk, extractor, merger, and normalizer targets, and runtime
reference content is passed to the target that declares the slot. Reference reference content is passed to the target that declares the slot. Reference
media types are inferred from file extensions, recorded as canonical base media media types are inferred from file extensions, recorded as canonical base media
types, and checked only when a module declares `AcceptedMediaTypes`; unknown types, and checked only when a module declares `AcceptedMediaTypes`; unknown
@@ -156,8 +156,8 @@ written to diagnostics, logs, errors, or manifests.
Pipeline-level `references` are defaults. They are valid when at least one Pipeline-level `references` are defaults. They are valid when at least one
eligible target in the full configured pipeline declares the slot, including eligible target in the full configured pipeline declares the slot, including
chunk, extractor, and normalizer targets. During a run, they apply only to the chunk, extractor, merger, and normalizer targets. During a run, they apply only
selected targets that declare the slot: to the selected targets that declare the slot:
```yaml ```yaml
pipelines: pipelines:
@@ -192,15 +192,17 @@ pipelines:
party: ./campaign/session-party.txt party: ./campaign/session-party.txt
``` ```
`chunk.references` and `normalize.references` are accepted in object-form `chunk.references`, `merge.references`, and `normalize.references` are accepted
bindings. They override pipeline-level defaults for slots declared by the chunk in object-form bindings. They override pipeline-level defaults for slots
or normalizer module. Extractor-local references apply only to the extractor, declared by that target module. Extractor-local references apply only to the
and normalizer-local references apply only to the normalizer. extractor, merger-local references apply only to the merger, and
normalizer-local references apply only to the normalizer.
Target-local reference fields use the same map shape at: Target-local reference fields use the same map shape at:
- `pipelines.<id>.chunk.references` - `pipelines.<id>.chunk.references`
- `pipelines.<id>.artifacts.<lane>.extract.references` - `pipelines.<id>.artifacts.<lane>.extract.references`
- `pipelines.<id>.artifacts.<lane>.merge.references`
- `pipelines.<id>.artifacts.<lane>.normalize.references` - `pipelines.<id>.artifacts.<lane>.normalize.references`
Each binding is valid only when that target module declares the slot. Each binding is valid only when that target module declares the slot.
@@ -226,13 +228,17 @@ Binding fields:
- `module`: module key. - `module`: module key.
- `llm_profile`: optional Scriptorium profile ID. Empty or omitted lets the - `llm_profile`: optional Scriptorium profile ID. Empty or omitted lets the
Scriptorium prompt default select the profile. Scriptorium prompt default select the profile.
- `retries`: non-negative retry count for extra runtime attempts after the
first attempt. The runner applies retries to `chunk`, `extract`, `merge`, and
`normalize` bindings.
- `options`: optional module-specific settings. - `options`: optional module-specific settings.
- `references`: optional reference bindings. Supported only for `chunk`, - `references`: optional reference bindings. Supported only for `chunk`,
`extract`, and `normalize` bindings. `input`, `merge`, validator, and `extract`, `merge`, and `normalize` bindings. `input`, validator, and
`output` bindings reject this field during validation. `output` bindings reject this field during validation.
The `--llm-profile` run flag overrides every effective LLM-capable module The `--llm-profile` run flag overrides every effective LLM-capable module
binding to use one Scriptorium profile ID. binding to use one Scriptorium profile ID: chunk, every selected lane extract,
merge, and normalize binding.
## Implemented Production Modules ## Implemented Production Modules
@@ -241,10 +247,10 @@ binding to use one Scriptorium profile ID.
| input | `seriatim` | Reads Seriatim transcript JSON. | | input | `seriatim` | Reads Seriatim transcript JSON. |
| chunk | `generic` | Splits source units into ordered chunks. | | chunk | `generic` | Splits source units into ordered chunks. |
| chunk | `dnd/scenes` | Uses an LLM to split transcript source units into D&D scenes. | | chunk | `dnd/scenes` | Uses an LLM to split transcript source units into D&D scenes. |
| extract | `dnd/spells` | Extracts `dnd.spell_cast` artifacts. | | extract | `dnd/spells` | Extracts D&D spell raw outputs. |
| merge | `appendorder` | Keeps candidates in append order. | | merge | `appendorder` | Merges JSON raw extract outputs in chunk order. |
| normalize | `noop` | Passes merged artifacts through unchanged. | | normalize | `noop` | Passes merged raw outputs through unchanged. |
| output | `json` | Produces JSON output files. | | output | `json` | Produces JSON output files for normalized `application/json` lanes. |
The `generic` chunker accepts: The `generic` chunker accepts:
@@ -302,6 +308,6 @@ Pipeline resolution additionally checks:
- required module keys are present; - required module keys are present;
- module keys are registered for the expected slot; - module keys are registered for the expected slot;
- module capability requirements are satisfied; - module capability requirements are satisfied;
- bound reference slots are declared by selected chunk, extractor, or - bound reference slots are declared by selected chunk, extractor, merger, or
normalizer targets; normalizer targets;
- required reference slots are bound for selected targets. - required reference slots are bound for selected targets.

View File

@@ -1,91 +1,27 @@
# D&D Spell-Cast Artifacts # D&D Spell Raw Output
This document is the durable artifact contract for approved This document is the durable raw output contract for the implemented
`dnd.spell_cast` artifacts produced by the implemented `dnd/spells` extractor. `dnd/spells` extractor.
## Artifact Identity ## Identity
- Extractor key: `dnd/spells` - Extractor key: `dnd/spells`
- Artifact type: `dnd.spell_cast`
- Schema version: `v1`
- Prompt ID: `dnd.spells` - Prompt ID: `dnd.spells`
- Response schema key: `dnd_spells` - Response schema key: `dnd_spells`
- Response schema ID: `notarius.dnd.spells` - Response schema ID: `notarius.dnd.spells`
- Response schema name: `notarius_dnd_spells_v1` - Response schema name: `notarius_dnd_spells_v1`
- Response schema version: `v1`
- Media type: `application/json`
The extractor requires source chunks and transcript source capability. It The extractor requires source chunks and transcript source capability. It
returns generic artifact candidates that are serialized by the JSON output returns the structured LLM response as raw JSON. The default `appendorder`
module. merger passes a single chunk output through and concatenates multiple
`spell_casts` arrays in chunk order. The default `noop` normalizer passes the
merge output through unchanged.
The extractor accepts optional UTF-8 text references: ## Output Shape
- `roster`: campaign roster or player-character notes. For a single chunk, `lanes/spells.json` has this shape:
- `glossary`: campaign glossary or spell/name notes.
References are supporting disambiguation material only. They are not source
evidence and are not addressable through `source_refs`.
## Artifact Envelope
Approved artifacts use the generic artifact envelope documented in
[JSON Output](json-output.md#artifact-files):
```json
{
"extractor_key": "dnd/spells",
"artifact_type": "dnd.spell_cast",
"schema_version": "v1",
"payload": {
"caster": "Aria",
"spell": "Cure Wounds",
"effect": "heals an injured ally",
"narrative_description": "Aria raises her holy symbol and casts Cure Wounds."
},
"source_refs": [
{
"source_id": "session-alpha",
"start_unit_id": "seg-001",
"end_unit_id": "seg-001"
}
]
}
```
## Payload Fields
The `payload` object contains:
- `caster`: in-world character or creature casting the spell;
- `spell`: spell name;
- `effect`: concise spell effect in the scene;
- `narrative_description`: short description of the spell cast in context.
All payload fields are strings and must be non-empty after trimming.
`caster` is the in-world caster, not the transcript speaker.
## Source References
Source references live on the artifact envelope as `source_refs`; they are not
duplicated inside the `payload`.
Each source reference uses the generic source-reference shape:
- `source_id`
- `start_unit_id`
- `end_unit_id`
Validation requires:
- at least one source reference;
- non-empty source ID and unit IDs;
- source ID matching the source document ID;
- start and end unit IDs existing in the source document;
- start unit appearing before or at the same position as end unit.
## Structured LLM Response Shape
The extractor asks the LLM for this top-level response shape:
```json ```json
{ {
@@ -98,8 +34,8 @@ The extractor asks the LLM for this top-level response shape:
"source_refs": [ "source_refs": [
{ {
"source_id": "session-alpha", "source_id": "session-alpha",
"start_unit_id": "seg-001", "start_unit_id": 1,
"end_unit_id": "seg-001" "end_unit_id": 1
} }
] ]
} }
@@ -109,31 +45,63 @@ The extractor asks the LLM for this top-level response shape:
`spell_casts` must be present. It may be empty when no spell casts are found. `spell_casts` must be present. It may be empty when no spell casts are found.
The response schema asset is embedded at For multiple chunks with the default merger, the lane output keeps the same
`internal/modules/extract/dnd/spells/assets/schemas/dnd_spells.v1.json`. top-level shape and concatenates `spell_casts` in chunk order:
## Validators ```json
{
"spell_casts": [
{
"caster": "Aria",
"spell": "Cure Wounds",
"effect": "heals an injured ally",
"narrative_description": "Aria raises her holy symbol and casts Cure Wounds.",
"source_refs": [
{
"source_id": "session-alpha",
"start_unit_id": 1,
"end_unit_id": 1
}
]
}
]
}
```
The extractor supplies two deterministic validators by default: ## Spell-Cast Fields
- `dnd/spells/shape` Each spell cast contains:
- `dnd/spells/source_refs`
Rejection reason codes: - `caster`: in-world character or creature casting the spell;
- `spell`: spell name;
- `effect`: concise spell effect in the scene;
- `narrative_description`: short description of the spell cast in context;
- `source_refs`: transcript source references supplied by the model.
- `invalid_payload`: payload JSON cannot be decoded as a spell-cast payload. `caster` is the in-world caster, not the transcript speaker.
- `missing_required_field`: `caster`, `spell`, `effect`, or
`narrative_description` is blank.
- `missing_source_ref`: candidate has no source references.
- `invalid_source_ref`: at least one source reference fails generic source
reference validation.
Warning reason codes: ## Source References
- `spell_not_near_source`: the extracted spell name was not found in the cited Each source reference uses the generic source-reference shape:
source text.
Rejected candidates are written to `rejected.json` by the JSON output module. - `source_id`
- `start_unit_id`
- `end_unit_id`
The extractor prompt and schema use integer `start_unit_id` and `end_unit_id`
values matching source-unit IDs.
## References
The extractor accepts optional UTF-8 text references:
- `players`
- `party`
- `glossary`
- `roster`, a deprecated compatibility alias for `party`
References are supporting disambiguation material only. They are not source
evidence and are not addressable through `source_refs`.
## Manifest Metadata ## Manifest Metadata
@@ -158,8 +126,3 @@ manifest metadata:
``` ```
Raw prompt and schema content are not included in manifest metadata. Raw prompt and schema content are not included in manifest metadata.
## Compatibility Limit
This contract covers only `dnd.spell_cast` artifacts produced by the
implemented spell-cast extractor.

View File

@@ -20,11 +20,11 @@ The `json` output module writes:
- `index.json` - `index.json`
- `manifest.json` - `manifest.json`
- `artifacts/<artifact-type>.json`, one file per approved artifact type - `lanes/<lane-id>.json`, one file per normalized raw lane output
- `rejected.json` - `rejected.json`
- `warnings.json` - `warnings.json`
Files are pretty-printed JSON with a trailing newline. Files are pretty-printed JSON with a trailing newline when the payload is JSON.
## `index.json` ## `index.json`
@@ -33,10 +33,15 @@ Shape:
```json ```json
{ {
"manifest_file": "manifest.json", "manifest_file": "manifest.json",
"artifact_files": [ "output_files": [
{ {
"artifact_type": "dnd.spell_cast", "lane_id": "spells",
"file": "artifacts/dnd.spell_cast.json" "media_type": "application/json",
"file": "lanes/spells.json",
"module_key": "noop",
"schema_id": "notarius.dnd.spells",
"schema_name": "notarius_dnd_spells_v1",
"schema_version": "v1"
} }
], ],
"rejected_file": "rejected.json", "rejected_file": "rejected.json",
@@ -44,8 +49,14 @@ Shape:
} }
``` ```
`artifact_files` is sorted by artifact type. It is empty when no artifacts are `output_files` is sorted by lane ID. Output file names are produced by
approved. sanitizing the lane ID:
- characters outside `A-Z`, `a-z`, `0-9`, `.`, `_`, and `-` become `_`;
- repeated `..` sequences are replaced;
- leading and trailing `.`, `_`, and `-` are trimmed;
- empty sanitized names are rejected;
- two lanes that sanitize to the same output file are rejected.
## `manifest.json` ## `manifest.json`
@@ -58,18 +69,6 @@ approved.
"pipeline_digest": "sha256:...", "pipeline_digest": "sha256:...",
"input_module": "seriatim", "input_module": "seriatim",
"chunker": "dnd/scenes", "chunker": "dnd/scenes",
"module_metadata": {
"chunker": {
"prompt_id": "dnd.scenes",
"prompt_version": "v1",
"prompt_sha256": "sha256:...",
"response_schema_key": "dnd_scenes",
"response_schema_id": "notarius.dnd.scenes",
"response_schema_name": "notarius_dnd_scenes_v1",
"response_schema_version": "v1",
"response_schema_sha256": "sha256:..."
}
},
"source_digests": ["sha256:..."], "source_digests": ["sha256:..."],
"extractors": ["dnd/spells"], "extractors": ["dnd/spells"],
"merger": "appendorder", "merger": "appendorder",
@@ -83,13 +82,6 @@ approved.
"normalizer": "noop" "normalizer": "noop"
} }
], ],
"llm_profiles": [
{
"id": "mistral-small-3",
"provider": "scriptorium",
"model": "configured-model"
}
],
"validation_status": "approved", "validation_status": "approved",
"started_at": "2026-01-01T00:00:00Z", "started_at": "2026-01-01T00:00:00Z",
"completed_at": "2026-01-01T00:00:01Z" "completed_at": "2026-01-01T00:00:01Z"
@@ -99,65 +91,47 @@ approved.
Fields with empty values may be omitted by JSON encoding. Fields with empty values may be omitted by JSON encoding.
`source_digests` contains source document digests only. Bound references are `source_digests` contains source document digests only. Bound references are
recorded separately under `references`, which contains provenance only: recorded separately under `references`, which contains provenance only: target
target stage, lane ID when present, slot name, origin type and URI, digest, stage, lane ID when present, slot name, origin type and URI, digest, media
media type, byte size, and binding source. Reference content is not written to type, byte size, and binding source. Reference content is not written to
durable output. durable output.
Reference `stage` is `chunk`, `extract`, or `normalize`. `lane_id` is omitted Reference `stage` is `chunk`, `extract`, `merge`, or `normalize`. `lane_id` is
for chunk references and present for extract and normalize references. omitted for chunk references and present for extract, merge, and normalize
references.
When references are bound, the manifest section has this shape: `validation_status` is `approved` when no raw outputs were rejected and
`rejected` when one or more raw outputs were rejected.
`normalized_outputs` summarizes each normalized lane output without embedding
payload bytes. Entries include lane ID, normalizer module key, source ID, media
type, and response schema provenance where available.
`rejected_outputs` summarizes rejected module outputs without embedding raw
payload bytes. Entries include stage, lane, module, chunk, validator or reason,
message, attempt count, and optional diagnostic artifact path.
## Output Payload Files
Each normalized raw output is written to `lanes/<sanitized-lane-id>.json`.
The JSON output encoder accepts only `application/json` normalized outputs. The
file contains the raw JSON payload pretty-printed.
For the current D&D spell extractor, `lanes/spells.json` has this shape:
```json ```json
{ {
"references": [ "spell_casts": [
{ {
"stage": "extract", "caster": "Aria",
"lane_id": "spells", "spell": "Cure Wounds",
"slot_name": "roster", "effect": "heals an injured ally",
"origin_type": "file", "narrative_description": "Aria raises her holy symbol and casts Cure Wounds.",
"origin_uri": "file:///absolute/path/roster.txt",
"digest": "sha256:...",
"media_type": "text/plain",
"size_bytes": 123,
"binding_source": "config"
}
]
}
```
Reference media types are inferred from file extensions and recorded as
canonical base media types. Unknown extensions are recorded as
`application/octet-stream`.
`module_metadata` is omitted when no singleton module provides metadata.
`validation_status` is `approved` when no candidates were rejected and
`rejected` when one or more candidates were rejected.
Top-level `module_metadata` is reserved for singleton pipeline modules
(`input`, `chunker`, and `output`). Lane-owned module metadata remains under
`artifact_lanes[].metadata`.
## Artifact Files
Each artifact file has this shape:
```json
{
"artifact_type": "dnd.spell_cast",
"artifacts": [
{
"extractor_key": "dnd/spells",
"artifact_type": "dnd.spell_cast",
"schema_version": "v1",
"payload": {},
"source_refs": [ "source_refs": [
{ {
"source_id": "session-alpha", "source_id": "session-alpha",
"start_unit_id": "seg-001", "start_unit_id": 1,
"end_unit_id": "seg-001" "end_unit_id": 1
} }
] ]
} }
@@ -165,50 +139,20 @@ Each artifact file has this shape:
} }
``` ```
Artifact envelope fields:
- `extractor_key`: extractor module key.
- `artifact_type`: artifact type.
- `schema_version`: artifact schema version.
- `payload`: artifact-type-specific JSON payload.
- `source_refs`: optional generic source references.
- `metadata`: optional artifact metadata.
Artifact file names are produced by sanitizing the artifact type:
- characters outside `A-Z`, `a-z`, `0-9`, `.`, `_`, and `-` become `_`;
- repeated `..` sequences are replaced;
- leading and trailing `.`, `_`, and `-` are trimmed;
- empty sanitized names are rejected.
For current D&D spell-cast artifacts, the file is
`artifacts/dnd.spell_cast.json`.
## `rejected.json` ## `rejected.json`
Shape: Shape:
```json ```json
{ {
"rejected": [ "rejected": []
{
"candidate": {
"index": 0,
"extractor_key": "dnd/spells",
"artifact_type": "dnd.spell_cast",
"schema_version": "v1",
"payload": {},
"source_refs": []
},
"validator_name": "dnd/spells/source_refs",
"reason_code": "missing_source_ref",
"message": "spell cast candidate must include at least one source ref"
}
]
} }
``` ```
`rejected` is an empty array when no candidates are rejected. When raw output validation rejects an output, entries use the
`contracts.RejectedOutput` shape, including stage, lane ID, module key,
validator name, reason code, message, attempt count, and optional diagnostic
artifact path.
## `warnings.json` ## `warnings.json`
@@ -218,26 +162,12 @@ Shape:
{ {
"warnings": [ "warnings": [
{ {
"scope": "output", "scope": "extract",
"reason_code": "example_warning", "reason_code": "example",
"message": "warning message" "message": "human-readable warning"
} }
] ]
} }
``` ```
`warnings` is an empty array when no warnings are reported. `warnings` is an empty array when no warnings are reported.
## Path Safety
The output module returns slash-separated logical paths. The CLI also validates
logical output names before writing:
- names must be non-empty;
- names must be relative;
- names must be clean;
- names must use `/`, not `\`;
- names must not contain `..`;
- resolved paths must stay under the run output directory.
Durable writes are atomic per file.

View File

@@ -28,7 +28,7 @@ output that provides the same required segment fields.
}, },
"segments": [ "segments": [
{ {
"id": "seg-001", "id": 1,
"start": 0, "start": 0,
"end": 4, "end": 4,
"speaker": "Aria", "speaker": "Aria",
@@ -56,10 +56,9 @@ The adapter rejects:
- missing, null, or non-object `metadata`; - missing, null, or non-object `metadata`;
- missing, null, non-array, or empty `segments`; - missing, null, non-array, or empty `segments`;
- segment values that are not objects; - segment values that are not objects;
- segment `id` values that are neither strings nor numbers; - segment `id` values that are not positive integer JSON numbers or numeric
strings;
- non-string `speaker` or `text`; - non-string `speaker` or `text`;
- empty segment IDs;
- segment IDs with leading or trailing whitespace;
- duplicate segment IDs; - duplicate segment IDs;
- missing or empty `speaker`; - missing or empty `speaker`;
- missing, empty, invalid, non-finite, or negative `start`; - missing, empty, invalid, non-finite, or negative `start`;
@@ -87,8 +86,7 @@ The adapter maps input to `SourceDocument`:
Each segment becomes one `SourceUnit`: Each segment becomes one `SourceUnit`:
- `segment.id` becomes `SourceUnit.ID`; numeric IDs are converted to their JSON - `segment.id` becomes integer `SourceUnit.ID`;
number text, so `1` becomes `"1"`;
- `segment.text` becomes `SourceUnit.Text`; - `segment.text` becomes `SourceUnit.Text`;
- `SourceUnit.Kind` is `transcript_segment`; - `SourceUnit.Kind` is `transcript_segment`;
- `speaker`, `start`, and `end` are stored in source-unit metadata. - `speaker`, `start`, and `end` are stored in source-unit metadata.

View File

@@ -15,7 +15,9 @@ CompleteStructured(ctx, request, out) (response, error)
The request contains prompt ID/version, profile ID, session ID, prompt input The request contains prompt ID/version, profile ID, session ID, prompt input
materials, and variables. The caller supplies a pointer target for decoded materials, and variables. The caller supplies a pointer target for decoded
structured output. structured output. The response also carries the raw structured output bytes
returned by the runtime so modules can preserve raw payloads in pipeline stage
outputs.
Modules that call the LLM own their prompts, schemas, prompt IDs, validators, Modules that call the LLM own their prompts, schemas, prompt IDs, validators,
and domain-specific interpretation. Provider adapters should not contain and domain-specific interpretation. Provider adapters should not contain
@@ -41,6 +43,10 @@ The runtime records the actual selected Scriptorium profile, provider, and model
used during execution. Manifest population does not rely on a precomputed used during execution. Manifest population does not rely on a precomputed
profile ID before pipeline execution. profile ID before pipeline execution.
Explicit profile validation and `--llm-profile` overrides apply to LLM-capable
pipeline stages: chunk, extract, merge, and normalize. Input, output, and
validator bindings are not part of the current production LLM profile scope.
## Scriptorium Adapter ## Scriptorium Adapter
`ScriptoriumClient` implements `contracts.StructuredLLMClient` by converting `ScriptoriumClient` implements `contracts.StructuredLLMClient` by converting
@@ -55,6 +61,7 @@ Notarius prompt requests into Scriptorium `RunRequest` values. It:
- lets Scriptorium render prompts, call the configured provider, and validate - lets Scriptorium render prompts, call the configured provider, and validate
structured output; structured output;
- unmarshals successful JSON into the caller-provided target; - unmarshals successful JSON into the caller-provided target;
- returns the validated raw structured output bytes to the caller;
- maps token usage and selected profile/model metadata into the Notarius - maps token usage and selected profile/model metadata into the Notarius
response and manifest profile recorder. response and manifest profile recorder.

View File

@@ -20,7 +20,7 @@ A production module package should provide:
Module specs should describe capabilities accurately. Resolution uses specs to Module specs should describe capabilities accurately. Resolution uses specs to
reject incompatible pipelines before execution. reject incompatible pipelines before execution.
Chunk, extract, and normalize modules that accept auxiliary reference material Chunk, extract, merge, and normalize modules that accept auxiliary reference material
must declare slots through both `ReferenceSlots()` and must declare slots through both `ReferenceSlots()` and
`ModuleSpec().ReferenceSlots`. The runtime slot list and registry metadata `ModuleSpec().ReferenceSlots`. The runtime slot list and registry metadata
should match so config validation can inspect slots without constructing module should match so config validation can inspect slots without constructing module
@@ -31,10 +31,10 @@ must still be UTF-8 text. When a slot declares accepted media types, Notarius
compares the canonical base media type inferred from the file extension, compares the canonical base media type inferred from the file extension,
case-insensitively and without parameters. case-insensitively and without parameters.
The resolver materializes reference content for chunk, extractor, and The resolver materializes reference content for chunk, extractor, merger, and
normalizer targets. Runtime delivery uses `contracts.ChunkRequest.References`, normalizer targets. Runtime delivery uses `contracts.ChunkRequest.References`,
`contracts.ExtractionRequest.References`, and `contracts.ExtractionRequest.References`, `contracts.MergeRequest.References`,
`contracts.NormalizeRequest.References`. Reference material is not source and `contracts.NormalizeRequest.References`. Reference material is not source
evidence and must not be converted into `SourceRef` values. If a module prompt evidence and must not be converted into `SourceRef` values. If a module prompt
uses references, pass them as prompt input materials through the structured LLM uses references, pass them as prompt input materials through the structured LLM
request. Prompt metadata hashes remain based on prompt asset source, not request. Prompt metadata hashes remain based on prompt asset source, not
@@ -48,7 +48,7 @@ Common D&D prompt fragments, reference slot helpers, prompt input assembly, and
reference rendering live under `internal/modules/sharedassets/dnd`. Module reference rendering live under `internal/modules/sharedassets/dnd`. Module
contracts should expose prompt IDs, versions, input material names, and contracts should expose prompt IDs, versions, input material names, and
non-secret prompt/schema hashes through manifest metadata; they should not non-secret prompt/schema hashes through manifest metadata; they should not
expose Scriptorium public types through chunk, extract, or normalize contracts. expose Scriptorium public types through chunk, extract, merge, or normalize contracts.
Chunk modules receive the structured LLM client, configured Scriptorium profile Chunk modules receive the structured LLM client, configured Scriptorium profile
ID, prompt session ID, and raw source input material through ID, prompt session ID, and raw source input material through
@@ -60,6 +60,10 @@ Normalize modules receive the structured LLM client, configured Scriptorium
profile ID, prompt session ID, and reference material through profile ID, prompt session ID, and reference material through
`contracts.NormalizeRequest` when they need model-backed reconciliation. `contracts.NormalizeRequest` when they need model-backed reconciliation.
Merge modules receive the structured LLM client, configured Scriptorium profile
ID, prompt session ID, raw source input material, and reference material through
`contracts.MergeRequest` when they need model-backed merge behavior.
## `seriatim` Input ## `seriatim` Input
Package: `internal/modules/input/seriatim` Package: `internal/modules/input/seriatim`
@@ -84,9 +88,10 @@ The `generic` chunker splits source units into ordered chunks. It validates the
source document, clones source units, assigns chunk IDs such as `chunk-000001`, source document, clones source units, assigns chunk IDs such as `chunk-000001`,
and records chunk metadata for start unit, end unit, and unit count. and records chunk metadata for start unit, end unit, and unit count.
The pipeline runner canonicalizes chunk units from the source document by ID The pipeline runner canonicalizes chunk units from the source document by
before extractors and mergers run. Chunker-owned context should stay in integer ID before extractors and mergers run. Chunkers also populate chunk
`SourceChunk.Metadata`. start and end unit IDs, content bytes, and media type. Chunker-owned context
should stay in `SourceChunk.Metadata`.
Options: Options:
@@ -126,12 +131,11 @@ Options: none. Non-empty options are rejected.
The chunker enforces full source-unit coverage from the first source unit to the The chunker enforces full source-unit coverage from the first source unit to the
last, sequential contiguous scenes, and no overlap. Its LLM-facing schema uses last, sequential contiguous scenes, and no overlap. Its LLM-facing schema uses
integer `start_unit_id` and `end_unit_id` values as 1-based source-unit numbers; integer `start_unit_id` and `end_unit_id` values matching source-unit IDs. It
the module canonicalizes valid integer references to source-unit IDs before assigns chunk IDs such as `scene-000001`, emits JSON chunk content, and stores
producing chunks. It assigns chunk IDs such as `scene-000001` and stores scene scene metadata including title, primary mode, participants, summary, boundary
metadata including title, primary mode, participants, summary, boundary note, note, confidence, boundary unit IDs, and unit count. Boundary caveats become
confidence, boundary unit IDs, and unit count. Boundary caveats become warnings warnings with reason code
with reason code
`scene_boundary_caveat`. Whitespace-only caveats are treated as malformed `scene_boundary_caveat`. Whitespace-only caveats are treated as malformed
structured output rather than silently dropped. structured output rather than silently dropped.
@@ -144,14 +148,12 @@ text, or secrets.
Package: `internal/modules/extract/dnd/spells` Package: `internal/modules/extract/dnd/spells`
The `dnd/spells` extractor owns D&D spell-cast artifact semantics. It supplies The `dnd/spells` extractor owns D&D spell-cast extraction semantics. It
the embedded Scriptorium prompt ID, prompt version, transcript and reference supplies the embedded Scriptorium prompt ID, prompt version, transcript and
input materials, response schema, and session ID to the runtime; converts reference input materials, response schema, and session ID to the runtime; then
spell-cast responses into artifact candidates; and supplies deterministic returns the structured LLM `spell_casts` response as raw JSON.
validators.
Its LLM-facing source-reference schema uses integer `start_unit_id` and Its LLM-facing source-reference schema uses integer `start_unit_id` and
`end_unit_id` values as 1-based source-unit numbers; the module canonicalizes `end_unit_id` values matching source-unit IDs.
valid integer references to source-unit IDs before validation and output.
Its prompt definition lives under `assets/prompts` and its schema under Its prompt definition lives under `assets/prompts` and its schema under
`assets/schemas`. Shared reusable D&D prompt fragments are provided by `assets/schemas`. Shared reusable D&D prompt fragments are provided by
@@ -167,15 +169,16 @@ Provides:
- `dnd.spell_casts` - `dnd.spell_casts`
Artifact type and schema version: Response schema identity:
- artifact type: `dnd.spell_cast` - schema ID: `notarius.dnd.spells`
- schema name: `notarius_dnd_spells_v1`
- schema version: `v1` - schema version: `v1`
The extractor adds prompt and response-schema provenance to lane manifest The extractor adds prompt and response-schema provenance to lane manifest
metadata under `artifact_lanes[].metadata.extractor`. Durable artifact payload metadata under `artifact_lanes[].metadata.extractor`. Durable raw output
details belong in the details belong in the
[D&D spell artifact contract](../integrations/dnd-spell-artifacts.md). [D&D spell raw output contract](../integrations/dnd-spell-artifacts.md).
The `dnd/scenes` chunker and `dnd/spells` extractor declare optional `players`, The `dnd/scenes` chunker and `dnd/spells` extractor declare optional `players`,
`party`, and `glossary` reference slots accepting UTF-8 plain text, Markdown, `party`, and `glossary` reference slots accepting UTF-8 plain text, Markdown,
@@ -183,32 +186,16 @@ YAML, or JSON. They also accept `roster` as a deprecated compatibility alias for
`party`. Their prompts frame references as supporting disambiguation material `party`. Their prompts frame references as supporting disambiguation material
only; spell-cast artifacts must still be grounded in the source transcript. only; spell-cast artifacts must still be grounded in the source transcript.
## D&D Spell Validators
The spell extractor returns two built-in validators:
- `dnd/spells/shape`: rejects malformed payloads and missing required fields.
- `dnd/spells/source_refs`: rejects candidates without valid source references.
It also emits a warning when the extracted spell name is not found in the
cited source text.
Reason codes include:
- `invalid_payload`
- `missing_required_field`
- `missing_source_ref`
- `invalid_source_ref`
- `spell_not_near_source`
These validators are supplied by the extractor when no validators are configured
for the lane.
## `appendorder` Merger ## `appendorder` Merger
Package: `internal/modules/merge/appendorder` Package: `internal/modules/merge/appendorder`
The `appendorder` merger clones and appends candidates in chunk order. It does The `appendorder` merger preserves chunk order for raw extract outputs. A
not deduplicate or reconcile candidates. single JSON extract output is passed through as the merge output. Multiple JSON
object outputs with one common top-level array field are merged by concatenating
that array field in chunk order. Other valid JSON shapes are merged as a JSON
array of decoded values in chunk order. Non-JSON media types and invalid JSON
are rejected.
Provides: Provides:
@@ -218,7 +205,7 @@ Provides:
Package: `internal/modules/normalize/noop` Package: `internal/modules/normalize/noop`
The `noop` normalizer clones merged candidates and returns them unchanged. The `noop` normalizer clones the raw merge output and returns it unchanged.
Requires: Requires:
@@ -232,9 +219,10 @@ Provides:
Package: `internal/modules/output/json` Package: `internal/modules/output/json`
The `json` output encoder converts approved artifacts, rejected artifacts, The `json` output encoder converts normalized raw outputs, rejected raw outputs,
warnings, and the run manifest into logical JSON output files. It groups warnings, and the run manifest into logical JSON output files. It writes one
approved artifacts by artifact type and sanitizes artifact-type file names. payload file per lane under `lanes/` and sanitizes lane IDs for file names.
Normalized output payloads must be valid `application/json`.
Requires: Requires:
@@ -262,7 +250,7 @@ When adding a module, keep source-format and extraction-domain boundaries clear:
- input modules may know external source formats; - input modules may know external source formats;
- extract modules may know artifact semantics and prompt/schema assets; - extract modules may know artifact semantics and prompt/schema assets;
- merge and normalize modules own candidate combination and reconciliation; - merge and normalize modules own raw output combination and reconciliation;
- output modules own serialization, not diagnostics or CLI reporting. - output modules own serialization, not diagnostics or CLI reporting.
Update [Development](../policy/development.md), [Configuration](../config.md), Update [Development](../policy/development.md), [Configuration](../config.md),

View File

@@ -21,8 +21,8 @@ belongs in modules, not in command handlers.
## Core Packages ## Core Packages
- `internal/core/artifacts`: artifact candidates, approved artifacts, rejected - `internal/core/artifacts`: run manifests and legacy artifact serialization
artifacts, validation decisions, and run manifests. shapes retained while pipeline handoff contracts use raw outputs.
- `internal/core/config`: defaults, YAML config parsing, environment overrides, - `internal/core/config`: defaults, YAML config parsing, environment overrides,
validation, redaction, and resolved pipeline config. validation, redaction, and resolved pipeline config.
- `internal/core/diagnostics`: per-run diagnostics directory creation, - `internal/core/diagnostics`: per-run diagnostics directory creation,

View File

@@ -33,18 +33,19 @@ The CLI writes the resolved pipeline and digest to diagnostics.
Pipeline profiles and artifact lanes may include reference binding maps keyed by Pipeline profiles and artifact lanes may include reference binding maps keyed by
reference slot name. During resolution, pipeline-level bindings act as defaults reference slot name. During resolution, pipeline-level bindings act as defaults
for selected chunk, extractor, and normalizer targets that declare the slot; for selected chunk, extractor, merger, and normalizer targets that declare the
target-local bindings override or add bindings for that target. Runtime slot; target-local bindings override or add bindings for that target. Runtime
`--reference` requests override target config bindings, and runtime unbinds `--reference` requests override target config bindings, and runtime unbinds
remove optional target bindings. Flat runtime slot names are resolved only when remove optional target bindings. Flat runtime slot names are resolved only when
exactly one selected target declares the slot; otherwise the CLI requires a more exactly one selected target declares the slot; otherwise the CLI requires a more
specific selector such as `chunk.slot`, `lane.extract.slot`, or specific selector such as `chunk.slot`, `lane.extract.slot`,
`lane.normalize.slot`. Resolution validates bindings against the declaring `lane.merge.slot`, or `lane.normalize.slot`. Resolution validates bindings
target specs and stores the bindings in target-aware resolved reference holders. against the declaring target specs and stores the bindings in target-aware
It does not read reference files or include reference bytes in source digests. resolved reference holders. It does not read reference files or include
reference bytes in source digests.
During run preparation, resolved file references for chunk, extractor, and During run preparation, resolved file references for chunk, extractor, merger,
normalizer targets are materialized before any LLM-backed pipeline work. Config and normalizer targets are materialized before any LLM-backed pipeline work. Config
bindings resolve relative to the config file, and CLI bindings resolve relative bindings resolve relative to the config file, and CLI bindings resolve relative
to the current working directory. Materialization accepts UTF-8 text files, to the current working directory. Materialization accepts UTF-8 text files,
computes `sha256:` content digests, records file origins, infers canonical base computes `sha256:` content digests, records file origins, infers canonical base
@@ -55,8 +56,8 @@ empty bound files. Media-type acceptance is checked only when a slot declares
manifests. The CLI writes provenance-only resolved reference diagnostics, and manifests. The CLI writes provenance-only resolved reference diagnostics, and
the run manifest records target-stage reference provenance separately from the run manifest records target-stage reference provenance separately from
source digests. Runtime reference content is passed to the matching chunker, source digests. Runtime reference content is passed to the matching chunker,
extractor, or normalizer request. LLM-backed modules pass that material onward extractor, merger, or normalizer request. LLM-backed modules pass that material
as named Scriptorium prompt inputs. onward as named Scriptorium prompt inputs.
The CLI carries raw input bytes into `pipeline.RunInput`. Input adapters parse The CLI carries raw input bytes into `pipeline.RunInput`. Input adapters parse
those bytes into the source document, while LLM-backed modules that need the those bytes into the source document, while LLM-backed modules that need the
@@ -65,9 +66,9 @@ origin metadata. The raw input payload is not written to manifests or default
diagnostics. diagnostics.
The CLI also carries an optional run `session_id`. The runner makes it available The CLI also carries an optional run `session_id`. The runner makes it available
to chunk, extract, and normalize requests; LLM-backed modules forward it through to chunk, extract, merge, and normalize requests; LLM-backed modules forward it
their structured completion requests so Scriptorium can include it in prompt through their structured completion requests so Scriptorium can include it in
execution metadata. prompt execution metadata.
## Registries And Module Specs ## Registries And Module Specs
@@ -83,10 +84,9 @@ Every production module registers a `ModuleSpec` with:
- `Provides`: capabilities added after that module runs; - `Provides`: capabilities added after that module runs;
- `Requires`: capabilities that must already be available. - `Requires`: capabilities that must already be available.
Chunk, extract, and normalize specs may also declare reference slots. Slot Chunk, extract, merge, and normalize specs may also declare reference slots. Slot
declarations are available from registry metadata without constructing module declarations are available from registry metadata without constructing module
instances. Input, merge, validate, and output specs must not declare reference instances. Input, validate, and output specs must not declare reference slots.
slots.
Capability checks prevent incompatible pipeline composition before a run starts. Capability checks prevent incompatible pipeline composition before a run starts.
@@ -102,8 +102,8 @@ Capability checks prevent incompatible pipeline composition before a run starts.
`pipeline.RunOutput` carries: `pipeline.RunOutput` carries:
- run manifest; - run manifest;
- approved artifacts; - normalized raw outputs;
- rejected artifacts; - rejected raw outputs;
- warnings; - warnings;
- logical output files returned by the output encoder. - logical output files returned by the output encoder.
@@ -116,10 +116,13 @@ The runner:
1. validates run input and registries; 1. validates run input and registries;
2. builds the input adapter and parses the raw input into a source document; 2. builds the input adapter and parses the raw input into a source document;
3. validates the source document; 3. validates the source document;
4. builds the chunker and produces source chunks; 4. builds the chunker and produces source chunks, retrying when configured;
5. validates source chunks against framework invariants; 5. validates source chunks against framework invariants and any registered raw
chunk validators;
6. runs each selected artifact lane in sorted resolved order; 6. runs each selected artifact lane in sorted resolved order;
7. builds the output encoder and validates logical output file names. 7. builds the output encoder and validates logical output file names.
8. passes accepted normalized raw outputs, rejected output records, warnings,
and the manifest to the output encoder.
## Chunk Results ## Chunk Results
@@ -129,22 +132,29 @@ configured LLM profile, module options, and run metadata. Deterministic and
LLM-backed chunkers use the same contract; provider construction stays outside LLM-backed chunkers use the same contract; provider construction stays outside
chunk modules. chunk modules.
After `Chunk` returns, the runner appends chunker warnings before returning any When chunking succeeds, the runner validates generic chunk invariants before
chunker error. When chunking succeeds, the runner validates generic chunk running extractors:
invariants before running extractors:
- chunk IDs must be non-empty and unique in the chunk result; - chunk IDs must be non-empty and unique in the chunk result;
- each chunk `SourceID` must match the source document ID; - each chunk `SourceID` must match the source document ID;
- each chunk `Index` must match its zero-based returned order; - each chunk `Index` must match its zero-based returned order;
- each chunk start and end unit ID must exist in the source document, with the
start unit at or before the end unit;
- each chunk must include non-empty extraction content and media type;
- each chunk must contain at least one source unit; - each chunk must contain at least one source unit;
- a chunk must not repeat a source unit; - a chunk must not repeat a source unit;
- every chunk source unit must exist in the source document; - every chunk source unit must exist in the source document;
- source units inside each chunk must appear in source-document order. - source units inside each chunk must appear in source-document order.
After validation, the runner rebuilds each chunk from source-document units by After validation, the runner rebuilds each chunk from source-document units by
ID, preserving the chunk boundary order and cloning chunk metadata. Extractors integer ID, preserving chunk boundaries, content bytes, media type, and cloned
and downstream stages therefore see canonical source units, while chunk metadata. Extractors and downstream stages therefore see canonical source
`SourceChunk.Metadata` remains the supported place for chunker-owned context. units, while `SourceChunk.Metadata` remains the supported place for
chunker-owned context.
If chunk validation rejects a chunk after configured retries, the runner records
a rejected raw output and skips downstream lane execution. Framework-level
chunking or validation errors that remain after configured retries fail the run.
The framework does not require complete source-unit coverage and does not reject The framework does not require complete source-unit coverage and does not reject
overlap between different chunks. Stricter policies, such as full coverage or overlap between different chunks. Stricter policies, such as full coverage or
@@ -155,33 +165,37 @@ Within an artifact lane, the runner:
1. builds the extractor, merger, and normalizer; 1. builds the extractor, merger, and normalizer;
2. records module manifest metadata when modules provide it; 2. records module manifest metadata when modules provide it;
3. extracts candidates from each chunk; 3. extracts one raw `ExtractOutput` from each accepted chunk, retrying when
4. normalizes candidate envelope fields such as index, extractor key, artifact configured;
type, and schema version; 4. fills runner-owned provenance on each extract output, including lane ID,
5. merges candidates; extractor key, source ID, chunk ID, and chunk index;
6. normalizes merged candidates; 5. validates raw extract outputs and omits rejected outputs from merge input;
7. validates candidate envelope consistency; 6. merges ordered accepted extract outputs into one raw `MergeOutput`, retrying
8. runs validators; when configured;
9. converts approved candidates to artifacts. 7. validates raw merge output and skips normalization for rejected merge output;
8. normalizes the accepted merge output into one raw `NormalizeOutput`,
retrying when configured;
9. validates raw normalize output and appends accepted normalized raw output to
`RunOutput.NormalizeOutputs`.
## Validators ## Validators
If a lane declares validators in config, the runner builds those validators from The current runner handoff is raw-output based. Extractors, mergers, and
the validator registry. Otherwise it uses validators returned by the extractor. normalizers do not advertise validator chains through their module interfaces.
Runner-side raw validation chains receive the raw module output plus
stage, lane, module, source, and chunk provenance. Empty raw validation chains
approve output by default.
Each validator must return exactly one decision for each eligible candidate. The Validator rejection is a non-fatal run outcome: the rejected output is recorded
runner enforces decision cardinality with `internal/framework/validate`. in `RunOutput.Rejected` and does not pass to the next stage. Validator execution
Rejected candidates are removed before the next validator runs. Approved errors are framework-level errors and retry according to the relevant binding.
candidates continue through the chain.
The production CLI currently registers no standalone validator modules. The
current D&D spell extractor supplies deterministic shape and source-reference
validators.
## Warnings And Failures ## Warnings And Failures
Warnings from chunking, extraction, merging, normalization, validation, and Warnings from the successful chunking, extraction, merging, and normalization
output encoding are accumulated in `RunOutput.Warnings`. attempts whose outputs are used are accumulated in `RunOutput.Warnings`, along
with output encoder warnings. Warnings from discarded retry attempts are not
promoted to final warnings.
Errors wrap the operation and module key or lane context. If execution fails Errors wrap the operation and module key or lane context. If execution fails
after a manifest exists, the returned manifest is marked `failed` and receives a after a manifest exists, the returned manifest is marked `failed` and receives a
@@ -189,14 +203,19 @@ completion timestamp.
On successful execution, the manifest validation status is: On successful execution, the manifest validation status is:
- `approved` when no candidates were rejected; - `approved` when no raw outputs were rejected;
- `rejected` when at least one candidate was rejected. - `rejected` when at least one raw output was rejected.
## Manifest Population ## Manifest Population
The manifest records run ID, pipeline ID, pipeline digest, module keys, top-level The manifest records run ID, pipeline ID, pipeline digest, module keys, top-level
module metadata, artifact lanes, LLM profile metadata, source digest, module metadata, artifact lanes, LLM profile metadata, source digest,
reference provenance, validation status, and timing. reference provenance, normalized raw output summaries, rejected output
summaries, validation status, and timing. Raw output summaries include lane ID,
normalizer module key, media type, source ID, and response-schema provenance
when present. Rejected output summaries include stage, lane, module, chunk,
validator or reason, message, attempt count, and optional diagnostic artifact
path. The manifest does not include raw output payload bytes.
Singleton pipeline modules may add non-secret metadata by implementing Singleton pipeline modules may add non-secret metadata by implementing
`contracts.ManifestMetadataProvider`. The runner records that metadata under `contracts.ManifestMetadataProvider`. The runner records that metadata under
@@ -206,3 +225,12 @@ Lane-owned modules may add non-secret metadata through
`artifact_lanes[].metadata`. The runner records extractor, merger, and `artifact_lanes[].metadata`. The runner records extractor, merger, and
normalizer metadata there. The D&D spell extractor uses lane metadata for normalizer metadata there. The D&D spell extractor uses lane metadata for
prompt and response-schema provenance. prompt and response-schema provenance.
## JSON Output
The production JSON output encoder writes `manifest.json`, `index.json`,
`warnings.json`, `rejected.json`, and one pretty-printed JSON file per accepted
normalized lane output under `lanes/`. It accepts only normalized outputs with
valid `application/json` payloads. Unsupported media types, invalid JSON, unsafe
logical paths, and duplicate sanitized lane file names fail the run before
durable output files are written.

View File

@@ -16,8 +16,8 @@ go run ./cmd/notarius run dnd-session \
--diagnostics-dir /tmp/notarius --diagnostics-dir /tmp/notarius
``` ```
The command prints a success line with the pipeline ID, approved and rejected The command prints a success line with the pipeline ID, normalized output count,
artifact counts, and the output path. rejected output count, and the output path.
## Output Directory ## Output Directory
@@ -32,15 +32,14 @@ different root.
The `json` output module writes these files: The `json` output module writes these files:
- `index.json`: file index with paths to the manifest, artifact files, - `index.json`: file index with paths to the manifest, lane output files,
rejected artifacts, and warnings. rejected outputs, and warnings.
- `manifest.json`: run manifest with resolved pipeline provenance, top-level - `manifest.json`: run manifest with resolved pipeline provenance, top-level
module metadata, module keys, reference provenance, validation status, and module metadata, module keys, reference provenance, validation status, and
timing. timing.
- `artifacts/<artifact-type>.json`: approved artifacts grouped by artifact - `lanes/<lane-id>.json`: normalized raw JSON output payloads, one file per
type. For the current D&D spell extractor, this includes lane. For the current D&D spell extractor, this includes `lanes/spells.json`.
`artifacts/dnd.spell_cast.json` when spell-cast artifacts are approved. - `rejected.json`: rejected raw output records.
- `rejected.json`: rejected candidates and validator decisions.
- `warnings.json`: warnings reported by pipeline modules or the output encoder. - `warnings.json`: warnings reported by pipeline modules or the output encoder.
Output writes are atomic per file. Logical output file names must be clean, Output writes are atomic per file. Logical output file names must be clean,
@@ -108,7 +107,7 @@ retained for inspection and may include `run-manifest.json`, `warnings.json`,
A successful run with warnings exits with code `0`, prints a warning count to A successful run with warnings exits with code `0`, prints a warning count to
stderr, and writes warnings to durable output and diagnostics when retained. stderr, and writes warnings to durable output and diagnostics when retained.
The run manifest `validation_status` indicates whether final artifacts were The run manifest `validation_status` indicates whether raw outputs were
approved or rejected after validation. approved or rejected after validation.
Reference-related warnings include empty bound reference files and D&D spell Reference-related warnings include empty bound reference files and D&D spell
@@ -134,7 +133,9 @@ There is no command to resume a failed run. Re-run `notarius run` after fixing
the cause. the cause.
Provider retries and timeouts are handled by Scriptorium according to the Provider retries and timeouts are handled by Scriptorium according to the
selected execution profile. There is no separate CLI retry command. selected execution profile. Pipeline module retries are controlled by module
binding `retries` values in config for chunk, extract, merge, and normalize.
There is no separate CLI retry command.
Notarius writes local files only. Remote storage and archive management are not Notarius writes local files only. Remote storage and archive management are not
part of the implemented CLI. part of the implemented CLI.

View File

@@ -1,419 +1,18 @@
# Raw Pipeline Data Model Implementation Plan # Raw Pipeline Implementation
This plan implements the target state in The raw pipeline migration described here has been implemented.
[`pipeline.md`](pipeline.md). It is intentionally staged for multiple coding
passes. Do not skip stages: later work assumes the contracts and tests from
earlier stages are already in place.
The desired end state is a fixed-shape pipeline: Current behavior is documented in:
```text - [Pipeline Internals](../internal/pipeline.md)
input -> chunk -> extract -> merge -> normalize -> output - [Modules](../internal/modules.md)
``` - [LLM Runtime](../internal/llm.md)
- [Configuration](../config.md)
- [CLI Reference](../cli.md)
- [Operations](../operations.md)
- [Troubleshooting](../troubleshooting.md)
- [JSON Output](../integrations/json-output.md)
- [D&D Spell Raw Output](../integrations/dnd-spell-artifacts.md)
`chunk` produces typed chunk envelopes. `extract`, `merge`, and `normalize` Follow-up validator work remains tracked separately in
produce raw byte payload envelopes with media type and provenance. Rejected [Validation Roadmap](validation.md).
outputs do not pass to the next stage.
## Stage 1: Integer Source Units And Chunk Payloads
Update the core source and chunk contracts before changing downstream stages.
Implementation tasks:
- Change `internal/core/source.SourceUnit.ID` from `string` to `int`.
- Change `internal/core/source.SourceRef.StartUnitID` and `EndUnitID` from
`string` to `int`.
- Treat source-unit IDs as positive, stable, input-adapter-owned integers.
`0` and negative IDs should be invalid.
- Update source validation helpers, including unit lookup and source-ref
ordering checks, to use integer IDs.
- Update `contracts.SourceChunk` to include:
- `StartUnitID int`;
- `EndUnitID int`;
- `Content []byte`;
- `MediaType string`.
- Keep `SourceChunk.Units []source.SourceUnit` for framework scheduling,
diagnostics, and modules that need unit metadata.
- Update chunk canonicalization in `internal/framework/pipeline` to enforce
minimal scheduling invariants:
- at least one chunk;
- non-empty chunk ID;
- source ID matches the source document;
- indexes are deterministic and contiguous from zero;
- start/end unit IDs exist and are ordered;
- `Units` are canonical copies from the source document;
- content is non-empty;
- media type is non-empty.
- Update the Seriatim input adapter to parse segment IDs as integers.
Accept JSON numbers and numeric strings only when they are positive integers.
Reject non-numeric IDs such as `seg-001`.
- Update Seriatim examples, fixtures, tests, and integration docs to use
integer segment IDs.
- Update D&D source-reference helper code under
`internal/modules/sharedassets/dnd` so source refs remain integer-valued all
the way into `source.SourceRef`.
- Remove fallback behavior that converts integer LLM unit refs back into string
source-unit IDs.
- Update the generic chunker and D&D scenes chunker to populate
`StartUnitID`, `EndUnitID`, `Content`, and `MediaType`.
- For current transcript chunkers, use `application/json` content containing a
canonical JSON encoding of the chunk's source units. Keep the original
`Units` field populated as framework-owned typed metadata.
Tests to update or add:
- `go test ./internal/core/source`
- `go test ./internal/modules/input/seriatim`
- `go test ./internal/modules/chunk/generic`
- `go test ./internal/modules/chunk/dnd/scenes`
- `go test ./internal/framework/contracts`
- `go test ./internal/framework/pipeline`
Completion criteria:
- All source refs and source-unit IDs in core/framework types are integers.
- Chunkers return extraction-ready content bytes plus media type.
- Framework chunk validation still allows partial coverage and overlap unless a
validator rejects them.
## Stage 2: Raw Module Output Contracts
Replace artifact-candidate stage contracts with raw payload envelopes.
Implementation tasks:
- Add contract types in `internal/framework/contracts` for raw stage payloads.
The exact names may differ, but the contracts must preserve:
- raw bytes;
- media type;
- lane ID;
- stage module key;
- source ID;
- chunk ID and chunk index where applicable;
- response schema ID, name, and version when applicable;
- metadata;
- warnings.
- Recommended shape:
- `ExtractOutput`
- `MergeOutput`
- `NormalizeOutput`
- a small shared payload/provenance helper if it reduces duplication.
- Update `Extractor` so `Extract` returns one raw `ExtractOutput` for one input
chunk instead of `[]artifacts.ArtifactCandidate`.
- Remove `Extractor.ArtifactType`, `Extractor.SchemaVersion`, and
`Extractor.Validators` from the generic extractor contract. Schema and prompt
provenance should come from the returned output and manifest metadata, not
artifact-specific methods.
- Update `Merger` so `Merge` receives ordered accepted `[]ExtractOutput` and
returns one raw `MergeOutput`.
- Update `Normalizer` so `Normalize` receives one accepted `MergeOutput` and
returns one raw `NormalizeOutput`.
- Update `OutputRequest` so output encoders receive the validated normalized
lane outputs rather than approved `Artifact` values.
- Add a raw rejected-output record type for manifests/output files. It should
identify stage, lane, module, chunk when relevant, validator or error reason,
message, attempt count, and diagnostic artifact path if present. It must not
include large raw content by default.
- Leave old artifact/candidate types in `internal/core/artifacts` only if
temporary compatibility code still needs them during the migration. They
should not remain in the final stage contracts.
Tests to update or add:
- Contract composition tests proving fake chunk, extract, merge, normalize, and
output modules compose using raw envelopes.
- Defensive-copy tests for raw content and metadata maps.
- Registry integration tests proving the new interfaces build and run.
Completion criteria:
- Framework contracts no longer require extractors, mergers, normalizers, or
output encoders to use `ArtifactCandidate` or `Artifact`.
- Stage outputs carry raw bytes and media type.
## Stage 3: Config, References, Profiles, And Retries
Make `merge` a first-class LLM/profile/reference stage and add retry policy.
Implementation tasks:
- Update `pipeline.referenceSlotStage` so `merge` modules may declare reference
slots.
- Add `MergeReferences` to `ResolvedArtifactLane`.
- Resolve and materialize merge references using the same declared-slot model
used by `chunk`, `extract`, and `normalize`.
- Update reference provenance so merge-stage references appear in manifests and
resolved reference diagnostics.
- Update config parsing, config validation, redaction, cloning, and examples so
`artifacts.<lane>.merge.references` is valid.
- Update CLI `--reference` selector parsing and ambiguity checks to support:
- `merge.<slot>=<path>` where unambiguous;
- `<lane>.merge.<slot>=<path>`;
- lane-qualified shorthand only when it resolves unambiguously.
- Update `--without-reference` behavior to support merge-stage slots.
- Update `ModuleStage` LLM-capable helpers so `chunk`, `extract`, `merge`, and
`normalize` are all considered LLM-capable.
- Update `--llm-profile` override handling so it applies to `chunk`, every lane
`extract`, every lane `merge`, and every lane `normalize`.
- Update explicit Scriptorium profile validation so it inspects only those four
LLM-capable stages.
- Add `retries` to `pipeline.ModuleBinding` as a non-negative integer count of
extra attempts after the first attempt. Default is `0`.
- Apply `retries` only to `chunk`, `extract`, `merge`, and `normalize` runtime
execution. Keep input and output retry behavior out of scope.
- Validate `retries >= 0` in config validation.
- Ensure redacted config preserves retry counts.
Tests to update or add:
- Config accepts merge references and rejects unknown/undeclared merge slots.
- CLI reference selectors work for merge and remain strict for ambiguous slots.
- `--llm-profile` overrides merge bindings as well as chunk/extract/normalize.
- Explicit profile validation includes merge and ignores non-LLM stages.
- Negative retries are rejected.
Completion criteria:
- Merge has the same runtime plumbing class as chunk/extract/normalize.
- Retry policy is representable in pipeline config and resolved bindings.
## Stage 4: Runner Orchestration And Retry Semantics
Rewrite pipeline execution around raw envelopes.
Implementation tasks:
- Introduce a small retry helper in `internal/framework/pipeline` that reruns
the same module with the same input when:
- the module returns a framework-level error;
- the module returns output that is rejected by that stage's validator chain.
- Preserve context cancellation: never retry after `ctx.Err()` is non-nil.
- Record attempt count and compact attempt diagnostics for manifests/output.
- Do not log or manifest raw payload bytes by default.
- Run chunk once per source input, with retries from the chunk binding.
- After successful chunk execution, run chunk validation when validator mapping
support exists. A rejected chunk result after final retry should stop
downstream execution and produce a rejected run record.
- Run extract once per accepted chunk, with per-chunk retries from the extract
binding.
- Omit rejected extract outputs from merge input.
- If a lane has no accepted extract outputs after retries, skip merge and
normalize for that lane, record rejected output details, and continue to the
next lane.
- Run merge once per lane with accepted extract outputs, with retries from the
merge binding.
- If merge output is rejected after final retry, omit that lane from normalize
and output.
- Run normalize once per accepted merge output, with retries from the normalize
binding.
- If normalize output is rejected after final retry, omit that lane from output.
- Treat validator rejection as non-fatal run outcome. Treat unrecovered
framework-level execution errors as run failures.
- Preserve deterministic ordering:
- chunks by chunk index;
- extract outputs by chunk index;
- lane outputs by resolved lane order.
- Continue to collect warnings from every successful module attempt whose output
is used. For failed attempts, record compact attempt diagnostics rather than
promoting warnings as final-stage warnings unless the implementation already
has a clear warning policy.
- Set manifest validation status to:
- `approved` when all produced normalized lane outputs pass;
- `rejected` when one or more module outputs are rejected;
- failed-run status through existing failure handling for unrecovered runtime
errors.
Validation seam scope:
- Add the runner-side raw validation seam needed by this data-model refactor.
- Represent the validation boundary as raw module output plus stage provenance,
consistent with [`validation.md`](validation.md).
- Support empty validator chains as approval.
- Add fake validator/test-only coverage for approval, rejection, and
retry-on-rejection behavior.
- Do not implement the full production validator package migration, default
validator mappings, or concrete generic validators in this plan. Those remain
owned by the validation roadmap.
Tests to update or add:
- Runner passes chunk content and media type to extractors.
- Runner passes ordered accepted extract outputs to merge.
- Runner passes accepted merge output to normalize.
- Rejected extract output is omitted from merge input.
- A lane with no accepted extracts is omitted and recorded.
- Rejected merge output prevents normalize for that lane.
- Rejected normalize output prevents output for that lane.
- Retry reruns the same module input after framework error.
- Retry reruns the same module input after validator rejection.
- Retry stops after configured attempts and records attempt count.
- Context cancellation stops retries.
Completion criteria:
- The runner no longer depends on candidate materialization between extract,
merge, normalize, and output.
- Rejected outputs never pass to the next stage.
## Stage 5: Production Module Migration
Update concrete modules to the new contracts.
Implementation tasks:
- Update `internal/modules/extract/dnd/spells`:
- call Scriptorium with the existing prompt/schema;
- capture the returned raw JSON response bytes;
- return one `ExtractOutput` with `application/json` media type and response
schema provenance;
- do not convert spell casts into `ArtifactCandidate`;
- do not run shape/source-ref/source-relatedness validation inside the module.
- Keep D&D spell prompt and schema asset metadata in manifest metadata.
- Update `internal/modules/merge/appendorder` to merge raw JSON extract outputs
deterministically:
- preserve input order by chunk index;
- if all extract outputs are JSON objects with one common top-level array
field, concatenate those arrays into one JSON object under that field;
- otherwise produce a JSON array containing each extract output's decoded JSON
value in order;
- reject non-JSON media types with a clear error unless a future option
deliberately supports them.
- Update `internal/modules/normalize/noop` to pass accepted merge output through
unchanged as normalized output.
- Update chunk modules from Stage 1 as needed after interface changes.
- Remove or quarantine old candidate validators from D&D module packages if
they no longer compile. Their replacement belongs under the validation
roadmap in `internal/validators`.
- Update fake/test modules throughout the repository to the raw contracts.
Tests to update or add:
- D&D spells extractor returns raw JSON with schema provenance and no candidate
materialization.
- Append-order merger concatenates common top-level JSON arrays.
- Append-order merger falls back to ordered JSON value arrays when shapes differ.
- Append-order merger rejects invalid JSON and non-JSON media types clearly.
- No-op normalizer returns defensive copies of merge output bytes and metadata.
- Production catalog still registers all modules successfully.
Completion criteria:
- Production modules compile against raw contracts.
- D&D spell extraction no longer rejects spell candidates inside the extractor.
## Stage 6: Output, Manifest, Diagnostics, And Files
Update durable output around normalized lane outputs.
Implementation tasks:
- Update `RunOutput` to carry normalized lane outputs and rejected module-output
records instead of approved/rejected artifacts.
- Update `artifacts.RunManifest` or introduce a more accurately named manifest
package if the old artifact naming becomes misleading. Prefer the smallest
rename that keeps manifest output clear and avoids a broad unrelated refactor.
- Preserve existing manifest fields that remain meaningful:
- pipeline ID and digest;
- module keys;
- lane IDs;
- source digest;
- LLM profile provenance;
- reference provenance;
- module metadata;
- validation status.
- Add manifest/reporting fields needed for raw outputs:
- normalized lane output media type;
- output schema provenance where present;
- rejected stage/lane/chunk/module information;
- retry attempt counts.
- Update `internal/modules/output/json`:
- write `manifest.json`;
- write `index.json`;
- write `warnings.json`;
- write `rejected.json`;
- write one file per normalized lane output.
- For the JSON output encoder, accept `application/json` normalized outputs and
write them as raw JSON files. Recommended file path:
`lanes/<safe-lane-id>.json`.
- Reject non-JSON normalized output media types in the JSON encoder with a clear
error. Future text/Markdown/binary encoders can support other media types.
- Keep output file name validation strict.
- Ensure diagnostics redaction still prevents raw source, prompt, reference,
schema, and payload bytes from appearing in ordinary errors or manifests.
Tests to update or add:
- JSON output writes lane output files and index entries deterministically.
- JSON output rejects invalid JSON bytes and unsupported media types.
- Rejected output records are written without raw payload bytes.
- Manifest includes raw-output provenance and retry attempt counts.
- Diagnostics tests still prove sensitive/large content is not emitted.
Completion criteria:
- Durable output no longer assumes artifact candidates.
- JSON output remains deterministic and safe.
## Stage 7: Documentation, Examples, And Full Validation
After behavior is implemented, update canonical current-behavior docs.
Implementation tasks:
- Update `docs/internal/pipeline.md` for the raw data model.
- Update `docs/internal/modules.md` for new module contracts.
- Update `docs/internal/llm.md` for merge-stage LLM/profile plumbing.
- Update `docs/config.md` for:
- integer source-unit expectations where config examples include source refs;
- merge references;
- module `retries`;
- LLM profile override scope.
- Update `docs/cli.md` for merge reference selectors and profile behavior.
- Update `docs/integrations/json-output.md` for lane output files.
- Update `docs/integrations/dnd-spell-artifacts.md`; if the artifact-specific
contract is no longer durable, rename or replace it with a D&D spell raw-output
integration doc.
- Update `docs/troubleshooting.md` for common raw-output, media-type, retry, and
validation failures.
- Update examples so Seriatim segment IDs are positive integers and expected
output shape matches lane-normalized output.
- Replace `docs/roadmap/implementation.md` with a completed-note document only
after this implementation is finished and reviewed.
- Leave `docs/roadmap/pipeline.md` as target-state roadmap context until the
feature is implemented, then move implemented behavior into canonical docs and
remove stale roadmap language.
Validation commands:
```sh
go test ./internal/core/source
go test ./internal/framework/contracts
go test ./internal/framework/pipeline
go test ./internal/core/config
go test ./internal/cli
go test ./internal/modules/input/seriatim
go test ./internal/modules/chunk/generic
go test ./internal/modules/chunk/dnd/scenes
go test ./internal/modules/extract/dnd/spells
go test ./internal/modules/merge/appendorder
go test ./internal/modules/normalize/noop
go test ./internal/modules/output/json
go test ./...
go vet ./...
go build ./cmd/notarius
```
Documentation inspection:
```sh
rg -n "ArtifactCandidate|approved artifacts|spell artifact|start_unit_id\": \"|end_unit_id\": \"" docs examples internal
rg -n "chunk`, `extract`, and `normalize|deterministic-only stage|json.RawMessage" docs/roadmap docs/internal docs/config.md docs/cli.md
```
Expected result:
- Remaining `ArtifactCandidate` mentions are only in deliberately retained
compatibility code or historical roadmap context.
- Current-behavior docs describe raw lane outputs, integer source-unit IDs,
merge LLM/reference support, and retry behavior.

View File

@@ -1,403 +1,12 @@
# Raw Pipeline Data Model # Raw Pipeline Data Model
This roadmap defines the target data model for the pipeline stages after input. The raw pipeline data model has been implemented.
It is paired with the validation system roadmap in
[`validation.md`](validation.md): validators should evaluate module outputs, but
the pipeline first needs a raw-output handoff model that does not require every
module response shape to be represented by bespoke Go structs.
## Goals Current behavior is documented in:
- Make typed chunk envelopes and raw module-output envelopes first-class - [Pipeline Internals](../internal/pipeline.md)
pipeline handoffs. - [Modules](../internal/modules.md)
- Keep framework-owned provenance around raw payloads so runs remain - [JSON Output](../integrations/json-output.md)
deterministic, ordered, and auditable.
- Avoid requiring each extractor, merger, or normalizer response schema to have
matching Go structs.
- Let extract modules produce one raw output per input chunk.
- Let merge modules receive accepted extract outputs and merge them into one raw
payload.
- Let normalize modules optionally post-process accepted merge output.
- Let output modules write normalized output bytes directly, along with
manifests, warnings, rejected outputs, and diagnostics as appropriate.
- Preserve the fixed workflow shape:
```text Validator mapping and concrete validator behavior remain tracked separately in
input -> chunk -> extract -> merge -> normalize -> output [Validation Roadmap](validation.md).
```
## Non-Goals
- Do not turn the pipeline into an arbitrary DAG or general workflow language.
- Do not make validators mutate, materialize, or rewrite module outputs.
- Do not require a generic source-reference convention for every raw payload in
this pass.
- Do not eliminate typed framework data where the framework genuinely needs it,
such as input source documents and chunk boundaries.
## Cross-Stage Runtime Plumbing
LLM/profile/reference plumbing should be available to every stage that may need
LLM-backed or reference-aware module behavior: `chunk`, `extract`, `merge`, and
`normalize`.
For all four stages, the framework should provide the same categories of runtime
support where the concrete module contract needs them:
- configured LLM profile and profile override handling;
- Scriptorium client access through framework-owned LLM contracts;
- session ID propagation;
- declared reference slots and resolved reference bindings;
- module options and metadata;
- prompt, schema, profile, and reference provenance for manifests and
diagnostics.
`merge` must not be treated as a deterministic-only stage. A merge module may be
simple and deterministic, but it may also be LLM-backed, reference-aware, and
validator-gated in the same way as `chunk`, `extract`, and `normalize`.
## Chunk Stage Target Model
The chunk stage receives the canonical `SourceDocument` from the input stage,
plus any original source input material needed for prompt construction. The
`SourceDocument` remains the source of truth for source identity, source-unit
ordering, source-unit IDs, and source provenance. Original raw input material
may be supplied to LLM-backed chunkers, but it should not replace the
`SourceDocument` as the framework handoff.
Input adapters are responsible for assigning stable integer source-unit IDs. How
those IDs are assigned depends on the input format. A numbered JSON transcript
may map source units directly to transcript segment numbers; a PDF input adapter
may assign page or extracted-text unit numbers; an adapter for unordered source
material may assign deterministic IDs as part of input parsing. Downstream
framework code should treat those IDs as opaque integers owned by the input
adapter.
The chunk module returns one or more ordered `SourceChunk` envelopes. A single
chunk representing the whole source is valid and should be supported.
Each chunk should include framework-readable provenance and ordering metadata,
plus content suitable for extraction. The content may be JSON, plain text,
Markdown, PDF page text, or another module/input-specific representation, as
long as the framework can still associate the chunk with the source and preserve
deterministic order.
Conceptually:
```go
type SourceChunk struct {
ID string
SourceID string
Index int
StartUnitID int
EndUnitID int
Content []byte
MediaType string
Units []source.SourceUnit
Metadata map[string]any
}
```
The exact implementation shape can differ, but it should preserve:
- chunk identity;
- source identity;
- deterministic chunk order;
- source locator or range, normally integer start/end unit IDs;
- chunk content and media type;
- optional source-unit projection when useful;
- metadata needed by extractors, validators, manifests, diagnostics, and output
encoders.
The framework owns the minimal invariants required to schedule extract work:
- the chunker returns at least one chunk;
- chunk IDs and indexes are stable and non-empty;
- chunk source IDs match the source document;
- chunk ordering is deterministic;
- chunk provenance is sufficient to trace the chunk back to the input source.
Domain-specific chunk acceptability belongs in validators. For example, full
coverage, no gaps, no overlap, scene metadata quality, expected media type, and
D&D scene-boundary policy should be explicit validator concerns rather than
hidden framework rules, except where a minimal invariant is required for
extraction to run safely.
## Extract Stage Target Model
The extract stage receives one input chunk from the chunk stage and produces one
raw extract output for that chunk.
The extractor owns:
- prompt selection;
- input material assembly;
- Scriptorium request construction;
- response-schema selection;
- LLM profile and session usage;
- returned raw payload metadata.
The framework owns:
- chunk iteration;
- deterministic ordering;
- association between each extract output and its input chunk;
- execution errors when an extractor does not return output;
- handoff of returned output to validation and later stages.
An extract output should be an envelope, not just raw bytes. Conceptually:
```go
type ExtractOutput struct {
LaneID string
ExtractorKey string
ChunkID string
ChunkIndex int
SourceID string
RawContent []byte
MediaType string
SchemaID string
SchemaName string
SchemaVersion string
Metadata map[string]any
Warnings []contracts.Warning
}
```
The exact implementation shape can differ, but it should preserve:
- raw returned content;
- chunk provenance;
- chunk order;
- source identity;
- module identity;
- response schema provenance;
- metadata needed by validators, mergers, manifests, diagnostics, and output
encoders.
Extractor modules should not be required to convert raw LLM output into
`ArtifactCandidate` values. Domain-specific Go projection may still exist for
specific modules when it is useful, but it should not be the generic pipeline
contract.
Rejected extract outputs are not passed to merge. This keeps downstream
contracts simple and makes rejection behavior explicit. Support for passing
rejected outputs forward as marked data is deferred future work.
## Merge Stage Target Model
The merge stage receives the accepted extract outputs for a lane. Each extract
output corresponds to one input chunk and carries enough metadata to recover the
original chunk order.
The merger owns:
- merge/reconciliation strategy;
- deterministic or LLM-backed merge logic;
- prompt and schema usage when LLM-backed;
- the shape of merged raw output.
The framework owns:
- passing the ordered accepted extract-output set to the merger;
- lane identity;
- source context;
- references;
- runtime LLM plumbing;
- validation handoff for merged output;
- manifest provenance.
Simple merge modules may concatenate raw extract outputs in chunk order. Other
merge modules may deterministically merge JSON documents, reconcile duplicates,
or use an LLM to produce a more coherent merged result.
Conceptually:
```go
type MergeRequest struct {
LaneID string
Source *source.SourceDocument
ExtractOutputs []ExtractOutput
SourceInput contracts.LLMInputMaterial
SessionID string
References contracts.ReferenceSet
LLMClient contracts.StructuredLLMClient
LLMProfile string
Options map[string]any
Metadata map[string]any
}
type MergeOutput struct {
LaneID string
MergerKey string
RawContent []byte
MediaType string
SchemaID string
SchemaName string
SchemaVersion string
Metadata map[string]any
Warnings []contracts.Warning
}
```
The exact implementation shape can differ, but the key contract is that
merge consumes ordered accepted extract outputs and produces one merged raw
output for the lane.
If no accepted extract outputs remain for a lane, the framework should not pass
rejected outputs to the merger. The lane should produce no merge output and
should be reported as rejected or omitted according to run reporting policy.
## Normalize Stage Target Model
The normalize stage receives accepted merge output for a lane and optionally
post-processes it into the final normalized raw payload for that lane.
Normalization may be a no-op, deterministic cleanup, schema conversion, or an
LLM-backed post-processing pass.
The normalizer owns:
- post-merge processing strategy;
- deterministic or LLM-backed normalization logic;
- prompt and schema usage when LLM-backed;
- the shape of normalized raw output.
The framework owns:
- passing accepted merge output to the normalizer;
- lane identity;
- source context;
- references;
- runtime LLM plumbing;
- validation handoff for normalized output;
- manifest provenance.
Conceptually:
```go
type NormalizeRequest struct {
LaneID string
Source *source.SourceDocument
MergeOutput MergeOutput
SourceInput contracts.LLMInputMaterial
SessionID string
References contracts.ReferenceSet
LLMClient contracts.StructuredLLMClient
LLMProfile string
Options map[string]any
Metadata map[string]any
}
type NormalizeOutput struct {
LaneID string
NormalizerKey string
RawContent []byte
MediaType string
SchemaID string
SchemaName string
SchemaVersion string
Metadata map[string]any
Warnings []contracts.Warning
}
```
The exact implementation shape can differ, but the key contract is that
normalize consumes one accepted merged output and produces one normalized raw
output for the lane.
## Output Stage Target Model
The output stage receives validated normalized output bytes and writes them to
its configured destination.
Output modules should not require normalized output to be converted into
`Artifact` values. Output modules should use the normalized output media type to
decide how to serialize or wrap the payload. A JSON output module can write raw
normalized JSON directly, while a text or Markdown output module can write text
payloads directly. Output modules may also write run manifests, warnings,
rejected outputs, and indexes.
Output modules may still choose to provide convenience layouts, grouping, or file
naming conventions, but those should be output concerns rather than constraints
on extractor or normalizer response schemas.
## Validation Relationship
Validation chains should attach to returned module outputs, not to hidden
module-internal conversions.
For chunk:
```text
chunk(input) -> SourceChunk set -> chunk validators -> extract input
```
For extract:
```text
extract(chunk) -> raw ExtractOutput -> extract validators -> merge input
```
For merge:
```text
merge(extract outputs) -> raw MergeOutput -> merge validators -> normalize input
```
For normalize:
```text
normalize(merge output) -> raw NormalizeOutput -> normalize validators -> output
```
Validators must be read-only. They inspect raw output and metadata, return
accept/reject decisions and warnings, and do not rewrite output.
An empty validator chain approves returned output for that validation point.
Framework/runtime errors remain separate from validation rejections: if a module
or Scriptorium call fails before output is returned, the pipeline reports an
execution error rather than asking validators to evaluate nonexistent output.
Rejected outputs do not pass to the next stage. An empty validator chain still
approves returned output for that validation point.
## Retry Policy
A retry means re-running the same module with the same input after that module
fails to produce valid output. Failure to produce valid output includes both:
- framework-level execution errors, such as module errors, Scriptorium errors,
provider errors, or missing returned output;
- validator rejection of returned output.
Retries should be configurable at the pipeline or lane level. Chunk retries are
per source input. Extract retries are per chunk. Merge and normalize retries,
when configured, are per lane. Retry attempts should preserve deterministic
reporting: the final accepted or rejected output should record attempt count and
enough diagnostics/provenance to understand prior failures without leaking
secrets or large payloads by default.
## Relationship To Validation Roadmap
This roadmap defines the data model that validation should evaluate. The
validation system roadmap in [`validation.md`](validation.md) defines validator
registration, mapping, execution classes, and concrete validator behavior.
The shared boundary between the roadmaps is a returned module output: validators
inspect typed chunk output or raw module-output envelopes and decide whether
that output may continue through the pipeline.

View File

@@ -118,16 +118,18 @@ Symptoms include:
Fix: Fix:
- Confirm the selected chunker, extractor, or normalizer declares the slot. The - Confirm the selected chunker, extractor, merger, or normalizer declares the slot. The
implemented `dnd/scenes` chunker and `dnd/spells` extractor declare optional implemented `dnd/scenes` chunker and `dnd/spells` extractor declare optional
`roster` and `glossary` slots. `roster` and `glossary` slots.
- Use a specific selector when more than one selected target declares the same - Use a specific selector when more than one selected target declares the same
slot. Examples include `chunk.context=./context.txt`, slot: `chunk.context=./context.txt`,
`spells.extract.context=./extract-context.txt`, and `spells.extract.context=./extract-context.txt`,
`spells.merge.context=./merge-context.txt`, or
`spells.normalize.context=./normalize-context.txt`. `spells.normalize.context=./normalize-context.txt`.
- `lane.slot=path` is valid only when exactly one selected extractor or - `lane.slot=path` is valid only when exactly one selected extractor, merger,
normalizer in that lane declares the slot. If both do, use or normalizer in that lane declares the slot. If more than one does, use
`lane.extract.slot=path` or `lane.normalize.slot=path`. `lane.extract.slot=path`, `lane.merge.slot=path`, or
`lane.normalize.slot=path`.
- Use `--without-reference selector` to remove optional config bindings; do not - Use `--without-reference selector` to remove optional config bindings; do not
pass an empty `--reference selector=`. pass an empty `--reference selector=`.
- Check whether a path came from config or CLI. Config paths are relative to - Check whether a path came from config or CLI. Config paths are relative to
@@ -198,7 +200,8 @@ Fix:
- Or use an existing Scriptorium profile ID with `--llm-profile`. - Or use an existing Scriptorium profile ID with `--llm-profile`.
Use `--llm-profile <id>` when one run should force every LLM-backed binding to Use `--llm-profile <id>` when one run should force every LLM-backed binding to
the same Scriptorium profile. the same Scriptorium profile. The override applies to effective chunk, extract,
merge, and normalize bindings.
## Missing API Key Environment Variable ## Missing API Key Environment Variable
@@ -283,17 +286,58 @@ Symptoms include:
- `create output directory` - `create output directory`
- `write output file` - `write output file`
- `output file name must` - `output file name must`
- `unsupported media type`
- `invalid JSON`
Fix: Fix:
- Ensure `--output-dir` points to a directory path or a path that can be - Ensure `--output-dir` points to a directory path or a path that can be
created. created.
- Check filesystem permissions and available disk space. - Check filesystem permissions and available disk space.
- The production JSON output encoder writes lane payloads under `lanes/` and
accepts only valid `application/json` normalized outputs. If an error names an
unsupported media type or invalid JSON, inspect the lane's merge and normalize
module output.
- If diagnostics were retained, inspect `run-report.json`, `run-manifest.json`, - If diagnostics were retained, inspect `run-report.json`, `run-manifest.json`,
and `error.log`. and `error.log`.
The CLI rejects unsafe logical output paths before writing files. The CLI rejects unsafe logical output paths before writing files.
## Raw Output Rejection
Symptoms include a successful run with:
- `validation_status` set to `rejected`;
- non-empty `rejected.json`;
- `rejected_outputs` entries in `manifest.json`.
Explanation and fixes:
- Validator rejection is a non-fatal run outcome. Rejected module outputs do not
pass to the next pipeline stage.
- Check `rejected.json` for the stage, lane, module, chunk, validator, reason,
message, and attempt count.
- Increase a module binding's `retries` only when re-running the same module
input can reasonably produce an acceptable output.
- If rejection is deterministic, fix the source input, module configuration, or
validator configuration rather than adding retries.
## Retry Exhaustion
Symptoms include:
- errors containing `failed after ... attempt(s)`;
- rejected output records with `attempt_count` greater than `1`.
Fix:
- `retries` is the number of extra attempts after the first attempt for chunk,
extract, merge, and normalize bindings.
- Framework-level errors after the last attempt fail the run.
- Validator rejections after the last attempt are recorded as rejected outputs.
- Check retained `error.log`, `run-manifest.json`, and `rejected.json` for the
operation, module key, lane, chunk, and attempt count.
## Diagnostics Directory Surprise ## Diagnostics Directory Surprise
Symptom: the diagnostics directory is missing after a successful run. Symptom: the diagnostics directory is missing after a successful run.

View File

@@ -5,14 +5,14 @@
}, },
"segments": [ "segments": [
{ {
"id": "seg-001", "id": 1,
"start": 0, "start": 0,
"end": 4, "end": 4,
"speaker": "Aria", "speaker": "Aria",
"text": "Aria raises her holy symbol and casts Cure Wounds." "text": "Aria raises her holy symbol and casts Cure Wounds."
}, },
{ {
"id": "seg-002", "id": 2,
"start": 4, "start": 4,
"end": 8, "end": 8,
"speaker": "DM", "speaker": "DM",

View File

@@ -99,7 +99,7 @@ func runPipelineCommand(args []string, stdout, stderr io.Writer, opts Options) i
referenceFlags := stringListFlag{} referenceFlags := stringListFlag{}
withoutReferenceFlags := stringListFlag{} withoutReferenceFlags := stringListFlag{}
fs.Var(&sessionID, "session-id", "prompt session identifier") fs.Var(&sessionID, "session-id", "prompt session identifier")
fs.Var(&referenceFlags, "reference", "reference binding, as slot=path, chunk.slot=path, lane.slot=path, lane.extract.slot=path, or lane.normalize.slot=path") fs.Var(&referenceFlags, "reference", "reference binding, as slot=path, chunk.slot=path, merge.slot=path, lane.slot=path, lane.extract.slot=path, lane.merge.slot=path, or lane.normalize.slot=path")
fs.Var(&withoutReferenceFlags, "without-reference", "unbind a reference, using the same selector forms as --reference") fs.Var(&withoutReferenceFlags, "without-reference", "unbind a reference, using the same selector forms as --reference")
if err := validateRunFlagValues(args); err != nil { if err := validateRunFlagValues(args); err != nil {
fmt.Fprintf(stderr, "notarius: %v\n", err) fmt.Fprintf(stderr, "notarius: %v\n", err)
@@ -275,7 +275,7 @@ func runPipelineCommand(args []string, stdout, stderr io.Writer, opts Options) i
PipelineID: effective.PipelineID, PipelineID: effective.PipelineID,
OutputPath: runOutputDir, OutputPath: runOutputDir,
DiagnosticsPath: runDir.Path(), DiagnosticsPath: runDir.Path(),
ApprovedCount: len(output.Approved), OutputCount: len(output.NormalizeOutputs),
RejectedCount: len(output.Rejected), RejectedCount: len(output.Rejected),
WarningCount: len(output.Warnings), WarningCount: len(output.Warnings),
ValidationStatus: output.Manifest.ValidationStatus, ValidationStatus: output.Manifest.ValidationStatus,
@@ -293,7 +293,7 @@ func runPipelineCommand(args []string, stdout, stderr io.Writer, opts Options) i
return failPipelineCommand(stderr, runDir, cfg.Diagnostics.Retention, fmt.Errorf("apply diagnostics retention: %w", err)) return failPipelineCommand(stderr, runDir, cfg.Diagnostics.Retention, fmt.Errorf("apply diagnostics retention: %w", err))
} }
fmt.Fprintf(stdout, "pipeline %q complete: approved=%d rejected=%d output=%s\n", effective.PipelineID, len(output.Approved), len(output.Rejected), runOutputDir) fmt.Fprintf(stdout, "pipeline %q complete: outputs=%d rejected=%d output=%s\n", effective.PipelineID, len(output.NormalizeOutputs), len(output.Rejected), runOutputDir)
if len(output.Warnings) > 0 { if len(output.Warnings) > 0 {
fmt.Fprintf(stderr, "notarius: run completed with %d warning(s)\n", len(output.Warnings)) fmt.Fprintf(stderr, "notarius: run completed with %d warning(s)\n", len(output.Warnings))
} }
@@ -305,7 +305,7 @@ type runReport struct {
PipelineID string `json:"pipeline_id"` PipelineID string `json:"pipeline_id"`
OutputPath string `json:"output_path"` OutputPath string `json:"output_path"`
DiagnosticsPath string `json:"diagnostics_path,omitempty"` DiagnosticsPath string `json:"diagnostics_path,omitempty"`
ApprovedCount int `json:"approved_count"` OutputCount int `json:"output_count"`
RejectedCount int `json:"rejected_count"` RejectedCount int `json:"rejected_count"`
WarningCount int `json:"warning_count"` WarningCount int `json:"warning_count"`
ValidationStatus string `json:"validation_status,omitempty"` ValidationStatus string `json:"validation_status,omitempty"`
@@ -492,6 +492,7 @@ func effectiveLLMProfileIDs(resolved pipeline.ResolvedPipeline) []string {
add(resolved.Chunk) add(resolved.Chunk)
for _, lane := range resolved.ArtifactLanes { for _, lane := range resolved.ArtifactLanes {
add(lane.Extract) add(lane.Extract)
add(lane.Merge)
add(lane.Normalize) add(lane.Normalize)
} }
ids := make([]string, 0, len(seen)) ids := make([]string, 0, len(seen))
@@ -834,17 +835,20 @@ func parseReferenceSelector(raw string, flagName string) (cliReferenceSelector,
if first == string(pipeline.StageChunk) { if first == string(pipeline.StageChunk) {
return cliReferenceSelector{Stage: pipeline.StageChunk, SlotName: slotName}, nil return cliReferenceSelector{Stage: pipeline.StageChunk, SlotName: slotName}, nil
} }
if first == string(pipeline.StageMerge) {
return cliReferenceSelector{Stage: pipeline.StageMerge, SlotName: slotName}, nil
}
return cliReferenceSelector{LaneID: first, SlotName: slotName}, nil return cliReferenceSelector{LaneID: first, SlotName: slotName}, nil
case 3: case 3:
laneID := strings.TrimSpace(parts[0]) laneID := strings.TrimSpace(parts[0])
stage := pipeline.ModuleStage(strings.TrimSpace(parts[1])) stage := pipeline.ModuleStage(strings.TrimSpace(parts[1]))
slotName := strings.TrimSpace(parts[2]) slotName := strings.TrimSpace(parts[2])
if stage != pipeline.StageExtract && stage != pipeline.StageNormalize { if stage != pipeline.StageExtract && stage != pipeline.StageMerge && stage != pipeline.StageNormalize {
return cliReferenceSelector{}, fmt.Errorf("%s lane-qualified selector must use lane.extract.slot or lane.normalize.slot", flagName) return cliReferenceSelector{}, fmt.Errorf("%s lane-qualified selector must use lane.extract.slot, lane.merge.slot, or lane.normalize.slot", flagName)
} }
return cliReferenceSelector{LaneID: laneID, Stage: stage, SlotName: slotName}, nil return cliReferenceSelector{LaneID: laneID, Stage: stage, SlotName: slotName}, nil
default: default:
return cliReferenceSelector{}, fmt.Errorf("%s must use slot, chunk.slot, lane.slot, lane.extract.slot, or lane.normalize.slot", flagName) return cliReferenceSelector{}, fmt.Errorf("%s must use slot, chunk.slot, merge.slot, lane.slot, lane.extract.slot, lane.merge.slot, or lane.normalize.slot", flagName)
} }
} }
@@ -944,7 +948,7 @@ func selectedReferenceTargets(cfg config.Config, pipelineID string, only []strin
} }
sort.Strings(selectedIDs) sort.Strings(selectedIDs)
targets := make([]selectedReferenceTarget, 0, 1+len(selectedIDs)*2) targets := make([]selectedReferenceTarget, 0, 1+len(selectedIDs)*3)
chunk := pipeline.Binding(profile.Chunk.Module) chunk := pipeline.Binding(profile.Chunk.Module)
chunk.Module = strings.TrimSpace(profile.Chunk.Module) chunk.Module = strings.TrimSpace(profile.Chunk.Module)
if chunk.Module == "" { if chunk.Module == "" {
@@ -977,6 +981,21 @@ func selectedReferenceTargets(cfg config.Config, pipelineID string, only []strin
slots: referenceSlotSet(extractSpec.ReferenceSlots), slots: referenceSlotSet(extractSpec.ReferenceSlots),
}) })
mergeModule := strings.TrimSpace(lane.Merge.Module)
if mergeModule == "" {
mergeModule = pipeline.DefaultMergeModule
}
mergeSpec, err := cliReferenceMergerSpec(catalog, mergeModule)
if err != nil {
return nil, fmt.Errorf("pipeline %q lane %q merge module %q: %w", strings.TrimSpace(pipelineID), laneID, mergeModule, err)
}
targets = append(targets, selectedReferenceTarget{
laneID: laneID,
stage: pipeline.StageMerge,
module: mergeModule,
slots: referenceSlotSet(mergeSpec.ReferenceSlots),
})
normalizeModule := strings.TrimSpace(lane.Normalize.Module) normalizeModule := strings.TrimSpace(lane.Normalize.Module)
if normalizeModule == "" { if normalizeModule == "" {
normalizeModule = pipeline.DefaultNormalizeModule normalizeModule = pipeline.DefaultNormalizeModule
@@ -1027,6 +1046,17 @@ func cliReferenceExtractorSpec(catalog pipeline.ModuleCatalog, module string) (p
return spec, nil return spec, nil
} }
func cliReferenceMergerSpec(catalog pipeline.ModuleCatalog, module string) (pipeline.ModuleSpec, error) {
if catalog.Mergers == nil {
return pipeline.ModuleSpec{}, fmt.Errorf("module %q is not registered", module)
}
spec, ok := catalog.Mergers.Spec(module)
if !ok {
return pipeline.ModuleSpec{}, fmt.Errorf("module %q is not registered", module)
}
return spec, nil
}
func cliReferenceNormalizerSpec(catalog pipeline.ModuleCatalog, module string) (pipeline.ModuleSpec, error) { func cliReferenceNormalizerSpec(catalog pipeline.ModuleCatalog, module string) (pipeline.ModuleSpec, error) {
if catalog.Normalizers == nil { if catalog.Normalizers == nil {
return pipeline.ModuleSpec{}, fmt.Errorf("module %q is not registered", module) return pipeline.ModuleSpec{}, fmt.Errorf("module %q is not registered", module)
@@ -1063,7 +1093,10 @@ func resolveCLIReferenceTarget(targets []selectedReferenceTarget, selector cliRe
} }
return selectedReferenceTarget{}, fmt.Errorf("reference chunk target is not selected") return selectedReferenceTarget{}, fmt.Errorf("reference chunk target is not selected")
} }
if selector.Stage == pipeline.StageExtract || selector.Stage == pipeline.StageNormalize { if selector.Stage == pipeline.StageExtract || selector.Stage == pipeline.StageMerge || selector.Stage == pipeline.StageNormalize {
if selector.LaneID == "" && selector.Stage == pipeline.StageMerge {
return resolveCLIReferenceStageTarget(targets, selector.Stage, slotName)
}
for _, target := range targets { for _, target := range targets {
if target.laneID == selector.LaneID && target.stage == selector.Stage { if target.laneID == selector.LaneID && target.stage == selector.Stage {
if _, ok := target.slots[slotName]; !ok { if _, ok := target.slots[slotName]; !ok {
@@ -1080,6 +1113,26 @@ func resolveCLIReferenceTarget(targets []selectedReferenceTarget, selector cliRe
return resolveCLIReferenceFlatTarget(targets, slotName) return resolveCLIReferenceFlatTarget(targets, slotName)
} }
func resolveCLIReferenceStageTarget(targets []selectedReferenceTarget, stage pipeline.ModuleStage, slotName string) (selectedReferenceTarget, error) {
matches := make([]selectedReferenceTarget, 0, 2)
for _, target := range targets {
if target.stage != stage {
continue
}
if _, ok := target.slots[slotName]; ok {
matches = append(matches, target)
}
}
switch len(matches) {
case 0:
return selectedReferenceTarget{}, fmt.Errorf("reference slot %q is not declared by any selected %s target", slotName, stage)
case 1:
return matches[0], nil
default:
return selectedReferenceTarget{}, fmt.Errorf("reference slot %q is declared by multiple selected %s targets (%s); use a more specific selector such as %s", slotName, stage, targetList(matches), selectorSuggestions(matches, slotName))
}
}
func resolveCLIReferenceLaneTarget(targets []selectedReferenceTarget, laneID string, slotName string) (selectedReferenceTarget, error) { func resolveCLIReferenceLaneTarget(targets []selectedReferenceTarget, laneID string, slotName string) (selectedReferenceTarget, error) {
laneSelected := false laneSelected := false
matches := make([]selectedReferenceTarget, 0, 2) matches := make([]selectedReferenceTarget, 0, 2)
@@ -1101,7 +1154,7 @@ func resolveCLIReferenceLaneTarget(targets []selectedReferenceTarget, laneID str
case 1: case 1:
return matches[0], nil return matches[0], nil
default: default:
return selectedReferenceTarget{}, fmt.Errorf("reference slot %q is declared by multiple selected targets in lane %q (%s); use %s.extract.%s or %s.normalize.%s", slotName, laneID, targetList(matches), laneID, slotName, laneID, slotName) return selectedReferenceTarget{}, fmt.Errorf("reference slot %q is declared by multiple selected targets in lane %q (%s); use a more specific selector such as %s", slotName, laneID, targetList(matches), selectorSuggestions(matches, slotName))
} }
} }

View File

@@ -690,7 +690,7 @@ func TestRunPipelineSuccessUsesProductionRegistriesAndFakeLLM(t *testing.T) {
if client.calls != 1 { if client.calls != 1 {
t.Fatalf("LLM calls = %d, want 1", client.calls) t.Fatalf("LLM calls = %d, want 1", client.calls)
} }
for _, want := range []string{"dnd-session", "approved=1", "rejected=0", outputDir} { for _, want := range []string{"dnd-session", "outputs=1", "rejected=0", outputDir} {
if !strings.Contains(stdout.String(), want) { if !strings.Contains(stdout.String(), want) {
t.Fatalf("stdout = %q, want substring %q", stdout.String(), want) t.Fatalf("stdout = %q, want substring %q", stdout.String(), want)
} }
@@ -719,8 +719,8 @@ func TestRunPipelineOnlySelectsRequestedLane(t *testing.T) {
if client.calls != 1 { if client.calls != 1 {
t.Fatalf("LLM calls = %d, want only selected lane to run once", client.calls) t.Fatalf("LLM calls = %d, want only selected lane to run once", client.calls)
} }
if !strings.Contains(stdout.String(), "approved=1") { if !strings.Contains(stdout.String(), "outputs=1") {
t.Fatalf("stdout = %q, want approved count", stdout.String()) t.Fatalf("stdout = %q, want output count", stdout.String())
} }
} }
@@ -743,7 +743,7 @@ func TestRunPipelineLLMFactoryFailure(t *testing.T) {
} }
} }
func TestRunPipelineValidationRejectionCompletesSuccessfully(t *testing.T) { func TestRunPipelineCarriesInvalidLLMSourceRefsAsRawOutput(t *testing.T) {
configPath := writeTestConfig(t, mvpConfigYAML("dnd-session", "dnd/spells")) configPath := writeTestConfig(t, mvpConfigYAML("dnd-session", "dnd/spells"))
inputPath := writeSeriatimInput(t) inputPath := writeSeriatimInput(t)
outputDir := t.TempDir() outputDir := t.TempDir()
@@ -759,8 +759,8 @@ func TestRunPipelineValidationRejectionCompletesSuccessfully(t *testing.T) {
if code != 0 { if code != 0 {
t.Fatalf("RunWithOptions() code = %d, stderr=%q", code, stderr.String()) t.Fatalf("RunWithOptions() code = %d, stderr=%q", code, stderr.String())
} }
if !strings.Contains(stdout.String(), "approved=0") || !strings.Contains(stdout.String(), "rejected=1") { if !strings.Contains(stdout.String(), "outputs=1") || !strings.Contains(stdout.String(), "rejected=0") {
t.Fatalf("stdout = %q, want rejection counts", stdout.String()) t.Fatalf("stdout = %q, want raw output count", stdout.String())
} }
} }
@@ -816,7 +816,7 @@ pipelines:
} }
} }
func TestRunConfigValidateIgnoresNonLLMStageScriptoriumProfiles(t *testing.T) { func TestRunConfigValidateIncludesMergeAndIgnoresNonLLMStageScriptoriumProfiles(t *testing.T) {
profilePath := writeScriptoriumProfileFile(t, "known", "http://profile.test/v1", "test-model") profilePath := writeScriptoriumProfileFile(t, "known", "http://profile.test/v1", "test-model")
configPath := writeTestConfig(t, `version: 2 configPath := writeTestConfig(t, `version: 2
scriptorium: scriptorium:
@@ -843,11 +843,11 @@ pipelines:
Catalog: fakeCatalog(t), Catalog: fakeCatalog(t),
}) })
if code != 0 { if code != 1 {
t.Fatalf("RunWithOptions() code = %d, stderr=%q, want success", code, stderr.String()) t.Fatalf("RunWithOptions() code = %d, want failure for missing merge profile", code)
} }
if strings.Contains(stderr.String(), "missing-") { if !strings.Contains(stderr.String(), "Scriptorium profile") || !strings.Contains(stderr.String(), "missing-merge") {
t.Fatalf("stderr = %q, want non-LLM stage profiles ignored", stderr.String()) t.Fatalf("stderr = %q, want missing merge profile error", stderr.String())
} }
} }
@@ -867,7 +867,7 @@ func TestEffectiveLLMProfileIDsUsesLLMCapableStagesOnly(t *testing.T) {
} }
got := effectiveLLMProfileIDs(resolved) got := effectiveLLMProfileIDs(resolved)
want := []string{"chunk-profile", "extract-profile", "normalize-profile"} want := []string{"chunk-profile", "extract-profile", "merge-profile", "normalize-profile"}
if !reflect.DeepEqual(got, want) { if !reflect.DeepEqual(got, want) {
t.Fatalf("effectiveLLMProfileIDs() = %#v, want %#v", got, want) t.Fatalf("effectiveLLMProfileIDs() = %#v, want %#v", got, want)
} }
@@ -1166,6 +1166,81 @@ func TestRunPipelineReferenceFlagBindsExplicitNormalizeSlot(t *testing.T) {
} }
} }
func TestRunPipelineReferenceFlagBindsExplicitMergeSlot(t *testing.T) {
configPath := writeTestConfig(t, testConfigYAML("example", "events"))
inputPath := filepath.Join(t.TempDir(), "missing.json")
referencePath := writeFile(t, "merge.md", "Merge notes\n")
diagnosticsDir := t.TempDir()
var stdout bytes.Buffer
var stderr bytes.Buffer
code := RunWithOptions([]string{
"run", "example",
"--config", configPath,
"--input", inputPath,
"--diagnostics-dir", diagnosticsDir,
"--reference", "events.merge.notes=" + referencePath,
}, &stdout, &stderr, Options{
Catalog: fakeCatalog(t, pipeline.ModuleSpec{
Key: "appendorder",
Stage: pipeline.StageMerge,
Requires: []string{"artifact"},
Provides: []string{"merged"},
ReferenceSlots: []contracts.ReferenceSlot{
{Name: "notes"},
},
}),
})
if code != 1 || !strings.Contains(stderr.String(), "read input") {
t.Fatalf("RunWithOptions() code = %d stderr=%q, want read input failure after resolution", code, stderr.String())
}
resolved := readResolvedPipeline(t, diagnosticsDir)
refs := resolved.ArtifactLanes[0].MergeReferences.Bindings
want := []pipeline.ReferenceBinding{
{LaneID: "events", SlotName: "notes", Source: referencePath, BindingSource: contracts.ReferenceBindingSourceCLI},
}
if !reflect.DeepEqual(refs, want) {
t.Fatalf("merge references = %#v, want %#v", refs, want)
}
}
func TestRunPipelineReferenceFlagBindsUnambiguousMergeSlot(t *testing.T) {
configPath := writeTestConfig(t, testConfigYAML("example", "events"))
inputPath := filepath.Join(t.TempDir(), "missing.json")
referencePath := writeFile(t, "merge.md", "Merge notes\n")
diagnosticsDir := t.TempDir()
var stdout bytes.Buffer
var stderr bytes.Buffer
code := RunWithOptions([]string{
"run", "example",
"--config", configPath,
"--input", inputPath,
"--diagnostics-dir", diagnosticsDir,
"--reference", "merge.notes=" + referencePath,
}, &stdout, &stderr, Options{
Catalog: fakeCatalog(t, pipeline.ModuleSpec{
Key: "appendorder",
Stage: pipeline.StageMerge,
Requires: []string{"artifact"},
Provides: []string{"merged"},
ReferenceSlots: []contracts.ReferenceSlot{
{Name: "notes"},
},
}),
})
if code != 1 || !strings.Contains(stderr.String(), "read input") {
t.Fatalf("RunWithOptions() code = %d stderr=%q, want read input failure after resolution", code, stderr.String())
}
resolved := readResolvedPipeline(t, diagnosticsDir)
refs := resolved.ArtifactLanes[0].MergeReferences.Bindings
if len(refs) != 1 || refs[0].Source != referencePath || refs[0].LaneID != "events" {
t.Fatalf("merge references = %#v, want unambiguous merge binding", refs)
}
}
func TestRunPipelineReferenceFlagBindsFlatSlotAcrossOneTarget(t *testing.T) { func TestRunPipelineReferenceFlagBindsFlatSlotAcrossOneTarget(t *testing.T) {
configPath := writeTestConfig(t, testConfigYAML("example", "events")) configPath := writeTestConfig(t, testConfigYAML("example", "events"))
inputPath := filepath.Join(t.TempDir(), "missing.json") inputPath := filepath.Join(t.TempDir(), "missing.json")
@@ -1366,8 +1441,8 @@ func TestRunPipelineReferenceFlagsRejectMalformedValues(t *testing.T) {
{name: "missing equals", args: []string{"--reference", "roster"}, want: "slot=path"}, {name: "missing equals", args: []string{"--reference", "roster"}, want: "slot=path"},
{name: "empty path", args: []string{"--reference", "roster="}, want: "path must not be empty"}, {name: "empty path", args: []string{"--reference", "roster="}, want: "path must not be empty"},
{name: "empty slot", args: []string{"--reference", "=./roster.yml"}, want: "slot must not be empty"}, {name: "empty slot", args: []string{"--reference", "=./roster.yml"}, want: "slot must not be empty"},
{name: "unsupported explicit stage", args: []string{"--reference", "a.b.c=./roster.yml"}, want: "lane.extract.slot or lane.normalize.slot"}, {name: "unsupported explicit stage", args: []string{"--reference", "a.b.c=./roster.yml"}, want: "lane.extract.slot, lane.merge.slot, or lane.normalize.slot"},
{name: "too many selector parts", args: []string{"--reference", "a.b.c.d=./roster.yml"}, want: "slot, chunk.slot, lane.slot, lane.extract.slot, or lane.normalize.slot"}, {name: "too many selector parts", args: []string{"--reference", "a.b.c.d=./roster.yml"}, want: "slot, chunk.slot, merge.slot, lane.slot, lane.extract.slot, lane.merge.slot, or lane.normalize.slot"},
{name: "unbind with equals", args: []string{"--without-reference", "roster=./roster.yml"}, want: "without =path"}, {name: "unbind with equals", args: []string{"--without-reference", "roster=./roster.yml"}, want: "without =path"},
} }
@@ -1393,9 +1468,10 @@ func TestRunPipelineReferenceFlagsRejectMalformedValues(t *testing.T) {
func TestRunPipelineWithoutReferenceRemovesConfigBindingsForEligibleTargets(t *testing.T) { func TestRunPipelineWithoutReferenceRemovesConfigBindingsForEligibleTargets(t *testing.T) {
configPath := writeTestConfig(t, testConfigYAMLWithPipelineReferences("example", "events", map[string]string{ configPath := writeTestConfig(t, testConfigYAMLWithPipelineReferences("example", "events", map[string]string{
"context": "./config-context.md", "context": "./config-context.md",
"notes": "./config-notes.md", "merge_notes": "./config-merge.md",
"roster": "./config-roster.yml", "notes": "./config-notes.md",
"roster": "./config-roster.yml",
})) }))
inputPath := filepath.Join(t.TempDir(), "missing.json") inputPath := filepath.Join(t.TempDir(), "missing.json")
diagnosticsDir := t.TempDir() diagnosticsDir := t.TempDir()
@@ -1409,6 +1485,7 @@ func TestRunPipelineWithoutReferenceRemovesConfigBindingsForEligibleTargets(t *t
"--diagnostics-dir", diagnosticsDir, "--diagnostics-dir", diagnosticsDir,
"--without-reference", "chunk.context", "--without-reference", "chunk.context",
"--without-reference", "events.extract.roster", "--without-reference", "events.extract.roster",
"--without-reference", "events.merge.merge_notes",
"--without-reference", "events.normalize.notes", "--without-reference", "events.normalize.notes",
}, &stdout, &stderr, Options{ }, &stdout, &stderr, Options{
Catalog: fakeCatalog(t, Catalog: fakeCatalog(t,
@@ -1421,6 +1498,15 @@ func TestRunPipelineWithoutReferenceRemovesConfigBindingsForEligibleTargets(t *t
{Name: "context"}, {Name: "context"},
}, },
}, },
pipeline.ModuleSpec{
Key: "appendorder",
Stage: pipeline.StageMerge,
Requires: []string{"artifact"},
Provides: []string{"merged"},
ReferenceSlots: []contracts.ReferenceSlot{
{Name: "merge_notes"},
},
},
pipeline.ModuleSpec{ pipeline.ModuleSpec{
Key: "fake/extract", Key: "fake/extract",
Stage: pipeline.StageExtract, Stage: pipeline.StageExtract,
@@ -1452,6 +1538,9 @@ func TestRunPipelineWithoutReferenceRemovesConfigBindingsForEligibleTargets(t *t
if refs := resolved.ArtifactLanes[0].ExtractReferences.Bindings; len(refs) != 0 { if refs := resolved.ArtifactLanes[0].ExtractReferences.Bindings; len(refs) != 0 {
t.Fatalf("extract references = %#v, want none", refs) t.Fatalf("extract references = %#v, want none", refs)
} }
if refs := resolved.ArtifactLanes[0].MergeReferences.Bindings; len(refs) != 0 {
t.Fatalf("merge references = %#v, want none", refs)
}
if refs := resolved.ArtifactLanes[0].NormalizeReferences.Bindings; len(refs) != 0 { if refs := resolved.ArtifactLanes[0].NormalizeReferences.Bindings; len(refs) != 0 {
t.Fatalf("normalize references = %#v, want none", refs) t.Fatalf("normalize references = %#v, want none", refs)
} }
@@ -1643,8 +1732,8 @@ func TestRunPipelineWritesDurableOutputFiles(t *testing.T) {
runOutputDir := onlyChildDir(t, outputDir) runOutputDir := onlyChildDir(t, outputDir)
for _, name := range []string{ for _, name := range []string{
"index.json", "index.json",
"lanes/spells.json",
"manifest.json", "manifest.json",
"artifacts/dnd.spell_cast.json",
"rejected.json", "rejected.json",
"warnings.json", "warnings.json",
} { } {
@@ -1819,7 +1908,7 @@ func TestRunPipelineWritesDiagnosticsArtifactsOnSuccess(t *testing.T) {
} }
} }
report := string(readFile(t, filepath.Join(runDir, diagnostics.ArtifactRunReport))) report := string(readFile(t, filepath.Join(runDir, diagnostics.ArtifactRunReport)))
if !strings.Contains(report, `"approved_count": 1`) || !strings.Contains(report, `"validation_status": "approved"`) || !strings.Contains(report, outputDir) { if !strings.Contains(report, `"output_count": 1`) || !strings.Contains(report, `"validation_status": "approved"`) || !strings.Contains(report, outputDir) {
t.Fatalf("unexpected run report: %s", report) t.Fatalf("unexpected run report: %s", report)
} }
} }
@@ -1989,30 +2078,27 @@ func TestExampleFixtureRunWritesExpectedJSON(t *testing.T) {
t.Fatalf("extractor metadata = %#v, want prompt/schema identifiers", extractorMetadata) t.Fatalf("extractor metadata = %#v, want prompt/schema identifiers", extractorMetadata)
} }
var artifactFile struct { var spellOutput struct {
ArtifactType string `json:"artifact_type"` SpellCasts []struct {
Artifacts []artifacts.Artifact `json:"artifacts"` Caster string `json:"caster"`
Spell string `json:"spell"`
Effect string `json:"effect"`
SourceRefs []source.SourceRef `json:"source_refs"`
} `json:"spell_casts"`
} }
readJSONFile(t, filepath.Join(runOutputDir, "artifacts", "dnd.spell_cast.json"), &artifactFile) readJSONFile(t, filepath.Join(runOutputDir, "lanes", "spells.json"), &spellOutput)
if artifactFile.ArtifactType != "dnd.spell_cast" || len(artifactFile.Artifacts) != 1 { if len(spellOutput.SpellCasts) != 1 {
t.Fatalf("artifact file = %#v, want one spell artifact", artifactFile) t.Fatalf("spell output = %#v, want one spell cast", spellOutput)
}
var payload struct {
Caster string `json:"caster"`
Spell string `json:"spell"`
Effect string `json:"effect"`
}
if err := json.Unmarshal(artifactFile.Artifacts[0].Payload, &payload); err != nil {
t.Fatalf("unmarshal spell payload: %v", err)
} }
payload := spellOutput.SpellCasts[0]
if payload.Caster != "Aria" || payload.Spell != "Cure Wounds" || payload.Effect == "" { if payload.Caster != "Aria" || payload.Spell != "Cure Wounds" || payload.Effect == "" {
t.Fatalf("payload = %#v, want deterministic spell output", payload) t.Fatalf("payload = %#v, want deterministic spell output", payload)
} }
if len(artifactFile.Artifacts[0].SourceRefs) != 1 { if len(payload.SourceRefs) != 1 {
t.Fatalf("source refs = %#v, want one source ref", artifactFile.Artifacts[0].SourceRefs) t.Fatalf("source refs = %#v, want one source ref", payload.SourceRefs)
} }
ref := artifactFile.Artifacts[0].SourceRefs[0] ref := payload.SourceRefs[0]
if ref.SourceID != "session-alpha" || ref.StartUnitID != "seg-001" || ref.EndUnitID != "seg-001" { if ref.SourceID != "session-alpha" || ref.StartUnitID != 1 || ref.EndUnitID != 1 {
t.Fatalf("source ref = %#v, want fixture source ref", ref) t.Fatalf("source ref = %#v, want fixture source ref", ref)
} }
@@ -2168,18 +2254,18 @@ func TestExampleFixtureFailureCoverage(t *testing.T) {
wantStderr: "completion unavailable", wantStderr: "completion unavailable",
}, },
{ {
name: "malformed LLM response", name: "malformed LLM response carried as raw output",
args: []string{"run", "dnd-session", "--config", configPath, "--input", inputPath}, args: []string{"run", "dnd-session", "--config", configPath, "--input", inputPath},
factory: fakeLLMFactory(newMalformedRunLLMClient(), nil), factory: fakeLLMFactory(newMalformedRunLLMClient(), nil),
wantCode: 1, wantCode: 0,
wantStderr: "spell_casts", wantOutputStatus: "approved",
}, },
{ {
name: "invalid source reference rejection", name: "invalid source reference raw output",
args: []string{"run", "dnd-session", "--config", configPath, "--input", inputPath}, args: []string{"run", "dnd-session", "--config", configPath, "--input", inputPath},
factory: fakeLLMFactory(newFakeRunLLMClient(true), nil), factory: fakeLLMFactory(newFakeRunLLMClient(true), nil),
wantCode: 0, wantCode: 0,
wantOutputStatus: "rejected", wantOutputStatus: "approved",
}, },
} }
@@ -2213,10 +2299,6 @@ func TestExampleFixtureFailureCoverage(t *testing.T) {
if manifest.ValidationStatus != test.wantOutputStatus { if manifest.ValidationStatus != test.wantOutputStatus {
t.Fatalf("validation status = %q, want %q", manifest.ValidationStatus, test.wantOutputStatus) t.Fatalf("validation status = %q, want %q", manifest.ValidationStatus, test.wantOutputStatus)
} }
rejected := string(readFile(t, filepath.Join(runOutputDir, "rejected.json")))
if !strings.Contains(rejected, "invalid_source_ref") {
t.Fatalf("rejected output = %s, want invalid source ref rejection", rejected)
}
} }
}) })
} }
@@ -2411,7 +2493,7 @@ func writeSeriatimInput(t *testing.T) string {
}, },
"segments": [ "segments": [
{ {
"id": "seg-001", "id": 1,
"start": 0, "start": 0,
"end": 1, "end": 1,
"speaker": "Aria", "speaker": "Aria",
@@ -2454,8 +2536,8 @@ func (client *fakeRunLLMClient) CompleteStructured(ctx context.Context, req cont
payload := map[string]any{ payload := map[string]any{
"scenes": []map[string]any{ "scenes": []map[string]any{
{ {
"start_unit_id": "seg-001", "start_unit_id": 1,
"end_unit_id": "seg-002", "end_unit_id": 2,
"short_title": "Opening spell", "short_title": "Opening spell",
"primary_mode": "Narrative", "primary_mode": "Narrative",
"main_participants": []string{"Aria"}, "main_participants": []string{"Aria"},
@@ -2478,9 +2560,9 @@ func (client *fakeRunLLMClient) CompleteStructured(ctx context.Context, req cont
} }
return contracts.StructuredCompletionResponse{Content: encoded}, nil return contracts.StructuredCompletionResponse{Content: encoded}, nil
} }
startUnitID := "seg-001" startUnitID := 1
if client.invalidSourceRef { if client.invalidSourceRef {
startUnitID = "missing-segment" startUnitID = 999
} }
payload := client.payload payload := client.payload
if payload == nil { if payload == nil {
@@ -2491,11 +2573,11 @@ func (client *fakeRunLLMClient) CompleteStructured(ctx context.Context, req cont
"spell": "Cure Wounds", "spell": "Cure Wounds",
"effect": "Heals a wounded ally.", "effect": "Heals a wounded ally.",
"narrative_description": "Aria casts Cure Wounds.", "narrative_description": "Aria casts Cure Wounds.",
"source_refs": []map[string]string{ "source_refs": []map[string]any{
{ {
"source_id": "session-alpha", "source_id": "session-alpha",
"start_unit_id": startUnitID, "start_unit_id": startUnitID,
"end_unit_id": "seg-001", "end_unit_id": 1,
}, },
}, },
}, },
@@ -2653,7 +2735,7 @@ func (fakeRunInputAdapter) Parse(ctx context.Context, req contracts.ParseRequest
Format: "test", Format: "test",
Digest: "sha256:source", Digest: "sha256:source",
Units: []source.SourceUnit{ Units: []source.SourceUnit{
{ID: "unit-1", Kind: "text", Text: string(req.Raw)}, {ID: 1, Kind: "text", Text: string(req.Raw)},
}, },
}, nil }, nil
} }
@@ -2671,7 +2753,7 @@ func (fakeRunChunker) ReferenceSlots() []contracts.ReferenceSlot {
func (fakeRunChunker) Chunk(ctx context.Context, req contracts.ChunkRequest) (contracts.ChunkResult, error) { func (fakeRunChunker) Chunk(ctx context.Context, req contracts.ChunkRequest) (contracts.ChunkResult, error) {
return contracts.ChunkResult{ return contracts.ChunkResult{
Chunks: []contracts.SourceChunk{ Chunks: []contracts.SourceChunk{
{ID: "chunk-1", SourceID: req.Source.ID, Index: 0, Units: req.Source.Units}, {ID: "chunk-1", SourceID: req.Source.ID, Index: 0, StartUnitID: 1, EndUnitID: 1, Content: []byte(`{"units":[1]}`), MediaType: "application/json", Units: req.Source.Units},
}, },
}, nil }, nil
} }
@@ -2682,30 +2764,17 @@ func (fakeRunExtractor) Key() string {
return "fake/extract" return "fake/extract"
} }
func (fakeRunExtractor) ArtifactType() string {
return "fake.artifact"
}
func (fakeRunExtractor) SchemaVersion() string {
return "v1"
}
func (fakeRunExtractor) ReferenceSlots() []contracts.ReferenceSlot { func (fakeRunExtractor) ReferenceSlots() []contracts.ReferenceSlot {
return []contracts.ReferenceSlot{{Name: "roster"}} return []contracts.ReferenceSlot{{Name: "roster"}}
} }
func (fakeRunExtractor) Validators() []contracts.Validator {
return nil
}
func (fakeRunExtractor) Extract(ctx context.Context, req contracts.ExtractionRequest) (contracts.ExtractionResult, error) { func (fakeRunExtractor) Extract(ctx context.Context, req contracts.ExtractionRequest) (contracts.ExtractionResult, error) {
return contracts.ExtractionResult{ return contracts.ExtractionResult{
Candidates: []artifacts.ArtifactCandidate{ Output: contracts.ExtractOutput{
{ Schema: contracts.ResponseSchema{ID: "fake.artifact", Name: "fake_artifact", Version: "v1"},
Payload: []byte(`{"value":true}`), Payload: contracts.RawPayload{
SourceRefs: []source.SourceRef{ Content: []byte(`{"value":true}`),
{SourceID: "source", StartUnitID: "unit-1", EndUnitID: "unit-1"}, MediaType: "application/json",
},
}, },
}, },
}, nil }, nil
@@ -2718,11 +2787,20 @@ func (fakeRunMerger) Key() string {
} }
func (fakeRunMerger) Merge(ctx context.Context, req contracts.MergeRequest) (contracts.MergeResult, error) { func (fakeRunMerger) Merge(ctx context.Context, req contracts.MergeRequest) (contracts.MergeResult, error) {
var candidates []artifacts.ArtifactCandidate output := contracts.MergeOutput{
for _, chunkArtifacts := range req.ChunkArtifacts { LaneID: req.LaneID,
candidates = append(candidates, chunkArtifacts.Candidates...) SourceID: req.Source.ID,
Schema: contracts.ResponseSchema{ID: "fake.artifact", Name: "fake_artifact", Version: "v1"},
Payload: contracts.RawPayload{
Content: []byte(`{"merged":true}`),
MediaType: "application/json",
},
} }
return contracts.MergeResult{Candidates: candidates}, nil if len(req.ExtractOutputs) > 0 {
output.Schema = req.ExtractOutputs[0].Schema
output.Payload = req.ExtractOutputs[0].Payload
}
return contracts.MergeResult{Output: output}, nil
} }
type fakeRunNormalizer struct{} type fakeRunNormalizer struct{}
@@ -2736,7 +2814,14 @@ func (fakeRunNormalizer) ReferenceSlots() []contracts.ReferenceSlot {
} }
func (fakeRunNormalizer) Normalize(ctx context.Context, req contracts.NormalizeRequest) (contracts.NormalizeResult, error) { func (fakeRunNormalizer) Normalize(ctx context.Context, req contracts.NormalizeRequest) (contracts.NormalizeResult, error) {
return contracts.NormalizeResult{Candidates: append([]artifacts.ArtifactCandidate(nil), req.Candidates...)}, nil return contracts.NormalizeResult{
Output: contracts.NormalizeOutput{
LaneID: req.LaneID,
SourceID: req.MergeOutput.SourceID,
Schema: req.MergeOutput.Schema,
Payload: req.MergeOutput.Payload,
},
}, nil
} }
func onlyChildDir(t *testing.T, root string) string { func onlyChildDir(t *testing.T, root string) string {

View File

@@ -60,26 +60,55 @@ type ReferenceProvenance struct {
BindingSource string `json:"binding_source,omitempty"` BindingSource string `json:"binding_source,omitempty"`
} }
type OutputSchemaProvenance struct {
ID string `json:"id,omitempty"`
Name string `json:"name,omitempty"`
Version string `json:"version,omitempty"`
}
type NormalizedOutputManifest struct {
LaneID string `json:"lane_id"`
ModuleKey string `json:"module_key,omitempty"`
SourceID string `json:"source_id,omitempty"`
MediaType string `json:"media_type,omitempty"`
Schema OutputSchemaProvenance `json:"schema,omitempty"`
}
type RejectedOutputManifest struct {
Stage string `json:"stage"`
LaneID string `json:"lane_id,omitempty"`
ModuleKey string `json:"module_key,omitempty"`
ChunkID string `json:"chunk_id,omitempty"`
ChunkIndex int `json:"chunk_index,omitempty"`
ValidatorName string `json:"validator_name,omitempty"`
ReasonCode string `json:"reason_code,omitempty"`
Message string `json:"message,omitempty"`
AttemptCount int `json:"attempt_count,omitempty"`
DiagnosticArtifactPath string `json:"diagnostic_artifact_path,omitempty"`
}
type RunManifest struct { type RunManifest struct {
RunID string `json:"run_id,omitempty"` RunID string `json:"run_id,omitempty"`
PipelineID string `json:"pipeline_id,omitempty"` PipelineID string `json:"pipeline_id,omitempty"`
PipelineDigest string `json:"pipeline_digest,omitempty"` PipelineDigest string `json:"pipeline_digest,omitempty"`
InputModule string `json:"input_module,omitempty"` InputModule string `json:"input_module,omitempty"`
Chunker string `json:"chunker,omitempty"` Chunker string `json:"chunker,omitempty"`
SourceDigests []string `json:"source_digests,omitempty"` SourceDigests []string `json:"source_digests,omitempty"`
Extractors []string `json:"extractors,omitempty"` Extractors []string `json:"extractors,omitempty"`
Merger string `json:"merger,omitempty"` Merger string `json:"merger,omitempty"`
Normalizer string `json:"normalizer,omitempty"` Normalizer string `json:"normalizer,omitempty"`
OutputEncoder string `json:"output_encoder,omitempty"` OutputEncoder string `json:"output_encoder,omitempty"`
ModuleMetadata map[string]map[string]any `json:"module_metadata,omitempty"` ModuleMetadata map[string]map[string]any `json:"module_metadata,omitempty"`
ArtifactLanes []ArtifactLaneManifest `json:"artifact_lanes,omitempty"` ArtifactLanes []ArtifactLaneManifest `json:"artifact_lanes,omitempty"`
References []ReferenceProvenance `json:"references,omitempty"` References []ReferenceProvenance `json:"references,omitempty"`
LLMProfiles []LLMProfileManifest `json:"llm_profiles,omitempty"` NormalizedOutputs []NormalizedOutputManifest `json:"normalized_outputs,omitempty"`
Metadata map[string]any `json:"metadata,omitempty"` RejectedOutputs []RejectedOutputManifest `json:"rejected_outputs,omitempty"`
SchemaVersion string `json:"schema_version,omitempty"` LLMProfiles []LLMProfileManifest `json:"llm_profiles,omitempty"`
ValidationStatus string `json:"validation_status,omitempty"` Metadata map[string]any `json:"metadata,omitempty"`
StartedAt *time.Time `json:"started_at,omitempty"` SchemaVersion string `json:"schema_version,omitempty"`
CompletedAt *time.Time `json:"completed_at,omitempty"` ValidationStatus string `json:"validation_status,omitempty"`
StartedAt *time.Time `json:"started_at,omitempty"`
CompletedAt *time.Time `json:"completed_at,omitempty"`
} }
func ArtifactFromCandidate(candidate ArtifactCandidate) Artifact { func ArtifactFromCandidate(candidate ArtifactCandidate) Artifact {

View File

@@ -16,7 +16,7 @@ func TestArtifactFromCandidatePreservesCandidateFields(t *testing.T) {
SchemaVersion: "v1", SchemaVersion: "v1",
Payload: json.RawMessage(`{"name":"example"}`), Payload: json.RawMessage(`{"name":"example"}`),
SourceRefs: []source.SourceRef{ SourceRefs: []source.SourceRef{
{SourceID: "source-1", StartUnitID: "u1", EndUnitID: "u2"}, {SourceID: "source-1", StartUnitID: 1, EndUnitID: 2},
}, },
Metadata: map[string]any{ Metadata: map[string]any{
"confidence": 0.75, "confidence": 0.75,
@@ -45,13 +45,13 @@ func TestArtifactFromCandidatePreservesCandidateFields(t *testing.T) {
} }
candidate.Payload[0] = '[' candidate.Payload[0] = '['
candidate.SourceRefs[0].StartUnitID = "changed" candidate.SourceRefs[0].StartUnitID = 99
candidate.Metadata["confidence"] = 0.5 candidate.Metadata["confidence"] = 0.5
if string(artifact.Payload) != `{"name":"example"}` { if string(artifact.Payload) != `{"name":"example"}` {
t.Fatalf("Payload changed after candidate mutation: %s", artifact.Payload) t.Fatalf("Payload changed after candidate mutation: %s", artifact.Payload)
} }
if artifact.SourceRefs[0].StartUnitID != "u1" { if artifact.SourceRefs[0].StartUnitID != 1 {
t.Fatalf("SourceRefs changed after candidate mutation: %#v", artifact.SourceRefs) t.Fatalf("SourceRefs changed after candidate mutation: %#v", artifact.SourceRefs)
} }
if artifact.Metadata["confidence"] != 0.75 { if artifact.Metadata["confidence"] != 0.75 {
@@ -67,7 +67,7 @@ func TestJSONMarshalUsesExpectedFieldNames(t *testing.T) {
SchemaVersion: "v1", SchemaVersion: "v1",
Payload: json.RawMessage(`{"value":true}`), Payload: json.RawMessage(`{"value":true}`),
SourceRefs: []source.SourceRef{ SourceRefs: []source.SourceRef{
{SourceID: "source-1", StartUnitID: "u1", EndUnitID: "u1"}, {SourceID: "source-1", StartUnitID: 1, EndUnitID: 1},
}, },
Metadata: map[string]any{ Metadata: map[string]any{
"reviewed": true, "reviewed": true,

View File

@@ -68,6 +68,7 @@ func applyLLMProfileOverride(profile *pipeline.PipelineProfile, profileID string
profile.Chunk.LLMProfile = profileID profile.Chunk.LLMProfile = profileID
for laneID, lane := range profile.Artifacts { for laneID, lane := range profile.Artifacts {
lane.Extract.LLMProfile = profileID lane.Extract.LLMProfile = profileID
lane.Merge.LLMProfile = profileID
lane.Normalize.LLMProfile = profileID lane.Normalize.LLMProfile = profileID
profile.Artifacts[laneID] = lane profile.Artifacts[laneID] = lane
} }

View File

@@ -193,8 +193,8 @@ func TestResolveLLMProfileOverrideAppliesBeforeDigest(t *testing.T) {
t.Fatalf("output profile = %q, want original output-profile", effective.ResolvedPipeline.Output.LLMProfile) t.Fatalf("output profile = %q, want original output-profile", effective.ResolvedPipeline.Output.LLMProfile)
} }
eventLane := effective.ResolvedPipeline.ArtifactLanes[0] eventLane := effective.ResolvedPipeline.ArtifactLanes[0]
if eventLane.Merge.LLMProfile != "merge-profile" { if eventLane.Merge.LLMProfile != "runtime" {
t.Fatalf("merge profile = %q, want original merge-profile", eventLane.Merge.LLMProfile) t.Fatalf("merge profile = %q, want runtime", eventLane.Merge.LLMProfile)
} }
if len(eventLane.Validators) != 1 || eventLane.Validators[0].LLMProfile != "validator-profile" { if len(eventLane.Validators) != 1 || eventLane.Validators[0].LLMProfile != "validator-profile" {
t.Fatalf("validator profiles = %#v, want original validator-profile", eventLane.Validators) t.Fatalf("validator profiles = %#v, want original validator-profile", eventLane.Validators)
@@ -204,7 +204,7 @@ func TestResolveLLMProfileOverrideAppliesBeforeDigest(t *testing.T) {
func llmCapableBindings(resolved pipeline.ResolvedPipeline) []pipeline.ModuleBinding { func llmCapableBindings(resolved pipeline.ResolvedPipeline) []pipeline.ModuleBinding {
bindings := []pipeline.ModuleBinding{resolved.Chunk} bindings := []pipeline.ModuleBinding{resolved.Chunk}
for _, lane := range resolved.ArtifactLanes { for _, lane := range resolved.ArtifactLanes {
bindings = append(bindings, lane.Extract, lane.Normalize) bindings = append(bindings, lane.Extract, lane.Merge, lane.Normalize)
} }
return bindings return bindings
} }

View File

@@ -53,6 +53,7 @@ type FileDiagnosticsConfig struct {
type fileModuleBinding struct { type fileModuleBinding struct {
Module string Module string
LLMProfile string LLMProfile string
Retries int
Options map[string]any Options map[string]any
References map[string]string References map[string]string
} }
@@ -83,6 +84,12 @@ func (b *fileModuleBinding) UnmarshalYAML(node *yaml.Node) error {
return err return err
} }
b.LLMProfile = strings.TrimSpace(llmProfile) b.LLMProfile = strings.TrimSpace(llmProfile)
case "retries":
var retries int
if err := valueNode.Decode(&retries); err != nil {
return err
}
b.Retries = retries
case "options": case "options":
var options map[string]any var options map[string]any
if err := valueNode.Decode(&options); err != nil { if err := valueNode.Decode(&options); err != nil {
@@ -109,6 +116,7 @@ func (b fileModuleBinding) toPipelineBinding() pipeline.ModuleBinding {
return pipeline.ModuleBinding{ return pipeline.ModuleBinding{
Module: strings.TrimSpace(b.Module), Module: strings.TrimSpace(b.Module),
LLMProfile: strings.TrimSpace(b.LLMProfile), LLMProfile: strings.TrimSpace(b.LLMProfile),
Retries: b.Retries,
Options: cloneOptions(b.Options), Options: cloneOptions(b.Options),
References: normalizedStringMap(b.References), References: normalizedStringMap(b.References),
} }

View File

@@ -127,6 +127,7 @@ pipelines:
input: fake/input input: fake/input
chunk: chunk:
module: generic module: generic
retries: 2
options: options:
size: 10 size: 10
flags: flags:
@@ -138,9 +139,12 @@ pipelines:
extract: extract:
module: fake/extract module: fake/extract
llm_profile: fast llm_profile: fast
retries: 3
options: options:
temperature: 0 temperature: 0
merge: appendorder merge:
module: appendorder
retries: 1
normalize: normalize:
module: noop module: noop
output: json output: json
@@ -153,6 +157,9 @@ pipelines:
if profile.Chunk.Module != "generic" { if profile.Chunk.Module != "generic" {
t.Fatalf("unexpected chunk binding: %+v", profile.Chunk) t.Fatalf("unexpected chunk binding: %+v", profile.Chunk)
} }
if profile.Chunk.Retries != 2 {
t.Fatalf("chunk retries = %d, want 2", profile.Chunk.Retries)
}
if profile.Chunk.Options["size"] != 10 { if profile.Chunk.Options["size"] != 10 {
t.Fatalf("expected chunk options to preserve scalar, got %#v", profile.Chunk.Options) t.Fatalf("expected chunk options to preserve scalar, got %#v", profile.Chunk.Options)
} }
@@ -168,6 +175,9 @@ pipelines:
if lane.Extract.Module != "fake/extract" || lane.Extract.LLMProfile != "fast" { if lane.Extract.Module != "fake/extract" || lane.Extract.LLMProfile != "fast" {
t.Fatalf("unexpected extract binding: %+v", lane.Extract) t.Fatalf("unexpected extract binding: %+v", lane.Extract)
} }
if lane.Extract.Retries != 3 || lane.Merge.Retries != 1 {
t.Fatalf("unexpected retries: extract=%d merge=%d", lane.Extract.Retries, lane.Merge.Retries)
}
if lane.Extract.Options["temperature"] != 0 { if lane.Extract.Options["temperature"] != 0 {
t.Fatalf("expected object options, got %#v", lane.Extract.Options) t.Fatalf("expected object options, got %#v", lane.Extract.Options)
} }
@@ -224,6 +234,10 @@ pipelines:
references: references:
roster: ./legacy-roster.yml roster: ./legacy-roster.yml
lore: ./lore.md lore: ./lore.md
merge:
module: appendorder
references:
" merge_notes ": " ./merge.md "
normalize: normalize:
module: noop module: noop
references: references:
@@ -246,6 +260,9 @@ pipelines:
if !reflect.DeepEqual(lane.Extract.References, wantExtract) { if !reflect.DeepEqual(lane.Extract.References, wantExtract) {
t.Fatalf("extract references = %#v, want legacy merged with extract override %#v", lane.Extract.References, wantExtract) t.Fatalf("extract references = %#v, want legacy merged with extract override %#v", lane.Extract.References, wantExtract)
} }
if !reflect.DeepEqual(lane.Merge.References, map[string]string{"merge_notes": "./merge.md"}) {
t.Fatalf("merge references = %#v, want trimmed map", lane.Merge.References)
}
if !reflect.DeepEqual(lane.Normalize.References, map[string]string{"normalization_notes": "./normalization.md"}) { if !reflect.DeepEqual(lane.Normalize.References, map[string]string{"normalization_notes": "./normalization.md"}) {
t.Fatalf("normalize references = %#v, want trimmed map", lane.Normalize.References) t.Fatalf("normalize references = %#v, want trimmed map", lane.Normalize.References)
} }

View File

@@ -42,6 +42,7 @@ func cloneResolvedArtifactLane(in pipeline.ResolvedArtifactLane) pipeline.Resolv
out.Merge = cloneModuleBinding(in.Merge) out.Merge = cloneModuleBinding(in.Merge)
out.Normalize = cloneModuleBinding(in.Normalize) out.Normalize = cloneModuleBinding(in.Normalize)
out.ExtractReferences = pipeline.CloneReferenceTarget(in.ExtractReferences) out.ExtractReferences = pipeline.CloneReferenceTarget(in.ExtractReferences)
out.MergeReferences = pipeline.CloneReferenceTarget(in.MergeReferences)
out.NormalizeReferences = pipeline.CloneReferenceTarget(in.NormalizeReferences) out.NormalizeReferences = pipeline.CloneReferenceTarget(in.NormalizeReferences)
if len(in.Validators) > 0 { if len(in.Validators) > 0 {
out.Validators = make([]pipeline.ModuleBinding, len(in.Validators)) out.Validators = make([]pipeline.ModuleBinding, len(in.Validators))

View File

@@ -78,7 +78,7 @@ func validatePipelineProfiles(profiles map[string]pipeline.PipelineProfile) erro
if err := validateBinding(id, laneID, "extract", lane.Extract, true); err != nil { if err := validateBinding(id, laneID, "extract", lane.Extract, true); err != nil {
return err return err
} }
if err := validateBinding(id, laneID, "merge", lane.Merge, false); err != nil { if err := validateBinding(id, laneID, "merge", lane.Merge, true); err != nil {
return err return err
} }
if err := validateBinding(id, laneID, "normalize", lane.Normalize, true); err != nil { if err := validateBinding(id, laneID, "normalize", lane.Normalize, true); err != nil {
@@ -104,6 +104,12 @@ func validateBinding(
if err := validateBindingLLMProfile(pipelineID, laneID, slot, binding); err != nil { if err := validateBindingLLMProfile(pipelineID, laneID, slot, binding); err != nil {
return err return err
} }
if binding.Retries < 0 {
if laneID != "" {
return fmt.Errorf("pipeline %q lane %q %s retries must be greater than or equal to zero", pipelineID, laneID, slot)
}
return fmt.Errorf("pipeline %q %s retries must be greater than or equal to zero", pipelineID, slot)
}
if len(binding.References) == 0 { if len(binding.References) == 0 {
return nil return nil
} }

View File

@@ -54,6 +54,18 @@ func TestValidateRejectsInvalidNumericFields(t *testing.T) {
}, },
want: "total LLM concurrency", want: "total LLM concurrency",
}, },
{
name: "negative retries",
mutate: func(cfg Config) Config {
profile := cfg.Pipelines["example"]
lane := profile.Artifacts["events"]
lane.Merge.Retries = -1
profile.Artifacts["events"] = lane
cfg.Pipelines["example"] = profile
return cfg
},
want: "retries",
},
} }
for _, tc := range tests { for _, tc := range tests {
@@ -238,18 +250,6 @@ func TestValidateRejectsReferencesOnUnsupportedBindings(t *testing.T) {
}, },
want: []string{"example", "input", "references", "not supported"}, want: []string{"example", "input", "references", "not supported"},
}, },
{
name: "merge",
mutate: func(cfg Config) Config {
profile := cfg.Pipelines["example"]
lane := profile.Artifacts["events"]
lane.Merge.References = map[string]string{"roster": "./roster.yml"}
profile.Artifacts["events"] = lane
cfg.Pipelines["example"] = profile
return cfg
},
want: []string{"example", "events", "merge", "references", "not supported"},
},
{ {
name: "validator", name: "validator",
mutate: func(cfg Config) Config { mutate: func(cfg Config) Config {

View File

@@ -10,7 +10,7 @@ type SourceDocument struct {
} }
type SourceUnit struct { type SourceUnit struct {
ID string `json:"id"` ID int `json:"id"`
Kind string `json:"kind"` Kind string `json:"kind"`
Text string `json:"text"` Text string `json:"text"`
Metadata map[string]any `json:"metadata,omitempty"` Metadata map[string]any `json:"metadata,omitempty"`
@@ -18,6 +18,6 @@ type SourceUnit struct {
type SourceRef struct { type SourceRef struct {
SourceID string `json:"source_id"` SourceID string `json:"source_id"`
StartUnitID string `json:"start_unit_id"` StartUnitID int `json:"start_unit_id"`
EndUnitID string `json:"end_unit_id"` EndUnitID int `json:"end_unit_id"`
} }

View File

@@ -96,13 +96,8 @@ func TestValidateDocumentMissingUnitFields(t *testing.T) {
}{ }{
{ {
name: "id", name: "id",
mutate: func(doc *SourceDocument) { doc.Units[1].ID = "" }, mutate: func(doc *SourceDocument) { doc.Units[1].ID = 0 },
wantErr: "source unit[1].id must not be empty", wantErr: "source unit[1].id must be positive",
},
{
name: "id surrounding whitespace",
mutate: func(doc *SourceDocument) { doc.Units[1].ID = " u2 " },
wantErr: "source unit[1].id \" u2 \" must not contain leading or trailing whitespace",
}, },
{ {
name: "kind", name: "kind",
@@ -135,14 +130,14 @@ func TestValidateDocumentMissingUnitFields(t *testing.T) {
func TestValidateDocumentDuplicateUnitIDs(t *testing.T) { func TestValidateDocumentDuplicateUnitIDs(t *testing.T) {
doc := validDocument() doc := validDocument()
doc.Units[1].ID = "u1" doc.Units[1].ID = 1
err := ValidateDocument(doc) err := ValidateDocument(doc)
if err == nil { if err == nil {
t.Fatal("ValidateDocument() error = nil, want error") t.Fatal("ValidateDocument() error = nil, want error")
} }
if err.Error() != "source unit id \"u1\" is duplicated" { if err.Error() != "source unit id 1 is duplicated" {
t.Fatalf("ValidateDocument() error = %q", err.Error()) t.Fatalf("ValidateDocument() error = %q", err.Error())
} }
} }
@@ -151,8 +146,8 @@ func TestValidateRefValid(t *testing.T) {
doc := validDocument() doc := validDocument()
ref := SourceRef{ ref := SourceRef{
SourceID: "source-1", SourceID: "source-1",
StartUnitID: "u1", StartUnitID: 1,
EndUnitID: "u2", EndUnitID: 2,
} }
if err := ValidateRef(doc, ref); err != nil { if err := ValidateRef(doc, ref); err != nil {
@@ -164,8 +159,8 @@ func TestValidateRefSourceIDMismatch(t *testing.T) {
doc := validDocument() doc := validDocument()
ref := SourceRef{ ref := SourceRef{
SourceID: "source-2", SourceID: "source-2",
StartUnitID: "u1", StartUnitID: 1,
EndUnitID: "u2", EndUnitID: 2,
} }
err := ValidateRef(doc, ref) err := ValidateRef(doc, ref)
@@ -186,43 +181,33 @@ func TestValidateRefMissingUnitIDs(t *testing.T) {
}{ }{
{ {
name: "missing source id", name: "missing source id",
ref: SourceRef{StartUnitID: "u1", EndUnitID: "u2"}, ref: SourceRef{StartUnitID: 1, EndUnitID: 2},
wantErr: "source ref source_id must not be empty", wantErr: "source ref source_id must not be empty",
}, },
{ {
name: "source id surrounding whitespace", name: "source id surrounding whitespace",
ref: SourceRef{SourceID: " source-1 ", StartUnitID: "u1", EndUnitID: "u2"}, ref: SourceRef{SourceID: " source-1 ", StartUnitID: 1, EndUnitID: 2},
wantErr: "source ref source_id \" source-1 \" must not contain leading or trailing whitespace", wantErr: "source ref source_id \" source-1 \" must not contain leading or trailing whitespace",
}, },
{ {
name: "missing start id", name: "missing start id",
ref: SourceRef{SourceID: "source-1", EndUnitID: "u2"}, ref: SourceRef{SourceID: "source-1", EndUnitID: 2},
wantErr: "source ref start_unit_id must not be empty", wantErr: "source ref start_unit_id must be positive",
},
{
name: "start id surrounding whitespace",
ref: SourceRef{SourceID: "source-1", StartUnitID: " u1 ", EndUnitID: "u2"},
wantErr: "source ref start_unit_id \" u1 \" must not contain leading or trailing whitespace",
}, },
{ {
name: "missing end id", name: "missing end id",
ref: SourceRef{SourceID: "source-1", StartUnitID: "u1"}, ref: SourceRef{SourceID: "source-1", StartUnitID: 1},
wantErr: "source ref end_unit_id must not be empty", wantErr: "source ref end_unit_id must be positive",
},
{
name: "end id surrounding whitespace",
ref: SourceRef{SourceID: "source-1", StartUnitID: "u1", EndUnitID: " u2 "},
wantErr: "source ref end_unit_id \" u2 \" must not contain leading or trailing whitespace",
}, },
{ {
name: "unknown start id", name: "unknown start id",
ref: SourceRef{SourceID: "source-1", StartUnitID: "u9", EndUnitID: "u2"}, ref: SourceRef{SourceID: "source-1", StartUnitID: 9, EndUnitID: 2},
wantErr: "source ref start_unit_id \"u9\" was not found", wantErr: "source ref start_unit_id 9 was not found",
}, },
{ {
name: "unknown end id", name: "unknown end id",
ref: SourceRef{SourceID: "source-1", StartUnitID: "u1", EndUnitID: "u9"}, ref: SourceRef{SourceID: "source-1", StartUnitID: 1, EndUnitID: 9},
wantErr: "source ref end_unit_id \"u9\" was not found", wantErr: "source ref end_unit_id 9 was not found",
}, },
} }
@@ -244,8 +229,8 @@ func TestValidateRefReversedUnitOrder(t *testing.T) {
doc := validDocument() doc := validDocument()
ref := SourceRef{ ref := SourceRef{
SourceID: "source-1", SourceID: "source-1",
StartUnitID: "u2", StartUnitID: 2,
EndUnitID: "u1", EndUnitID: 1,
} }
err := ValidateRef(doc, ref) err := ValidateRef(doc, ref)
@@ -261,7 +246,7 @@ func TestValidateRefReversedUnitOrder(t *testing.T) {
func TestUnitIndex(t *testing.T) { func TestUnitIndex(t *testing.T) {
doc := validDocument() doc := validDocument()
index, ok := UnitIndex(doc, "u2") index, ok := UnitIndex(doc, 2)
if !ok { if !ok {
t.Fatal("UnitIndex() ok = false, want true") t.Fatal("UnitIndex() ok = false, want true")
} }
@@ -269,7 +254,7 @@ func TestUnitIndex(t *testing.T) {
t.Fatalf("UnitIndex() index = %d, want 1", index) t.Fatalf("UnitIndex() index = %d, want 1", index)
} }
index, ok = UnitIndex(doc, "u9") index, ok = UnitIndex(doc, 9)
if ok { if ok {
t.Fatal("UnitIndex() ok = true, want false") t.Fatal("UnitIndex() ok = true, want false")
} }
@@ -286,12 +271,12 @@ func validDocument() *SourceDocument {
Digest: "sha256:abc123", Digest: "sha256:abc123",
Units: []SourceUnit{ Units: []SourceUnit{
{ {
ID: "u1", ID: 1,
Kind: "paragraph", Kind: "paragraph",
Text: "First unit.", Text: "First unit.",
}, },
{ {
ID: "u2", ID: 2,
Kind: "paragraph", Kind: "paragraph",
Text: "Second unit.", Text: "Second unit.",
}, },

View File

@@ -28,13 +28,10 @@ func ValidateDocument(doc *SourceDocument) error {
return fmt.Errorf("source document units must not be empty") return fmt.Errorf("source document units must not be empty")
} }
seenUnitIDs := make(map[string]struct{}, len(doc.Units)) seenUnitIDs := make(map[int]struct{}, len(doc.Units))
for i, unit := range doc.Units { for i, unit := range doc.Units {
if isBlank(unit.ID) { if unit.ID <= 0 {
return fmt.Errorf("source unit[%d].id must not be empty", i) return fmt.Errorf("source unit[%d].id must be positive", i)
}
if hasSurroundingWhitespace(unit.ID) {
return fmt.Errorf("source unit[%d].id %q must not contain leading or trailing whitespace", i, unit.ID)
} }
if isBlank(unit.Kind) { if isBlank(unit.Kind) {
return fmt.Errorf("source unit[%d].kind must not be empty", i) return fmt.Errorf("source unit[%d].kind must not be empty", i)
@@ -43,7 +40,7 @@ func ValidateDocument(doc *SourceDocument) error {
return fmt.Errorf("source unit[%d].text must not be empty", i) return fmt.Errorf("source unit[%d].text must not be empty", i)
} }
if _, ok := seenUnitIDs[unit.ID]; ok { if _, ok := seenUnitIDs[unit.ID]; ok {
return fmt.Errorf("source unit id %q is duplicated", unit.ID) return fmt.Errorf("source unit id %d is duplicated", unit.ID)
} }
seenUnitIDs[unit.ID] = struct{}{} seenUnitIDs[unit.ID] = struct{}{}
} }
@@ -61,17 +58,11 @@ func ValidateRef(doc *SourceDocument, ref SourceRef) error {
if hasSurroundingWhitespace(ref.SourceID) { if hasSurroundingWhitespace(ref.SourceID) {
return fmt.Errorf("source ref source_id %q must not contain leading or trailing whitespace", ref.SourceID) return fmt.Errorf("source ref source_id %q must not contain leading or trailing whitespace", ref.SourceID)
} }
if isBlank(ref.StartUnitID) { if ref.StartUnitID <= 0 {
return fmt.Errorf("source ref start_unit_id must not be empty") return fmt.Errorf("source ref start_unit_id must be positive")
} }
if hasSurroundingWhitespace(ref.StartUnitID) { if ref.EndUnitID <= 0 {
return fmt.Errorf("source ref start_unit_id %q must not contain leading or trailing whitespace", ref.StartUnitID) return fmt.Errorf("source ref end_unit_id must be positive")
}
if isBlank(ref.EndUnitID) {
return fmt.Errorf("source ref end_unit_id must not be empty")
}
if hasSurroundingWhitespace(ref.EndUnitID) {
return fmt.Errorf("source ref end_unit_id %q must not contain leading or trailing whitespace", ref.EndUnitID)
} }
if ref.SourceID != doc.ID { if ref.SourceID != doc.ID {
return fmt.Errorf("source ref source_id %q does not match document id %q", ref.SourceID, doc.ID) return fmt.Errorf("source ref source_id %q does not match document id %q", ref.SourceID, doc.ID)
@@ -79,20 +70,20 @@ func ValidateRef(doc *SourceDocument, ref SourceRef) error {
startIndex, ok := UnitIndex(doc, ref.StartUnitID) startIndex, ok := UnitIndex(doc, ref.StartUnitID)
if !ok { if !ok {
return fmt.Errorf("source ref start_unit_id %q was not found", ref.StartUnitID) return fmt.Errorf("source ref start_unit_id %d was not found", ref.StartUnitID)
} }
endIndex, ok := UnitIndex(doc, ref.EndUnitID) endIndex, ok := UnitIndex(doc, ref.EndUnitID)
if !ok { if !ok {
return fmt.Errorf("source ref end_unit_id %q was not found", ref.EndUnitID) return fmt.Errorf("source ref end_unit_id %d was not found", ref.EndUnitID)
} }
if startIndex > endIndex { if startIndex > endIndex {
return fmt.Errorf("source ref start_unit_id %q appears after end_unit_id %q", ref.StartUnitID, ref.EndUnitID) return fmt.Errorf("source ref start_unit_id %d appears after end_unit_id %d", ref.StartUnitID, ref.EndUnitID)
} }
return nil return nil
} }
func UnitIndex(doc *SourceDocument, unitID string) (int, bool) { func UnitIndex(doc *SourceDocument, unitID int) (int, bool) {
if doc == nil { if doc == nil {
return 0, false return 0, false
} }

View File

@@ -27,7 +27,6 @@ func TestContractsComposeAcrossPackages(t *testing.T) {
extractor := compositionExtractor{} extractor := compositionExtractor{}
merger := compositionMerger{} merger := compositionMerger{}
normalizer := compositionNormalizer{} normalizer := compositionNormalizer{}
validator := compositionValidator{}
encoder := compositionOutputEncoder{} encoder := compositionOutputEncoder{}
doc, err := adapter.Parse(ctx, contracts.ParseRequest{SourceID: "source-1"}) doc, err := adapter.Parse(ctx, contracts.ParseRequest{SourceID: "source-1"})
@@ -58,70 +57,37 @@ func TestContractsComposeAcrossPackages(t *testing.T) {
if err != nil { if err != nil {
t.Fatalf("Extract() error = %v, want nil", err) t.Fatalf("Extract() error = %v, want nil", err)
} }
if len(extraction.Candidates) != 1 { if extraction.Output.Payload.MediaType != "application/json" {
t.Fatalf("len(Candidates) = %d, want 1", len(extraction.Candidates)) t.Fatalf("extract media type = %q, want application/json", extraction.Output.Payload.MediaType)
}
candidate := extraction.Candidates[0]
for _, ref := range candidate.SourceRefs {
if err := source.ValidateRef(doc, ref); err != nil {
t.Fatalf("ValidateRef() error = %v, want nil", err)
}
} }
merge, err := merger.Merge(ctx, contracts.MergeRequest{ merge, err := merger.Merge(ctx, contracts.MergeRequest{
Source: doc, Source: doc,
LaneID: candidate.ArtifactType, LaneID: "generic-lane",
ChunkArtifacts: []contracts.ChunkArtifacts{ ExtractOutputs: []contracts.ExtractOutput{extraction.Output},
{
Chunk: chunking.Chunks[0],
Candidates: extraction.Candidates,
},
},
}) })
if err != nil { if err != nil {
t.Fatalf("Merge() error = %v, want nil", err) t.Fatalf("Merge() error = %v, want nil", err)
} }
if len(merge.Candidates) != 1 { if string(merge.Output.Payload.Content) != `{"value":"example"}` {
t.Fatalf("len(merge.Candidates) = %d, want 1", len(merge.Candidates)) t.Fatalf("merge output = %s, want extract payload", merge.Output.Payload.Content)
} }
normalize, err := normalizer.Normalize(ctx, contracts.NormalizeRequest{ normalize, err := normalizer.Normalize(ctx, contracts.NormalizeRequest{
Source: doc, Source: doc,
LaneID: candidate.ArtifactType, LaneID: "generic-lane",
Candidates: merge.Candidates, MergeOutput: merge.Output,
}) })
if err != nil { if err != nil {
t.Fatalf("Normalize() error = %v, want nil", err) t.Fatalf("Normalize() error = %v, want nil", err)
} }
if len(normalize.Candidates) != 1 { if string(normalize.Output.Payload.Content) != `{"value":"example"}` {
t.Fatalf("len(normalize.Candidates) = %d, want 1", len(normalize.Candidates)) t.Fatalf("normalize output = %s, want merge payload", normalize.Output.Payload.Content)
}
validation, err := validator.Validate(ctx, contracts.ValidationRequest{
Source: doc,
Candidates: normalize.Candidates,
})
if err != nil {
t.Fatalf("Validate() error = %v, want nil", err)
}
if len(validation.Decisions) != 1 {
t.Fatalf("len(Decisions) = %d, want 1", len(validation.Decisions))
}
decision := validation.Decisions[0]
if !decision.Approved {
t.Fatal("Approved = false, want true")
}
if decision.CandidateIndex != candidate.Index {
t.Fatalf("CandidateIndex = %d, want %d", decision.CandidateIndex, candidate.Index)
} }
output, err := encoder.Encode(ctx, contracts.OutputRequest{ output, err := encoder.Encode(ctx, contracts.OutputRequest{
Manifest: artifacts.RunManifest{RunID: "run-1"}, Manifest: artifacts.RunManifest{RunID: "run-1"},
Approved: []artifacts.Artifact{ NormalizeOutputs: []contracts.NormalizeOutput{normalize.Output},
artifacts.ArtifactFromCandidate(normalize.Candidates[0]),
},
}) })
if err != nil { if err != nil {
t.Fatalf("Encode() error = %v, want nil", err) t.Fatalf("Encode() error = %v, want nil", err)
@@ -150,8 +116,8 @@ func (adapter compositionAdapter) Parse(ctx context.Context, req contracts.Parse
Format: "text/plain", Format: "text/plain",
Digest: "sha256:abc123", Digest: "sha256:abc123",
Units: []source.SourceUnit{ Units: []source.SourceUnit{
{ID: "u1", Kind: "unit", Text: "First source unit."}, {ID: 1, Kind: "unit", Text: "First source unit."},
{ID: "u2", Kind: "unit", Text: "Second source unit."}, {ID: 2, Kind: "unit", Text: "Second source unit."},
}, },
}, nil }, nil
} }
@@ -177,11 +143,15 @@ func (chunker compositionChunker) Chunk(ctx context.Context, req contracts.Chunk
return contracts.ChunkResult{ return contracts.ChunkResult{
Chunks: []contracts.SourceChunk{ Chunks: []contracts.SourceChunk{
{ {
ID: req.Source.ID + ":chunk:0", ID: req.Source.ID + ":chunk:0",
SourceID: req.Source.ID, SourceID: req.Source.ID,
Index: 0, Index: 0,
Units: append([]source.SourceUnit(nil), req.Source.Units...), StartUnitID: req.Source.Units[0].ID,
Metadata: map[string]any{"strategy": "whole-document"}, EndUnitID: req.Source.Units[len(req.Source.Units)-1].ID,
Content: []byte(`{"units":[{"id":1,"kind":"unit","text":"First source unit."},{"id":2,"kind":"unit","text":"Second source unit."}]}`),
MediaType: "application/json",
Units: append([]source.SourceUnit(nil), req.Source.Units...),
Metadata: map[string]any{"strategy": "whole-document"},
}, },
}, },
}, nil }, nil
@@ -199,49 +169,24 @@ func (extractor compositionExtractor) Key() string {
return "generic-extractor" return "generic-extractor"
} }
func (extractor compositionExtractor) ArtifactType() string {
return "generic-artifact"
}
func (extractor compositionExtractor) SchemaVersion() string {
return "v1"
}
func (extractor compositionExtractor) ReferenceSlots() []contracts.ReferenceSlot { func (extractor compositionExtractor) ReferenceSlots() []contracts.ReferenceSlot {
return nil return nil
} }
func (extractor compositionExtractor) Validators() []contracts.Validator {
return []contracts.Validator{compositionValidator{}}
}
func (extractor compositionExtractor) Extract(ctx context.Context, req contracts.ExtractionRequest) (contracts.ExtractionResult, error) { func (extractor compositionExtractor) Extract(ctx context.Context, req contracts.ExtractionRequest) (contracts.ExtractionResult, error) {
if req.Source == nil { if req.Source == nil {
return contracts.ExtractionResult{}, errors.New("source document is required") return contracts.ExtractionResult{}, errors.New("source document is required")
} }
units := req.Source.Units
if req.Chunk != nil {
units = req.Chunk.Units
}
if req.AmbientContext["synopsis"] == "" { if req.AmbientContext["synopsis"] == "" {
return contracts.ExtractionResult{}, errors.New("ambient synopsis is required") return contracts.ExtractionResult{}, errors.New("ambient synopsis is required")
} }
return contracts.ExtractionResult{ return contracts.ExtractionResult{
Candidates: []artifacts.ArtifactCandidate{ Output: contracts.ExtractOutput{
{ Schema: contracts.ResponseSchema{ID: "schema-id", Name: "schema-name", Version: "v1"},
Index: 0, Payload: contracts.RawPayload{
ExtractorKey: extractor.Key(), Content: []byte(`{"value":"example"}`),
ArtifactType: extractor.ArtifactType(), MediaType: "application/json",
SchemaVersion: extractor.SchemaVersion(),
Payload: json.RawMessage(`{"value":"example"}`),
SourceRefs: []source.SourceRef{
{
SourceID: req.Source.ID,
StartUnitID: units[0].ID,
EndUnitID: units[len(units)-1].ID,
},
},
}, },
}, },
}, nil }, nil
@@ -254,12 +199,14 @@ func (merger compositionMerger) Key() string {
} }
func (merger compositionMerger) Merge(ctx context.Context, req contracts.MergeRequest) (contracts.MergeResult, error) { func (merger compositionMerger) Merge(ctx context.Context, req contracts.MergeRequest) (contracts.MergeResult, error) {
var candidates []artifacts.ArtifactCandidate output := req.ExtractOutputs[0]
for _, chunkArtifacts := range req.ChunkArtifacts { return contracts.MergeResult{Output: contracts.MergeOutput{
candidates = append(candidates, chunkArtifacts.Candidates...) LaneID: req.LaneID,
} MergerKey: merger.Key(),
SourceID: output.SourceID,
return contracts.MergeResult{Candidates: candidates}, nil Schema: output.Schema,
Payload: cloneCompositionPayload(output.Payload),
}}, nil
} }
type compositionNormalizer struct{} type compositionNormalizer struct{}
@@ -273,7 +220,33 @@ func (normalizer compositionNormalizer) ReferenceSlots() []contracts.ReferenceSl
} }
func (normalizer compositionNormalizer) Normalize(ctx context.Context, req contracts.NormalizeRequest) (contracts.NormalizeResult, error) { func (normalizer compositionNormalizer) Normalize(ctx context.Context, req contracts.NormalizeRequest) (contracts.NormalizeResult, error) {
return contracts.NormalizeResult{Candidates: req.Candidates}, nil return contracts.NormalizeResult{Output: contracts.NormalizeOutput{
LaneID: req.LaneID,
NormalizerKey: normalizer.Key(),
SourceID: req.MergeOutput.SourceID,
Schema: req.MergeOutput.Schema,
Payload: cloneCompositionPayload(req.MergeOutput.Payload),
}}, nil
}
func cloneCompositionPayload(payload contracts.RawPayload) contracts.RawPayload {
return contracts.RawPayload{
Content: append([]byte(nil), payload.Content...),
MediaType: payload.MediaType,
Metadata: cloneCompositionMetadata(payload.Metadata),
Warnings: append([]contracts.Warning(nil), payload.Warnings...),
}
}
func cloneCompositionMetadata(metadata map[string]any) map[string]any {
if len(metadata) == 0 {
return nil
}
out := make(map[string]any, len(metadata))
for key, value := range metadata {
out[key] = value
}
return out
} }
type compositionValidator struct{} type compositionValidator struct{}
@@ -307,11 +280,11 @@ func (encoder compositionOutputEncoder) Key() string {
func (encoder compositionOutputEncoder) Encode(ctx context.Context, req contracts.OutputRequest) (contracts.OutputResult, error) { func (encoder compositionOutputEncoder) Encode(ctx context.Context, req contracts.OutputRequest) (contracts.OutputResult, error) {
payload := struct { payload := struct {
RunID string `json:"run_id"` RunID string `json:"run_id"`
ApprovedCount int `json:"approved_count"` OutputCount int `json:"output_count"`
}{ }{
RunID: req.Manifest.RunID, RunID: req.Manifest.RunID,
ApprovedCount: len(req.Approved), OutputCount: len(req.NormalizeOutputs),
} }
encoded, err := json.Marshal(payload) encoded, err := json.Marshal(payload)
if err != nil { if err != nil {

View File

@@ -89,11 +89,15 @@ type InputAdapter interface {
} }
type SourceChunk struct { type SourceChunk struct {
ID string `json:"id"` ID string `json:"id"`
SourceID string `json:"source_id"` SourceID string `json:"source_id"`
Index int `json:"index"` Index int `json:"index"`
Units []source.SourceUnit `json:"units"` StartUnitID int `json:"start_unit_id"`
Metadata map[string]any `json:"metadata,omitempty"` EndUnitID int `json:"end_unit_id"`
Content []byte `json:"-"`
MediaType string `json:"media_type"`
Units []source.SourceUnit `json:"units"`
Metadata map[string]any `json:"metadata,omitempty"`
} }
type ChunkRequest struct { type ChunkRequest struct {
@@ -182,36 +186,89 @@ type ExtractionRequest struct {
} }
type ExtractionResult struct { type ExtractionResult struct {
Candidates []artifacts.ArtifactCandidate `json:"candidates,omitempty"` Output ExtractOutput `json:"output"`
Warnings []Warning `json:"warnings,omitempty"` Warnings []Warning `json:"warnings,omitempty"`
} }
type Extractor interface { type Extractor interface {
Key() string Key() string
ArtifactType() string
SchemaVersion() string
ReferenceSlots() []ReferenceSlot ReferenceSlots() []ReferenceSlot
Validators() []Validator
Extract(ctx context.Context, req ExtractionRequest) (ExtractionResult, error) Extract(ctx context.Context, req ExtractionRequest) (ExtractionResult, error)
} }
type ChunkArtifacts struct { type RawPayload struct {
Chunk SourceChunk `json:"chunk"` Content []byte `json:"-"`
Candidates []artifacts.ArtifactCandidate `json:"candidates"` MediaType string `json:"media_type"`
Metadata map[string]any `json:"metadata,omitempty"`
Warnings []Warning `json:"warnings,omitempty"`
}
type RawValidationRequest struct {
Stage string `json:"stage"`
LaneID string `json:"lane_id,omitempty"`
ModuleKey string `json:"module_key"`
Source *source.SourceDocument `json:"-"`
SourceID string `json:"source_id,omitempty"`
ChunkID string `json:"chunk_id,omitempty"`
ChunkIndex int `json:"chunk_index,omitempty"`
Schema ResponseSchema `json:"schema,omitempty"`
Payload RawPayload `json:"payload"`
Metadata map[string]any `json:"metadata,omitempty"`
}
type RawValidationResult struct {
Approved bool `json:"approved"`
ReasonCode string `json:"reason_code,omitempty"`
Message string `json:"message,omitempty"`
DiagnosticArtifactPath string `json:"diagnostic_artifact_path,omitempty"`
Warnings []Warning `json:"warnings,omitempty"`
}
type RawValidator interface {
Name() string
ValidateRaw(ctx context.Context, req RawValidationRequest) (RawValidationResult, error)
}
type ResponseSchema struct {
ID string `json:"id,omitempty"`
Name string `json:"name,omitempty"`
Version string `json:"version,omitempty"`
}
type ExtractOutput struct {
LaneID string `json:"lane_id"`
ExtractorKey string `json:"extractor_key"`
SourceID string `json:"source_id"`
ChunkID string `json:"chunk_id"`
ChunkIndex int `json:"chunk_index"`
Schema ResponseSchema `json:"schema,omitempty"`
Payload RawPayload `json:"payload"`
} }
type MergeRequest struct { type MergeRequest struct {
Source *source.SourceDocument `json:"-"` Source *source.SourceDocument `json:"-"`
LaneID string `json:"lane_id"` LaneID string `json:"lane_id"`
ChunkArtifacts []ChunkArtifacts `json:"chunk_artifacts"` ExtractOutputs []ExtractOutput `json:"extract_outputs"`
SourceInput LLMInputMaterial `json:"source_input,omitempty"`
SessionID string `json:"session_id,omitempty"`
References ReferenceSet `json:"references,omitempty"`
LLMClient StructuredLLMClient `json:"-"`
LLMProfile string `json:"llm_profile,omitempty"` LLMProfile string `json:"llm_profile,omitempty"`
Options map[string]any `json:"options,omitempty"` Options map[string]any `json:"options,omitempty"`
Metadata map[string]any `json:"metadata,omitempty"` Metadata map[string]any `json:"metadata,omitempty"`
} }
type MergeResult struct { type MergeResult struct {
Candidates []artifacts.ArtifactCandidate `json:"candidates"` Output MergeOutput `json:"output"`
Warnings []Warning `json:"warnings,omitempty"` Warnings []Warning `json:"warnings,omitempty"`
}
type MergeOutput struct {
LaneID string `json:"lane_id"`
MergerKey string `json:"merger_key"`
SourceID string `json:"source_id,omitempty"`
Schema ResponseSchema `json:"schema,omitempty"`
Payload RawPayload `json:"payload"`
} }
type Merger interface { type Merger interface {
@@ -220,21 +277,29 @@ type Merger interface {
} }
type NormalizeRequest struct { type NormalizeRequest struct {
Source *source.SourceDocument `json:"-"` Source *source.SourceDocument `json:"-"`
LaneID string `json:"lane_id"` LaneID string `json:"lane_id"`
Candidates []artifacts.ArtifactCandidate `json:"candidates"` MergeOutput MergeOutput `json:"merge_output"`
SourceInput LLMInputMaterial `json:"source_input,omitempty"` SourceInput LLMInputMaterial `json:"source_input,omitempty"`
SessionID string `json:"session_id,omitempty"` SessionID string `json:"session_id,omitempty"`
References ReferenceSet `json:"references,omitempty"` References ReferenceSet `json:"references,omitempty"`
LLMClient StructuredLLMClient `json:"-"` LLMClient StructuredLLMClient `json:"-"`
LLMProfile string `json:"llm_profile,omitempty"` LLMProfile string `json:"llm_profile,omitempty"`
Options map[string]any `json:"options,omitempty"` Options map[string]any `json:"options,omitempty"`
Metadata map[string]any `json:"metadata,omitempty"` Metadata map[string]any `json:"metadata,omitempty"`
} }
type NormalizeResult struct { type NormalizeResult struct {
Candidates []artifacts.ArtifactCandidate `json:"candidates"` Output NormalizeOutput `json:"output"`
Warnings []Warning `json:"warnings,omitempty"` Warnings []Warning `json:"warnings,omitempty"`
}
type NormalizeOutput struct {
LaneID string `json:"lane_id"`
NormalizerKey string `json:"normalizer_key"`
SourceID string `json:"source_id,omitempty"`
Schema ResponseSchema `json:"schema,omitempty"`
Payload RawPayload `json:"payload"`
} }
type Normalizer interface { type Normalizer interface {
@@ -277,13 +342,13 @@ type Warning struct {
} }
type OutputRequest struct { type OutputRequest struct {
Manifest artifacts.RunManifest `json:"manifest"` Manifest artifacts.RunManifest `json:"manifest"`
Approved []artifacts.Artifact `json:"approved,omitempty"` NormalizeOutputs []NormalizeOutput `json:"normalize_outputs,omitempty"`
Rejected []artifacts.RejectedArtifact `json:"rejected,omitempty"` Rejected []RejectedOutput `json:"rejected,omitempty"`
Warnings []Warning `json:"warnings,omitempty"` Warnings []Warning `json:"warnings,omitempty"`
LLMProfile string `json:"llm_profile,omitempty"` LLMProfile string `json:"llm_profile,omitempty"`
Options map[string]any `json:"options,omitempty"` Options map[string]any `json:"options,omitempty"`
Metadata map[string]any `json:"metadata,omitempty"` Metadata map[string]any `json:"metadata,omitempty"`
} }
type OutputFile struct { type OutputFile struct {
@@ -302,6 +367,19 @@ type OutputEncoder interface {
Encode(ctx context.Context, req OutputRequest) (OutputResult, error) Encode(ctx context.Context, req OutputRequest) (OutputResult, error)
} }
type RejectedOutput struct {
Stage string `json:"stage"`
LaneID string `json:"lane_id,omitempty"`
ModuleKey string `json:"module_key,omitempty"`
ChunkID string `json:"chunk_id,omitempty"`
ChunkIndex int `json:"chunk_index,omitempty"`
ValidatorName string `json:"validator_name,omitempty"`
ReasonCode string `json:"reason_code,omitempty"`
Message string `json:"message"`
AttemptCount int `json:"attempt_count,omitempty"`
DiagnosticArtifactPath string `json:"diagnostic_artifact_path,omitempty"`
}
type ManifestMetadataProvider interface { type ManifestMetadataProvider interface {
ManifestMetadata() map[string]any ManifestMetadata() map[string]any
} }

View File

@@ -19,13 +19,9 @@ var _ Validator = fakeValidator{}
var _ StructuredLLMClient = fakeLLMClient{} var _ StructuredLLMClient = fakeLLMClient{}
var _ OutputEncoder = fakeOutputEncoder{} var _ OutputEncoder = fakeOutputEncoder{}
func TestFakeExtractorReturnsCandidateAndValidator(t *testing.T) { func TestFakeExtractorReturnsRawOutput(t *testing.T) {
validator := fakeValidator{name: "generic-validator"}
extractor := fakeExtractor{ extractor := fakeExtractor{
key: "generic-extractor", key: "generic-extractor",
artifactType: "generic-artifact",
schemaVersion: "v1",
validators: []Validator{validator},
} }
doc := &source.SourceDocument{ doc := &source.SourceDocument{
ID: "source-1", ID: "source-1",
@@ -33,7 +29,7 @@ func TestFakeExtractorReturnsCandidateAndValidator(t *testing.T) {
Format: "text/plain", Format: "text/plain",
Digest: "sha256:abc123", Digest: "sha256:abc123",
Units: []source.SourceUnit{ Units: []source.SourceUnit{
{ID: "u1", Kind: "section", Text: "Source text."}, {ID: 1, Kind: "section", Text: "Source text."},
}, },
} }
@@ -45,37 +41,14 @@ func TestFakeExtractorReturnsCandidateAndValidator(t *testing.T) {
if extractor.Key() != "generic-extractor" { if extractor.Key() != "generic-extractor" {
t.Fatalf("Key() = %q, want generic-extractor", extractor.Key()) t.Fatalf("Key() = %q, want generic-extractor", extractor.Key())
} }
if extractor.ArtifactType() != "generic-artifact" { if result.Output.ExtractorKey != "" {
t.Fatalf("ArtifactType() = %q, want generic-artifact", extractor.ArtifactType()) t.Fatalf("ExtractorKey = %q, want runner-owned empty value", result.Output.ExtractorKey)
} }
if extractor.SchemaVersion() != "v1" { if result.Output.Schema.Version != "v1" {
t.Fatalf("SchemaVersion() = %q, want v1", extractor.SchemaVersion()) t.Fatalf("Schema.Version = %q, want v1", result.Output.Schema.Version)
} }
if len(extractor.Validators()) != 1 { if result.Output.Payload.MediaType != "application/json" || string(result.Output.Payload.Content) != `{"value":"example"}` {
t.Fatalf("len(Validators()) = %d, want 1", len(extractor.Validators())) t.Fatalf("payload = %q %s, want JSON raw output", result.Output.Payload.MediaType, result.Output.Payload.Content)
}
if extractor.Validators()[0].Name() != "generic-validator" {
t.Fatalf("Validators()[0].Name() = %q, want generic-validator", extractor.Validators()[0].Name())
}
if len(result.Candidates) != 1 {
t.Fatalf("len(Candidates) = %d, want 1", len(result.Candidates))
}
candidate := result.Candidates[0]
if candidate.Index != 0 {
t.Fatalf("ArtifactCandidate.Index = %d, want 0", candidate.Index)
}
if candidate.ExtractorKey != extractor.Key() {
t.Fatalf("ArtifactCandidate.ExtractorKey = %q, want %q", candidate.ExtractorKey, extractor.Key())
}
if candidate.ArtifactType != extractor.ArtifactType() {
t.Fatalf("ArtifactCandidate.ArtifactType = %q, want %q", candidate.ArtifactType, extractor.ArtifactType())
}
if candidate.SchemaVersion != extractor.SchemaVersion() {
t.Fatalf("ArtifactCandidate.SchemaVersion = %q, want %q", candidate.SchemaVersion, extractor.SchemaVersion())
}
if string(candidate.Payload) != `{"value":"example"}` {
t.Fatalf("ArtifactCandidate.Payload = %s, want example payload", candidate.Payload)
} }
} }
@@ -86,7 +59,7 @@ func TestFakeChunkerReturnsSourceChunks(t *testing.T) {
Format: "text/plain", Format: "text/plain",
Digest: "sha256:abc123", Digest: "sha256:abc123",
Units: []source.SourceUnit{ Units: []source.SourceUnit{
{ID: "u1", Kind: "section", Text: "Source text."}, {ID: 1, Kind: "section", Text: "Source text."},
}, },
} }
chunker := fakeChunker{key: "generic-chunker"} chunker := fakeChunker{key: "generic-chunker"}
@@ -113,6 +86,12 @@ func TestFakeChunkerReturnsSourceChunks(t *testing.T) {
if chunk.Index != 0 { if chunk.Index != 0 {
t.Fatalf("SourceChunk.Index = %d, want 0", chunk.Index) t.Fatalf("SourceChunk.Index = %d, want 0", chunk.Index)
} }
if chunk.StartUnitID != 1 || chunk.EndUnitID != 1 {
t.Fatalf("SourceChunk boundaries = %d-%d, want 1-1", chunk.StartUnitID, chunk.EndUnitID)
}
if chunk.MediaType != "application/json" || string(chunk.Content) != `{"units":[{"id":1,"kind":"section","text":"Source text."}]}` {
t.Fatalf("SourceChunk payload = %q %s, want JSON units", chunk.MediaType, chunk.Content)
}
if len(chunk.Units) != 1 { if len(chunk.Units) != 1 {
t.Fatalf("len(SourceChunk.Units) = %d, want 1", len(chunk.Units)) t.Fatalf("len(SourceChunk.Units) = %d, want 1", len(chunk.Units))
} }
@@ -125,7 +104,7 @@ func TestFakeChunkerReceivesLLMClient(t *testing.T) {
Format: "text/plain", Format: "text/plain",
Digest: "sha256:abc123", Digest: "sha256:abc123",
Units: []source.SourceUnit{ Units: []source.SourceUnit{
{ID: "u1", Kind: "section", Text: "Source text."}, {ID: 1, Kind: "section", Text: "Source text."},
}, },
} }
client := fakeLLMClient{} client := fakeLLMClient{}
@@ -140,26 +119,26 @@ func TestFakeChunkerReceivesLLMClient(t *testing.T) {
} }
func TestFakeExtractorReceivesChunkAndAmbientContext(t *testing.T) { func TestFakeExtractorReceivesChunkAndAmbientContext(t *testing.T) {
extractor := fakeExtractor{ extractor := fakeExtractor{key: "generic-extractor"}
key: "generic-extractor",
artifactType: "generic-artifact",
schemaVersion: "v1",
}
doc := &source.SourceDocument{ doc := &source.SourceDocument{
ID: "source-1", ID: "source-1",
Kind: "document", Kind: "document",
Format: "text/plain", Format: "text/plain",
Digest: "sha256:abc123", Digest: "sha256:abc123",
Units: []source.SourceUnit{ Units: []source.SourceUnit{
{ID: "u1", Kind: "section", Text: "First source text."}, {ID: 1, Kind: "section", Text: "First source text."},
{ID: "u2", Kind: "section", Text: "Second source text."}, {ID: 2, Kind: "section", Text: "Second source text."},
}, },
} }
chunk := SourceChunk{ chunk := SourceChunk{
ID: "source-1:chunk:1", ID: "source-1:chunk:1",
SourceID: doc.ID, SourceID: doc.ID,
Index: 1, Index: 1,
Units: []source.SourceUnit{doc.Units[1]}, StartUnitID: 2,
EndUnitID: 2,
Content: []byte(`{"units":[{"id":2,"kind":"section","text":"Second source text."}]}`),
MediaType: "application/json",
Units: []source.SourceUnit{doc.Units[1]},
} }
result, err := extractor.Extract(context.Background(), ExtractionRequest{ result, err := extractor.Extract(context.Background(), ExtractionRequest{
@@ -170,20 +149,11 @@ func TestFakeExtractorReceivesChunkAndAmbientContext(t *testing.T) {
if err != nil { if err != nil {
t.Fatalf("Extract() error = %v, want nil", err) t.Fatalf("Extract() error = %v, want nil", err)
} }
if len(result.Candidates) != 1 { if result.Output.ChunkID != "" || result.Output.ChunkIndex != 0 {
t.Fatalf("len(Candidates) = %d, want 1", len(result.Candidates)) t.Fatalf("chunk provenance = %q/%d, want runner-owned zero values", result.Output.ChunkID, result.Output.ChunkIndex)
} }
if string(result.Output.Payload.Content) != `{"value":"chunked"}` {
candidate := result.Candidates[0] t.Fatalf("Payload.Content = %s, want chunked payload", result.Output.Payload.Content)
if string(candidate.Payload) != `{"value":"chunked"}` {
t.Fatalf("ArtifactCandidate.Payload = %s, want chunked payload", candidate.Payload)
}
if len(candidate.SourceRefs) != 1 {
t.Fatalf("len(SourceRefs) = %d, want 1", len(candidate.SourceRefs))
}
ref := candidate.SourceRefs[0]
if ref.StartUnitID != "u2" || ref.EndUnitID != "u2" {
t.Fatalf("SourceRef = %+v, want u2 range", ref)
} }
} }
@@ -353,19 +323,17 @@ func TestLLMInputSetCloneCopiesContent(t *testing.T) {
} }
func TestFakeMergeNormalizeAndOutputContracts(t *testing.T) { func TestFakeMergeNormalizeAndOutputContracts(t *testing.T) {
candidate := artifacts.ArtifactCandidate{ extractOutput := ExtractOutput{
Index: 0, LaneID: "generic-lane",
ExtractorKey: "generic-extractor", ExtractorKey: "generic-extractor",
ArtifactType: "generic-artifact", SourceID: "source-1",
SchemaVersion: "v1", ChunkID: "source-1:chunk:0",
Payload: json.RawMessage(`{"value":"example"}`), ChunkIndex: 0,
} Schema: ResponseSchema{ID: "schema-id", Name: "schema-name", Version: "v1"},
chunk := SourceChunk{ Payload: RawPayload{
ID: "source-1:chunk:0", Content: []byte(`{"value":"example"}`),
SourceID: "source-1", MediaType: "application/json",
Index: 0, Metadata: map[string]any{"confidence": 0.75},
Units: []source.SourceUnit{
{ID: "u1", Kind: "section", Text: "Source text."},
}, },
} }
merger := fakeMerger{key: "generic-merger"} merger := fakeMerger{key: "generic-merger"}
@@ -373,13 +341,8 @@ func TestFakeMergeNormalizeAndOutputContracts(t *testing.T) {
encoder := fakeOutputEncoder{key: "generic-output"} encoder := fakeOutputEncoder{key: "generic-output"}
merged, err := merger.Merge(context.Background(), MergeRequest{ merged, err := merger.Merge(context.Background(), MergeRequest{
LaneID: "generic-artifact", LaneID: "generic-lane",
ChunkArtifacts: []ChunkArtifacts{ ExtractOutputs: []ExtractOutput{extractOutput},
{
Chunk: chunk,
Candidates: []artifacts.ArtifactCandidate{candidate},
},
},
}) })
if err != nil { if err != nil {
t.Fatalf("Merge() error = %v, want nil", err) t.Fatalf("Merge() error = %v, want nil", err)
@@ -387,13 +350,13 @@ func TestFakeMergeNormalizeAndOutputContracts(t *testing.T) {
if merger.Key() != "generic-merger" { if merger.Key() != "generic-merger" {
t.Fatalf("Merger.Key() = %q, want generic-merger", merger.Key()) t.Fatalf("Merger.Key() = %q, want generic-merger", merger.Key())
} }
if len(merged.Candidates) != 1 { if string(merged.Output.Payload.Content) != `{"value":"example"}` {
t.Fatalf("len(merged.Candidates) = %d, want 1", len(merged.Candidates)) t.Fatalf("merged content = %s, want raw extract content", merged.Output.Payload.Content)
} }
normalized, err := normalizer.Normalize(context.Background(), NormalizeRequest{ normalized, err := normalizer.Normalize(context.Background(), NormalizeRequest{
LaneID: "generic-artifact", LaneID: "generic-lane",
Candidates: merged.Candidates, MergeOutput: merged.Output,
}) })
if err != nil { if err != nil {
t.Fatalf("Normalize() error = %v, want nil", err) t.Fatalf("Normalize() error = %v, want nil", err)
@@ -401,15 +364,13 @@ func TestFakeMergeNormalizeAndOutputContracts(t *testing.T) {
if normalizer.Key() != "generic-normalizer" { if normalizer.Key() != "generic-normalizer" {
t.Fatalf("Normalizer.Key() = %q, want generic-normalizer", normalizer.Key()) t.Fatalf("Normalizer.Key() = %q, want generic-normalizer", normalizer.Key())
} }
if len(normalized.Candidates) != 1 { if string(normalized.Output.Payload.Content) != `{"value":"example"}` {
t.Fatalf("len(normalized.Candidates) = %d, want 1", len(normalized.Candidates)) t.Fatalf("normalized content = %s, want raw merge content", normalized.Output.Payload.Content)
} }
encoded, err := encoder.Encode(context.Background(), OutputRequest{ encoded, err := encoder.Encode(context.Background(), OutputRequest{
Manifest: artifacts.RunManifest{RunID: "run-1"}, Manifest: artifacts.RunManifest{RunID: "run-1"},
Approved: []artifacts.Artifact{ NormalizeOutputs: []NormalizeOutput{normalized.Output},
artifacts.ArtifactFromCandidate(normalized.Candidates[0]),
},
}) })
if err != nil { if err != nil {
t.Fatalf("Encode() error = %v, want nil", err) t.Fatalf("Encode() error = %v, want nil", err)
@@ -423,7 +384,7 @@ func TestFakeMergeNormalizeAndOutputContracts(t *testing.T) {
if encoded.Files[0].ContentType != "application/json" { if encoded.Files[0].ContentType != "application/json" {
t.Fatalf("ContentType = %q, want application/json", encoded.Files[0].ContentType) t.Fatalf("ContentType = %q, want application/json", encoded.Files[0].ContentType)
} }
if string(encoded.Files[0].Bytes) != `{"run_id":"run-1","approved_count":1}` { if string(encoded.Files[0].Bytes) != `{"run_id":"run-1","output_count":1}` {
t.Fatalf("Bytes = %s, want encoded output", encoded.Files[0].Bytes) t.Fatalf("Bytes = %s, want encoded output", encoded.Files[0].Bytes)
} }
} }
@@ -487,10 +448,14 @@ func (chunker fakeChunker) Chunk(ctx context.Context, req ChunkRequest) (ChunkRe
return ChunkResult{ return ChunkResult{
Chunks: []SourceChunk{ Chunks: []SourceChunk{
{ {
ID: req.Source.ID + ":chunk:0", ID: req.Source.ID + ":chunk:0",
SourceID: req.Source.ID, SourceID: req.Source.ID,
Index: 0, Index: 0,
Units: append([]source.SourceUnit(nil), req.Source.Units...), StartUnitID: req.Source.Units[0].ID,
EndUnitID: req.Source.Units[len(req.Source.Units)-1].ID,
Content: []byte(`{"units":[{"id":1,"kind":"section","text":"Source text."}]}`),
MediaType: "application/json",
Units: append([]source.SourceUnit(nil), req.Source.Units...),
}, },
}, },
}, nil }, nil
@@ -515,57 +480,29 @@ func (chunker *recordingChunker) Chunk(ctx context.Context, req ChunkRequest) (C
} }
type fakeExtractor struct { type fakeExtractor struct {
key string key string
artifactType string
schemaVersion string
validators []Validator
} }
func (extractor fakeExtractor) Key() string { func (extractor fakeExtractor) Key() string {
return extractor.key return extractor.key
} }
func (extractor fakeExtractor) ArtifactType() string {
return extractor.artifactType
}
func (extractor fakeExtractor) SchemaVersion() string {
return extractor.schemaVersion
}
func (extractor fakeExtractor) ReferenceSlots() []ReferenceSlot { func (extractor fakeExtractor) ReferenceSlots() []ReferenceSlot {
return nil return nil
} }
func (extractor fakeExtractor) Validators() []Validator {
return extractor.validators
}
func (extractor fakeExtractor) Extract(ctx context.Context, req ExtractionRequest) (ExtractionResult, error) { func (extractor fakeExtractor) Extract(ctx context.Context, req ExtractionRequest) (ExtractionResult, error) {
units := req.Source.Units
if req.Chunk != nil {
units = req.Chunk.Units
}
payload := json.RawMessage(`{"value":"example"}`) payload := json.RawMessage(`{"value":"example"}`)
if req.AmbientContext["mode"] == "chunked" { if req.AmbientContext["mode"] == "chunked" {
payload = json.RawMessage(`{"value":"chunked"}`) payload = json.RawMessage(`{"value":"chunked"}`)
} }
return ExtractionResult{ return ExtractionResult{
Candidates: []artifacts.ArtifactCandidate{ Output: ExtractOutput{
{ Schema: ResponseSchema{ID: "schema-id", Name: "schema-name", Version: "v1"},
Index: 0, Payload: RawPayload{
ExtractorKey: extractor.key, Content: append([]byte(nil), payload...),
ArtifactType: extractor.artifactType, MediaType: "application/json",
SchemaVersion: extractor.schemaVersion,
Payload: payload,
SourceRefs: []source.SourceRef{
{
SourceID: req.Source.ID,
StartUnitID: units[0].ID,
EndUnitID: units[len(units)-1].ID,
},
},
}, },
}, },
}, nil }, nil
@@ -580,12 +517,14 @@ func (merger fakeMerger) Key() string {
} }
func (merger fakeMerger) Merge(ctx context.Context, req MergeRequest) (MergeResult, error) { func (merger fakeMerger) Merge(ctx context.Context, req MergeRequest) (MergeResult, error) {
var candidates []artifacts.ArtifactCandidate output := req.ExtractOutputs[0]
for _, chunkArtifacts := range req.ChunkArtifacts { return MergeResult{Output: MergeOutput{
candidates = append(candidates, chunkArtifacts.Candidates...) LaneID: req.LaneID,
} MergerKey: merger.key,
SourceID: output.SourceID,
return MergeResult{Candidates: candidates}, nil Schema: output.Schema,
Payload: cloneTestRawPayload(output.Payload),
}}, nil
} }
type fakeNormalizer struct { type fakeNormalizer struct {
@@ -601,7 +540,33 @@ func (normalizer fakeNormalizer) ReferenceSlots() []ReferenceSlot {
} }
func (normalizer fakeNormalizer) Normalize(ctx context.Context, req NormalizeRequest) (NormalizeResult, error) { func (normalizer fakeNormalizer) Normalize(ctx context.Context, req NormalizeRequest) (NormalizeResult, error) {
return NormalizeResult{Candidates: req.Candidates}, nil return NormalizeResult{Output: NormalizeOutput{
LaneID: req.LaneID,
NormalizerKey: normalizer.key,
SourceID: req.MergeOutput.SourceID,
Schema: req.MergeOutput.Schema,
Payload: cloneTestRawPayload(req.MergeOutput.Payload),
}}, nil
}
func cloneTestRawPayload(payload RawPayload) RawPayload {
return RawPayload{
Content: append([]byte(nil), payload.Content...),
MediaType: payload.MediaType,
Metadata: cloneTestMetadata(payload.Metadata),
Warnings: append([]Warning(nil), payload.Warnings...),
}
}
func cloneTestMetadata(metadata map[string]any) map[string]any {
if len(metadata) == 0 {
return nil
}
out := make(map[string]any, len(metadata))
for key, value := range metadata {
out[key] = value
}
return out
} }
type fakeValidator struct { type fakeValidator struct {
@@ -651,7 +616,7 @@ func (encoder fakeOutputEncoder) Encode(ctx context.Context, req OutputRequest)
{ {
Name: "artifacts/generic.json", Name: "artifacts/generic.json",
ContentType: "application/json", ContentType: "application/json",
Bytes: []byte(`{"run_id":"` + req.Manifest.RunID + `","approved_count":1}`), Bytes: []byte(`{"run_id":"` + req.Manifest.RunID + `","output_count":1}`),
}, },
}, },
}, nil }, nil

View File

@@ -9,8 +9,12 @@ import (
) )
func validateAndCanonicalizeChunkResult(doc *source.SourceDocument, chunks []contracts.SourceChunk) ([]contracts.SourceChunk, error) { func validateAndCanonicalizeChunkResult(doc *source.SourceDocument, chunks []contracts.SourceChunk) ([]contracts.SourceChunk, error) {
sourceUnitIndexes := make(map[string]int, len(doc.Units)) if len(chunks) == 0 {
sourceUnits := make(map[string]source.SourceUnit, len(doc.Units)) return nil, fmt.Errorf("chunks must not be empty")
}
sourceUnitIndexes := make(map[int]int, len(doc.Units))
sourceUnits := make(map[int]source.SourceUnit, len(doc.Units))
for index, unit := range doc.Units { for index, unit := range doc.Units {
sourceUnitIndexes[unit.ID] = index sourceUnitIndexes[unit.ID] = index
sourceUnits[unit.ID] = unit sourceUnits[unit.ID] = unit
@@ -33,25 +37,42 @@ func validateAndCanonicalizeChunkResult(doc *source.SourceDocument, chunks []con
if chunk.Index != chunkIndex { if chunk.Index != chunkIndex {
return nil, fmt.Errorf("chunk %q index %d does not match returned order %d", chunk.ID, chunk.Index, chunkIndex) return nil, fmt.Errorf("chunk %q index %d does not match returned order %d", chunk.ID, chunk.Index, chunkIndex)
} }
startIndex, ok := sourceUnitIndexes[chunk.StartUnitID]
if !ok {
return nil, fmt.Errorf("chunk %q start_unit_id %d was not found in source document %q", chunk.ID, chunk.StartUnitID, doc.ID)
}
endIndex, ok := sourceUnitIndexes[chunk.EndUnitID]
if !ok {
return nil, fmt.Errorf("chunk %q end_unit_id %d was not found in source document %q", chunk.ID, chunk.EndUnitID, doc.ID)
}
if startIndex > endIndex {
return nil, fmt.Errorf("chunk %q start_unit_id %d appears after end_unit_id %d", chunk.ID, chunk.StartUnitID, chunk.EndUnitID)
}
if len(chunk.Units) == 0 { if len(chunk.Units) == 0 {
return nil, fmt.Errorf("chunk %q units must not be empty", chunk.ID) return nil, fmt.Errorf("chunk %q units must not be empty", chunk.ID)
} }
if len(chunk.Content) == 0 {
return nil, fmt.Errorf("chunk %q content must not be empty", chunk.ID)
}
if strings.TrimSpace(chunk.MediaType) == "" {
return nil, fmt.Errorf("chunk %q media_type must not be empty", chunk.ID)
}
seenUnitIDs := make(map[string]struct{}, len(chunk.Units)) seenUnitIDs := make(map[int]struct{}, len(chunk.Units))
previousSourceIndex := -1 previousSourceIndex := -1
canonicalUnits := make([]source.SourceUnit, 0, len(chunk.Units)) canonicalUnits := make([]source.SourceUnit, 0, len(chunk.Units))
for unitIndex, unit := range chunk.Units { for unitIndex, unit := range chunk.Units {
if strings.TrimSpace(unit.ID) == "" { if unit.ID <= 0 {
return nil, fmt.Errorf("chunk %q unit[%d].id must not be empty", chunk.ID, unitIndex) return nil, fmt.Errorf("chunk %q unit[%d].id must be positive", chunk.ID, unitIndex)
} }
if _, ok := seenUnitIDs[unit.ID]; ok { if _, ok := seenUnitIDs[unit.ID]; ok {
return nil, fmt.Errorf("chunk %q repeats source unit %q", chunk.ID, unit.ID) return nil, fmt.Errorf("chunk %q repeats source unit %d", chunk.ID, unit.ID)
} }
seenUnitIDs[unit.ID] = struct{}{} seenUnitIDs[unit.ID] = struct{}{}
sourceIndex, ok := sourceUnitIndexes[unit.ID] sourceIndex, ok := sourceUnitIndexes[unit.ID]
if !ok { if !ok {
return nil, fmt.Errorf("chunk %q source unit %q was not found in source document %q", chunk.ID, unit.ID, doc.ID) return nil, fmt.Errorf("chunk %q source unit %d was not found in source document %q", chunk.ID, unit.ID, doc.ID)
} }
if sourceIndex <= previousSourceIndex { if sourceIndex <= previousSourceIndex {
return nil, fmt.Errorf("chunk %q source units must appear in source document order", chunk.ID) return nil, fmt.Errorf("chunk %q source units must appear in source document order", chunk.ID)
@@ -61,11 +82,15 @@ func validateAndCanonicalizeChunkResult(doc *source.SourceDocument, chunks []con
} }
canonicalChunks = append(canonicalChunks, contracts.SourceChunk{ canonicalChunks = append(canonicalChunks, contracts.SourceChunk{
ID: chunk.ID, ID: chunk.ID,
SourceID: chunk.SourceID, SourceID: chunk.SourceID,
Index: chunk.Index, Index: chunk.Index,
Units: canonicalUnits, StartUnitID: chunk.StartUnitID,
Metadata: cloneMetadata(chunk.Metadata), EndUnitID: chunk.EndUnitID,
Content: append([]byte(nil), chunk.Content...),
MediaType: chunk.MediaType,
Units: canonicalUnits,
Metadata: cloneMetadata(chunk.Metadata),
}) })
} }

View File

@@ -113,14 +113,8 @@ type defaultExtractor struct{}
func (defaultExtractor) Key() string { return "extract" } func (defaultExtractor) Key() string { return "extract" }
func (defaultExtractor) ArtifactType() string { return "record" }
func (defaultExtractor) SchemaVersion() string { return "v1" }
func (defaultExtractor) ReferenceSlots() []contracts.ReferenceSlot { return nil } func (defaultExtractor) ReferenceSlots() []contracts.ReferenceSlot { return nil }
func (defaultExtractor) Validators() []contracts.Validator { return nil }
func (defaultExtractor) Extract(ctx context.Context, req contracts.ExtractionRequest) (contracts.ExtractionResult, error) { func (defaultExtractor) Extract(ctx context.Context, req contracts.ExtractionRequest) (contracts.ExtractionResult, error) {
return contracts.ExtractionResult{}, nil return contracts.ExtractionResult{}, nil
} }

View File

@@ -367,22 +367,10 @@ func (extractor registryFakeExtractor) Key() string {
return extractor.key return extractor.key
} }
func (extractor registryFakeExtractor) ArtifactType() string {
return "generic-artifact"
}
func (extractor registryFakeExtractor) SchemaVersion() string {
return "v1"
}
func (extractor registryFakeExtractor) ReferenceSlots() []contracts.ReferenceSlot { func (extractor registryFakeExtractor) ReferenceSlots() []contracts.ReferenceSlot {
return nil return nil
} }
func (extractor registryFakeExtractor) Validators() []contracts.Validator {
return nil
}
func (extractor registryFakeExtractor) Extract(ctx context.Context, req contracts.ExtractionRequest) (contracts.ExtractionResult, error) { func (extractor registryFakeExtractor) Extract(ctx context.Context, req contracts.ExtractionRequest) (contracts.ExtractionResult, error) {
return contracts.ExtractionResult{}, nil return contracts.ExtractionResult{}, nil
} }

View File

@@ -301,7 +301,7 @@ func (adapter fakeAdapter) Parse(ctx context.Context, req contracts.ParseRequest
Format: "text/plain", Format: "text/plain",
Digest: "sha256:abc123", Digest: "sha256:abc123",
Units: []source.SourceUnit{ Units: []source.SourceUnit{
{ID: "u1", Kind: "unit", Text: "Source unit."}, {ID: 1, Kind: "unit", Text: "Source unit."},
}, },
}, nil }, nil
} }

View File

@@ -97,7 +97,7 @@ func validateModuleSpec(kind string, expectedStage ModuleStage, spec ModuleSpec)
} }
func referenceSlotStage(stage ModuleStage) bool { func referenceSlotStage(stage ModuleStage) bool {
return stage == StageChunk || stage == StageExtract || stage == StageNormalize return stage == StageChunk || stage == StageExtract || stage == StageMerge || stage == StageNormalize
} }
func sortedRegistryKeys[C any](constructors map[string]C) []string { func sortedRegistryKeys[C any](constructors map[string]C) []string {

View File

@@ -15,6 +15,7 @@ func TestValidateModuleSpecAllowsReferenceSlotsForEligibleStages(t *testing.T) {
}{ }{
{name: "chunker", kind: "chunker", stage: StageChunk}, {name: "chunker", kind: "chunker", stage: StageChunk},
{name: "extractor", kind: "extractor", stage: StageExtract}, {name: "extractor", kind: "extractor", stage: StageExtract},
{name: "merger", kind: "merger", stage: StageMerge},
{name: "normalizer", kind: "normalizer", stage: StageNormalize}, {name: "normalizer", kind: "normalizer", stage: StageNormalize},
} }
@@ -42,7 +43,6 @@ func TestValidateModuleSpecRejectsReferenceSlotsForIneligibleStages(t *testing.T
stage ModuleStage stage ModuleStage
}{ }{
{name: "input", kind: "input adapter", stage: StageInput}, {name: "input", kind: "input adapter", stage: StageInput},
{name: "merge", kind: "merger", stage: StageMerge},
{name: "validate", kind: "validator", stage: StageValidate}, {name: "validate", kind: "validator", stage: StageValidate},
{name: "output", kind: "output encoder", stage: StageOutput}, {name: "output", kind: "output encoder", stage: StageOutput},
} }
@@ -99,6 +99,7 @@ func TestValidateModuleSpecRejectsInvalidReferenceSlotsForEligibleStages(t *test
}{ }{
{name: "chunk", kind: "chunker", stage: StageChunk}, {name: "chunk", kind: "chunker", stage: StageChunk},
{name: "extract", kind: "extractor", stage: StageExtract}, {name: "extract", kind: "extractor", stage: StageExtract},
{name: "merge", kind: "merger", stage: StageMerge},
{name: "normalize", kind: "normalizer", stage: StageNormalize}, {name: "normalize", kind: "normalizer", stage: StageNormalize},
} }

View File

@@ -22,6 +22,7 @@ const (
type ModuleBinding struct { type ModuleBinding struct {
Module string `json:"module"` Module string `json:"module"`
LLMProfile string `json:"llm_profile,omitempty"` LLMProfile string `json:"llm_profile,omitempty"`
Retries int `json:"retries,omitempty"`
Options map[string]any `json:"options,omitempty"` Options map[string]any `json:"options,omitempty"`
References map[string]string `json:"references,omitempty"` References map[string]string `json:"references,omitempty"`
} }
@@ -78,6 +79,7 @@ type ResolvedArtifactLane struct {
Normalize ModuleBinding Normalize ModuleBinding
Validators []ModuleBinding Validators []ModuleBinding
ExtractReferences ResolvedReferenceTarget `json:"extract_references"` ExtractReferences ResolvedReferenceTarget `json:"extract_references"`
MergeReferences ResolvedReferenceTarget `json:"merge_references"`
NormalizeReferences ResolvedReferenceTarget `json:"normalize_references"` NormalizeReferences ResolvedReferenceTarget `json:"normalize_references"`
} }
@@ -251,6 +253,20 @@ func resolveArtifactLane(
if missing, ok := capabilities.missing(mergeSpec.Requires); ok { if missing, ok := capabilities.missing(mergeSpec.Requires); ok {
return ResolvedArtifactLane{}, nil, capabilityError(pipelineID, laneID, StageMerge, lane.Merge.Module, missing) return ResolvedArtifactLane{}, nil, capabilityError(pipelineID, laneID, StageMerge, lane.Merge.Module, missing)
} }
mergeReferences, err := resolveReferenceTargetBindings(referenceResolutionTarget{
PipelineID: pipelineID,
LaneID: laneID,
Stage: StageMerge,
Module: lane.Merge.Module,
Slots: mergeSpec.ReferenceSlots,
PipelineReferences: pipelineReferences,
LocalReferences: lane.Merge.References,
Options: options,
})
if err != nil {
return ResolvedArtifactLane{}, nil, err
}
lane.MergeReferences = referenceTarget(StageMerge, laneID, lane.Merge.Module, mergeReferences)
capabilities.add(mergeSpec.Provides...) capabilities.add(mergeSpec.Provides...)
normalizeSpec, err := normalizerSpec(catalog, lane.Normalize.Module) normalizeSpec, err := normalizerSpec(catalog, lane.Normalize.Module)
@@ -346,6 +362,15 @@ func validatePipelineReferenceDefaults(
declaredByAnyTarget[slot.Name] = struct{}{} declaredByAnyTarget[slot.Name] = struct{}{}
} }
merge := resolveBinding(laneProfile.Merge, DefaultMergeModule)
mergeSpec, err := mergerSpec(catalog, merge.Module)
if err != nil {
return moduleLookupError(pipelineID, laneID, StageMerge, merge.Module, err)
}
for _, slot := range mergeSpec.ReferenceSlots {
declaredByAnyTarget[slot.Name] = struct{}{}
}
normalize := resolveBinding(laneProfile.Normalize, DefaultNormalizeModule) normalize := resolveBinding(laneProfile.Normalize, DefaultNormalizeModule)
normalizeSpec, err := normalizerSpec(catalog, normalize.Module) normalizeSpec, err := normalizerSpec(catalog, normalize.Module)
if err != nil { if err != nil {
@@ -500,7 +525,7 @@ func normalizeReferenceOptionTarget(pipelineID string, operation string, stage M
if laneID != "" { if laneID != "" {
return "", "", fmt.Errorf("pipeline %q reference %s for chunk must not include a lane id", pipelineID, operation) return "", "", fmt.Errorf("pipeline %q reference %s for chunk must not include a lane id", pipelineID, operation)
} }
case StageExtract, StageNormalize: case StageExtract, StageMerge, StageNormalize:
if laneID == "" { if laneID == "" {
return "", "", fmt.Errorf("pipeline %q reference %s lane id must not be empty", pipelineID, operation) return "", "", fmt.Errorf("pipeline %q reference %s lane id must not be empty", pipelineID, operation)
} }
@@ -600,6 +625,7 @@ func resolveBinding(binding ModuleBinding, defaultModule string) ModuleBinding {
return ModuleBinding{ return ModuleBinding{
Module: module, Module: module,
LLMProfile: llmProfile, LLMProfile: llmProfile,
Retries: binding.Retries,
Options: cloneOptions(binding.Options), Options: cloneOptions(binding.Options),
References: normalizeReferenceMap(binding.References), References: normalizeReferenceMap(binding.References),
} }

View File

@@ -160,6 +160,9 @@ func TestResolvePipelineAppliesReferenceBindings(t *testing.T) {
if events.NormalizeReferences.Stage != StageNormalize || events.NormalizeReferences.LaneID != "events" || events.NormalizeReferences.Module != DefaultNormalizeModule { if events.NormalizeReferences.Stage != StageNormalize || events.NormalizeReferences.LaneID != "events" || events.NormalizeReferences.Module != DefaultNormalizeModule {
t.Fatalf("normalize reference target = %#v, want event normalizer target", events.NormalizeReferences) t.Fatalf("normalize reference target = %#v, want event normalizer target", events.NormalizeReferences)
} }
if events.MergeReferences.Stage != StageMerge || events.MergeReferences.LaneID != "events" || events.MergeReferences.Module != DefaultMergeModule {
t.Fatalf("merge reference target = %#v, want event merger target", events.MergeReferences)
}
if resolved.ChunkReferences.Stage != StageChunk || resolved.ChunkReferences.Module != DefaultChunkModule { if resolved.ChunkReferences.Stage != StageChunk || resolved.ChunkReferences.Module != DefaultChunkModule {
t.Fatalf("chunk reference target = %#v, want chunk target", resolved.ChunkReferences) t.Fatalf("chunk reference target = %#v, want chunk target", resolved.ChunkReferences)
} }
@@ -260,12 +263,44 @@ func TestResolvePipelineAppliesPipelineReferenceDefaultToNormalizerTarget(t *tes
} }
} }
func TestResolvePipelineAppliesPipelineReferenceDefaultToMergeTarget(t *testing.T) {
profile := baselineProfile()
profile.References = map[string]string{"merge_notes": "./merge.md"}
catalog := newProfileCatalogWithOverrides(t, ModuleSpec{
Key: "appendorder",
Stage: StageMerge,
Requires: []string{"candidate"},
Provides: []string{"merged"},
ReferenceSlots: []contracts.ReferenceSlot{{Name: "merge_notes"}},
})
resolved, err := ResolvePipeline(profile, ResolveOptions{}, catalog)
if err != nil {
t.Fatalf("ResolvePipeline() error = %v, want nil", err)
}
want := []ReferenceBinding{{LaneID: "events", SlotName: "merge_notes", Source: "./merge.md", BindingSource: contracts.ReferenceBindingSourceConfig}}
if !reflect.DeepEqual(resolved.ArtifactLanes[0].MergeReferences.Bindings, want) {
t.Fatalf("merge references = %#v, want %#v", resolved.ArtifactLanes[0].MergeReferences.Bindings, want)
}
if refs := resolved.ChunkReferences.Bindings; len(refs) != 0 {
t.Fatalf("chunk references = %#v, want none", refs)
}
if refs := resolved.ArtifactLanes[0].ExtractReferences.Bindings; len(refs) != 0 {
t.Fatalf("extract references = %#v, want none", refs)
}
if refs := resolved.ArtifactLanes[0].NormalizeReferences.Bindings; len(refs) != 0 {
t.Fatalf("normalize references = %#v, want none", refs)
}
}
func TestResolvePipelineAppliesOnePipelineReferenceDefaultToMultipleTargets(t *testing.T) { func TestResolvePipelineAppliesOnePipelineReferenceDefaultToMultipleTargets(t *testing.T) {
profile := baselineProfile() profile := baselineProfile()
profile.References = map[string]string{"context": "./context.md"} profile.References = map[string]string{"context": "./context.md"}
catalog := newProfileCatalogWithOverrides(t, catalog := newProfileCatalogWithOverrides(t,
ModuleSpec{Key: "generic", Stage: StageChunk, Requires: []string{"source"}, Provides: []string{"chunk"}, ReferenceSlots: []contracts.ReferenceSlot{{Name: "context"}}}, ModuleSpec{Key: "generic", Stage: StageChunk, Requires: []string{"source"}, Provides: []string{"chunk"}, ReferenceSlots: []contracts.ReferenceSlot{{Name: "context"}}},
ModuleSpec{Key: "event-extractor", Stage: StageExtract, Requires: []string{"chunk"}, Provides: []string{"candidate"}, ReferenceSlots: []contracts.ReferenceSlot{{Name: "context"}}}, ModuleSpec{Key: "event-extractor", Stage: StageExtract, Requires: []string{"chunk"}, Provides: []string{"candidate"}, ReferenceSlots: []contracts.ReferenceSlot{{Name: "context"}}},
ModuleSpec{Key: "appendorder", Stage: StageMerge, Requires: []string{"candidate"}, Provides: []string{"merged"}, ReferenceSlots: []contracts.ReferenceSlot{{Name: "context"}}},
ModuleSpec{Key: "noop", Stage: StageNormalize, Requires: []string{"merged"}, Provides: []string{"normalized"}, ReferenceSlots: []contracts.ReferenceSlot{{Name: "context"}}}, ModuleSpec{Key: "noop", Stage: StageNormalize, Requires: []string{"merged"}, Provides: []string{"normalized"}, ReferenceSlots: []contracts.ReferenceSlot{{Name: "context"}}},
) )
@@ -276,6 +311,7 @@ func TestResolvePipelineAppliesOnePipelineReferenceDefaultToMultipleTargets(t *t
assertBindingSource(t, resolved.ChunkReferences.Bindings, "context", "./context.md") assertBindingSource(t, resolved.ChunkReferences.Bindings, "context", "./context.md")
assertBindingSource(t, resolved.ArtifactLanes[0].ExtractReferences.Bindings, "context", "./context.md") assertBindingSource(t, resolved.ArtifactLanes[0].ExtractReferences.Bindings, "context", "./context.md")
assertBindingSource(t, resolved.ArtifactLanes[0].MergeReferences.Bindings, "context", "./context.md")
assertBindingSource(t, resolved.ArtifactLanes[0].NormalizeReferences.Bindings, "context", "./context.md") assertBindingSource(t, resolved.ArtifactLanes[0].NormalizeReferences.Bindings, "context", "./context.md")
} }
@@ -373,6 +409,28 @@ func TestResolvePipelineRejectsExtractLocalReferenceDeclaredOnlyByNormalizer(t *
assertErrorContains(t, err, "baseline", "events", "extract", "event-extractor", "normalization_notes", "not declared") assertErrorContains(t, err, "baseline", "events", "extract", "event-extractor", "normalization_notes", "not declared")
} }
func TestResolvePipelineRejectsMergeLocalReferenceDeclaredOnlyByNormalizer(t *testing.T) {
profile := baselineProfile()
lane := profile.Artifacts["events"]
lane.Merge.References = map[string]string{"normalization_notes": "./normalize.md"}
profile.Artifacts["events"] = lane
catalog := newProfileCatalogWithOverrides(t, ModuleSpec{
Key: "noop",
Stage: StageNormalize,
Requires: []string{"merged"},
Provides: []string{"normalized"},
ReferenceSlots: []contracts.ReferenceSlot{
{Name: "normalization_notes"},
},
})
_, err := ResolvePipeline(profile, ResolveOptions{}, catalog)
if err == nil {
t.Fatal("ResolvePipeline() error = nil, want error")
}
assertErrorContains(t, err, "merge", "normalization_notes", "not declared")
}
func TestResolvePipelineRejectsNormalizeLocalReferenceDeclaredOnlyByExtractor(t *testing.T) { func TestResolvePipelineRejectsNormalizeLocalReferenceDeclaredOnlyByExtractor(t *testing.T) {
profile := baselineProfile() profile := baselineProfile()
lane := profile.Artifacts["events"] lane := profile.Artifacts["events"]

View File

@@ -0,0 +1,73 @@
package pipeline
import (
"fmt"
"strings"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
)
type rawValidationKey struct {
stage ModuleStage
module string
}
type RawValidationRegistry struct {
chains map[rawValidationKey][]contracts.RawValidator
}
func NewRawValidationRegistry() *RawValidationRegistry {
return &RawValidationRegistry{
chains: make(map[rawValidationKey][]contracts.RawValidator),
}
}
func (r *RawValidationRegistry) Register(stage ModuleStage, module string, validators ...contracts.RawValidator) error {
if r == nil {
return fmt.Errorf("raw validation registry must not be nil")
}
normalizedModule := strings.TrimSpace(module)
if normalizedModule == "" {
return fmt.Errorf("raw validation module key must not be empty")
}
switch stage {
case StageChunk, StageExtract, StageMerge, StageNormalize:
default:
return fmt.Errorf("raw validation stage %q is not supported", stage)
}
if len(validators) == 0 {
return fmt.Errorf("raw validation chain for %q %q must not be empty", stage, normalizedModule)
}
chain := make([]contracts.RawValidator, 0, len(validators))
for i, validator := range validators {
if validator == nil {
return fmt.Errorf("raw validator %d for %q %q must not be nil", i, stage, normalizedModule)
}
if strings.TrimSpace(validator.Name()) == "" {
return fmt.Errorf("raw validator %d for %q %q must not have an empty name", i, stage, normalizedModule)
}
chain = append(chain, validator)
}
if r.chains == nil {
r.chains = make(map[rawValidationKey][]contracts.RawValidator)
}
key := rawValidationKey{stage: stage, module: normalizedModule}
if _, exists := r.chains[key]; exists {
return fmt.Errorf("raw validation chain for %q %q is already registered", stage, normalizedModule)
}
r.chains[key] = append([]contracts.RawValidator(nil), chain...)
return nil
}
func (r *RawValidationRegistry) Validators(stage ModuleStage, module string) []contracts.RawValidator {
if r == nil {
return nil
}
chain := r.chains[rawValidationKey{stage: stage, module: strings.TrimSpace(module)}]
if len(chain) == 0 {
return nil
}
return append([]contracts.RawValidator(nil), chain...)
}

View File

@@ -44,6 +44,7 @@ func MaterializeReferences(resolved ResolvedPipeline, catalog ModuleCatalog, opt
for i, lane := range resolved.ArtifactLanes { for i, lane := range resolved.ArtifactLanes {
materializedLane := lane materializedLane := lane
materializedLane.ExtractReferences = CloneReferenceTarget(lane.ExtractReferences) materializedLane.ExtractReferences = CloneReferenceTarget(lane.ExtractReferences)
materializedLane.MergeReferences = CloneReferenceTarget(lane.MergeReferences)
materializedLane.NormalizeReferences = CloneReferenceTarget(lane.NormalizeReferences) materializedLane.NormalizeReferences = CloneReferenceTarget(lane.NormalizeReferences)
extractReferenceSet, laneWarnings, err := materializeReferenceTarget(resolved.ID, lane.ExtractReferences, catalog, options) extractReferenceSet, laneWarnings, err := materializeReferenceTarget(resolved.ID, lane.ExtractReferences, catalog, options)
if err != nil { if err != nil {
@@ -52,6 +53,13 @@ func MaterializeReferences(resolved ResolvedPipeline, catalog ModuleCatalog, opt
materializedLane.ExtractReferences.ReferenceSet = extractReferenceSet materializedLane.ExtractReferences.ReferenceSet = extractReferenceSet
warnings = append(warnings, laneWarnings...) warnings = append(warnings, laneWarnings...)
mergeReferenceSet, laneWarnings, err := materializeReferenceTarget(resolved.ID, lane.MergeReferences, catalog, options)
if err != nil {
return ResolvedPipeline{}, nil, err
}
materializedLane.MergeReferences.ReferenceSet = mergeReferenceSet
warnings = append(warnings, laneWarnings...)
normalizeReferenceSet, laneWarnings, err := materializeReferenceTarget(resolved.ID, lane.NormalizeReferences, catalog, options) normalizeReferenceSet, laneWarnings, err := materializeReferenceTarget(resolved.ID, lane.NormalizeReferences, catalog, options)
if err != nil { if err != nil {
return ResolvedPipeline{}, nil, err return ResolvedPipeline{}, nil, err
@@ -140,6 +148,8 @@ func referenceTargetSpec(target ResolvedReferenceTarget, catalog ModuleCatalog)
return registrySpec(catalog.Chunkers, target.Module) return registrySpec(catalog.Chunkers, target.Module)
case StageExtract: case StageExtract:
return registrySpec(catalog.Extractors, target.Module) return registrySpec(catalog.Extractors, target.Module)
case StageMerge:
return registrySpec(catalog.Mergers, target.Module)
case StageNormalize: case StageNormalize:
return registrySpec(catalog.Normalizers, target.Module) return registrySpec(catalog.Normalizers, target.Module)
default: default:
@@ -291,6 +301,7 @@ func ReferenceProvenance(resolved ResolvedPipeline) []artifacts.ReferenceProvena
provenance = append(provenance, referenceTargetProvenance(resolved.ChunkReferences)...) provenance = append(provenance, referenceTargetProvenance(resolved.ChunkReferences)...)
for _, lane := range resolved.ArtifactLanes { for _, lane := range resolved.ArtifactLanes {
provenance = append(provenance, referenceTargetProvenance(lane.ExtractReferences)...) provenance = append(provenance, referenceTargetProvenance(lane.ExtractReferences)...)
provenance = append(provenance, referenceTargetProvenance(lane.MergeReferences)...)
provenance = append(provenance, referenceTargetProvenance(lane.NormalizeReferences)...) provenance = append(provenance, referenceTargetProvenance(lane.NormalizeReferences)...)
} }
return provenance return provenance

View File

@@ -109,17 +109,20 @@ func TestMaterializeReferencesStoresSetsAndProvenanceForAllTargets(t *testing.T)
configDir := t.TempDir() configDir := t.TempDir()
writeReferenceFile(t, filepath.Join(configDir, "chunk.txt"), []byte("chunk text")) writeReferenceFile(t, filepath.Join(configDir, "chunk.txt"), []byte("chunk text"))
writeReferenceFile(t, filepath.Join(configDir, "extract.txt"), []byte("extract text")) writeReferenceFile(t, filepath.Join(configDir, "extract.txt"), []byte("extract text"))
writeReferenceFile(t, filepath.Join(configDir, "merge.txt"), []byte("merge text"))
writeReferenceFile(t, filepath.Join(configDir, "normalize.txt"), []byte("normalize text")) writeReferenceFile(t, filepath.Join(configDir, "normalize.txt"), []byte("normalize text"))
profile := baselineProfile() profile := baselineProfile()
profile.References = map[string]string{ profile.References = map[string]string{
"scene_guide": "chunk.txt", "scene_guide": "chunk.txt",
"roster": "extract.txt", "roster": "extract.txt",
"merge_notes": "merge.txt",
"normalization_notes": "normalize.txt", "normalization_notes": "normalize.txt",
} }
catalog := referenceCatalogForTargets(t, catalog := referenceCatalogForTargets(t,
[]contracts.ReferenceSlot{{Name: "scene_guide"}}, []contracts.ReferenceSlot{{Name: "scene_guide"}},
[]contracts.ReferenceSlot{{Name: "roster"}}, []contracts.ReferenceSlot{{Name: "roster"}},
[]contracts.ReferenceSlot{{Name: "merge_notes"}},
[]contracts.ReferenceSlot{{Name: "normalization_notes"}}, []contracts.ReferenceSlot{{Name: "normalization_notes"}},
) )
resolved, err := ResolvePipeline(profile, ResolveOptions{}, catalog) resolved, err := ResolvePipeline(profile, ResolveOptions{}, catalog)
@@ -145,14 +148,18 @@ func TestMaterializeReferencesStoresSetsAndProvenanceForAllTargets(t *testing.T)
if string(extractItem.Content) != "extract text" { if string(extractItem.Content) != "extract text" {
t.Fatalf("extract content = %q, want extract text", extractItem.Content) t.Fatalf("extract content = %q, want extract text", extractItem.Content)
} }
mergeItem := materialized.ArtifactLanes[0].MergeReferences.ReferenceSet.Slots["merge_notes"].Items[0]
if string(mergeItem.Content) != "merge text" {
t.Fatalf("merge content = %q, want merge text", mergeItem.Content)
}
normalizeItem := materialized.ArtifactLanes[0].NormalizeReferences.ReferenceSet.Slots["normalization_notes"].Items[0] normalizeItem := materialized.ArtifactLanes[0].NormalizeReferences.ReferenceSet.Slots["normalization_notes"].Items[0]
if string(normalizeItem.Content) != "normalize text" { if string(normalizeItem.Content) != "normalize text" {
t.Fatalf("normalize content = %q, want normalize text", normalizeItem.Content) t.Fatalf("normalize content = %q, want normalize text", normalizeItem.Content)
} }
provenance := ReferenceProvenance(materialized) provenance := ReferenceProvenance(materialized)
if len(provenance) != 3 { if len(provenance) != 4 {
t.Fatalf("ReferenceProvenance() = %#v, want three entries", provenance) t.Fatalf("ReferenceProvenance() = %#v, want four entries", provenance)
} }
if provenance[0].Stage != string(StageChunk) || provenance[0].LaneID != "" || provenance[0].SlotName != "scene_guide" || provenance[0].Digest != chunkItem.Digest { if provenance[0].Stage != string(StageChunk) || provenance[0].LaneID != "" || provenance[0].SlotName != "scene_guide" || provenance[0].Digest != chunkItem.Digest {
t.Fatalf("ReferenceProvenance()[0] = %#v, want chunk scene guide provenance", provenance[0]) t.Fatalf("ReferenceProvenance()[0] = %#v, want chunk scene guide provenance", provenance[0])
@@ -160,7 +167,10 @@ func TestMaterializeReferencesStoresSetsAndProvenanceForAllTargets(t *testing.T)
if provenance[1].Stage != string(StageExtract) || provenance[1].LaneID != "events" || provenance[1].SlotName != "roster" || provenance[1].Digest != extractItem.Digest { if provenance[1].Stage != string(StageExtract) || provenance[1].LaneID != "events" || provenance[1].SlotName != "roster" || provenance[1].Digest != extractItem.Digest {
t.Fatalf("ReferenceProvenance()[1] = %#v, want extract roster provenance", provenance[1]) t.Fatalf("ReferenceProvenance()[1] = %#v, want extract roster provenance", provenance[1])
} }
if provenance[2].Stage != string(StageNormalize) || provenance[2].LaneID != "events" || provenance[2].SlotName != "normalization_notes" || provenance[2].Digest != normalizeItem.Digest { if provenance[2].Stage != string(StageMerge) || provenance[2].LaneID != "events" || provenance[2].SlotName != "merge_notes" || provenance[2].Digest != mergeItem.Digest {
t.Fatalf("ReferenceProvenance()[2] = %#v, want merge notes provenance", provenance[2])
}
if provenance[3].Stage != string(StageNormalize) || provenance[3].LaneID != "events" || provenance[3].SlotName != "normalization_notes" || provenance[3].Digest != normalizeItem.Digest {
t.Fatalf("ReferenceProvenance()[2] = %#v, want normalize notes provenance", provenance[2]) t.Fatalf("ReferenceProvenance()[2] = %#v, want normalize notes provenance", provenance[2])
} }
} }
@@ -185,7 +195,7 @@ func TestMaterializeReferencesRejectsNonUTF8ContentForChunkTarget(t *testing.T)
writeReferenceFile(t, path, []byte{0xff, 0xfe}) writeReferenceFile(t, path, []byte{0xff, 0xfe})
resolved := resolvedPipelineWithTargetReference(t, StageChunk, "", "scene_guide", "bad.txt", contracts.ReferenceBindingSourceConfig, contracts.ReferenceSlot{Name: "scene_guide"}) resolved := resolvedPipelineWithTargetReference(t, StageChunk, "", "scene_guide", "bad.txt", contracts.ReferenceBindingSourceConfig, contracts.ReferenceSlot{Name: "scene_guide"})
_, _, err := MaterializeReferences(resolved, referenceCatalogForTargets(t, []contracts.ReferenceSlot{{Name: "scene_guide"}}, nil, nil), ReferenceMaterializationOptions{ _, _, err := MaterializeReferences(resolved, referenceCatalogForTargets(t, []contracts.ReferenceSlot{{Name: "scene_guide"}}, nil, nil, nil), ReferenceMaterializationOptions{
ConfigPath: filepath.Join(configDir, "config.yml"), ConfigPath: filepath.Join(configDir, "config.yml"),
}) })
if err == nil || !strings.Contains(err.Error(), "chunk") || !strings.Contains(err.Error(), "UTF-8") || !strings.Contains(err.Error(), "scene_guide") || !strings.Contains(err.Error(), path) { if err == nil || !strings.Contains(err.Error(), "chunk") || !strings.Contains(err.Error(), "UTF-8") || !strings.Contains(err.Error(), "scene_guide") || !strings.Contains(err.Error(), path) {
@@ -309,7 +319,7 @@ func TestMaterializeReferencesRejectsUnacceptedMediaTypeForNormalizeTarget(t *te
slot := contracts.ReferenceSlot{Name: "normalization_notes", AcceptedMediaTypes: []string{"text/markdown"}} slot := contracts.ReferenceSlot{Name: "normalization_notes", AcceptedMediaTypes: []string{"text/markdown"}}
resolved := resolvedPipelineWithTargetReference(t, StageNormalize, "events", "normalization_notes", "notes.json", contracts.ReferenceBindingSourceConfig, slot) resolved := resolvedPipelineWithTargetReference(t, StageNormalize, "events", "normalization_notes", "notes.json", contracts.ReferenceBindingSourceConfig, slot)
_, _, err := MaterializeReferences(resolved, referenceCatalogForTargets(t, nil, nil, []contracts.ReferenceSlot{slot}), ReferenceMaterializationOptions{ _, _, err := MaterializeReferences(resolved, referenceCatalogForTargets(t, nil, nil, nil, []contracts.ReferenceSlot{slot}), ReferenceMaterializationOptions{
ConfigPath: filepath.Join(configDir, "config.yml"), ConfigPath: filepath.Join(configDir, "config.yml"),
}) })
if err == nil || !strings.Contains(err.Error(), "normalize") || !strings.Contains(err.Error(), "media type") || !strings.Contains(err.Error(), "application/json") || !strings.Contains(err.Error(), "normalization_notes") { if err == nil || !strings.Contains(err.Error(), "normalize") || !strings.Contains(err.Error(), "media type") || !strings.Contains(err.Error(), "application/json") || !strings.Contains(err.Error(), "normalization_notes") {
@@ -353,6 +363,7 @@ func TestMaterializeReferencesWarningScopesIncludeTargetContext(t *testing.T) {
catalog := referenceCatalogForTargets(t, catalog := referenceCatalogForTargets(t,
[]contracts.ReferenceSlot{{Name: "scene_guide"}}, []contracts.ReferenceSlot{{Name: "scene_guide"}},
[]contracts.ReferenceSlot{{Name: "roster"}}, []contracts.ReferenceSlot{{Name: "roster"}},
nil,
[]contracts.ReferenceSlot{{Name: "normalization_notes"}}, []contracts.ReferenceSlot{{Name: "normalization_notes"}},
) )
resolved, err := ResolvePipeline(profile, ResolveOptions{}, catalog) resolved, err := ResolvePipeline(profile, ResolveOptions{}, catalog)
@@ -407,6 +418,10 @@ func resolvedPipelineWithTargetReference(t *testing.T, stage ModuleStage, laneID
lane := profile.Artifacts[laneID] lane := profile.Artifacts[laneID]
lane.References = map[string]string{slotName: source} lane.References = map[string]string{slotName: source}
profile.Artifacts[laneID] = lane profile.Artifacts[laneID] = lane
case StageMerge:
lane := profile.Artifacts[laneID]
lane.Merge.References = map[string]string{slotName: source}
profile.Artifacts[laneID] = lane
case StageNormalize: case StageNormalize:
lane := profile.Artifacts[laneID] lane := profile.Artifacts[laneID]
lane.Normalize.References = map[string]string{slotName: source} lane.Normalize.References = map[string]string{slotName: source}
@@ -425,6 +440,8 @@ func resolvedPipelineWithTargetReference(t *testing.T, stage ModuleStage, laneID
resolved.ChunkReferences.Bindings[0].BindingSource = bindingSource resolved.ChunkReferences.Bindings[0].BindingSource = bindingSource
case StageExtract: case StageExtract:
resolved.ArtifactLanes[0].ExtractReferences.Bindings[0].BindingSource = bindingSource resolved.ArtifactLanes[0].ExtractReferences.Bindings[0].BindingSource = bindingSource
case StageMerge:
resolved.ArtifactLanes[0].MergeReferences.Bindings[0].BindingSource = bindingSource
case StageNormalize: case StageNormalize:
resolved.ArtifactLanes[0].NormalizeReferences.Bindings[0].BindingSource = bindingSource resolved.ArtifactLanes[0].NormalizeReferences.Bindings[0].BindingSource = bindingSource
} }
@@ -441,18 +458,20 @@ func referenceCatalogForStage(t *testing.T, stage ModuleStage, slots []contracts
t.Helper() t.Helper()
switch stage { switch stage {
case StageChunk: case StageChunk:
return referenceCatalogForTargets(t, slots, nil, nil) return referenceCatalogForTargets(t, slots, nil, nil, nil)
case StageExtract: case StageExtract:
return referenceCatalogForTargets(t, nil, slots, nil) return referenceCatalogForTargets(t, nil, slots, nil, nil)
case StageMerge:
return referenceCatalogForTargets(t, nil, nil, slots, nil)
case StageNormalize: case StageNormalize:
return referenceCatalogForTargets(t, nil, nil, slots) return referenceCatalogForTargets(t, nil, nil, nil, slots)
default: default:
t.Fatalf("unsupported reference target stage %q", stage) t.Fatalf("unsupported reference target stage %q", stage)
return ModuleCatalog{} return ModuleCatalog{}
} }
} }
func referenceCatalogForTargets(t *testing.T, chunkSlots, extractSlots, normalizeSlots []contracts.ReferenceSlot) ModuleCatalog { func referenceCatalogForTargets(t *testing.T, chunkSlots, extractSlots, mergeSlots, normalizeSlots []contracts.ReferenceSlot) ModuleCatalog {
t.Helper() t.Helper()
return newProfileCatalogWithOverrides(t, return newProfileCatalogWithOverrides(t,
ModuleSpec{ ModuleSpec{
@@ -469,6 +488,13 @@ func referenceCatalogForTargets(t *testing.T, chunkSlots, extractSlots, normaliz
Provides: []string{"candidate"}, Provides: []string{"candidate"},
ReferenceSlots: extractSlots, ReferenceSlots: extractSlots,
}, },
ModuleSpec{
Key: "appendorder",
Stage: StageMerge,
Requires: []string{"candidate"},
Provides: []string{"merged"},
ReferenceSlots: mergeSlots,
},
ModuleSpec{ ModuleSpec{
Key: "noop", Key: "noop",
Stage: StageNormalize, Stage: StageNormalize,

View File

@@ -5,11 +5,8 @@ import (
"reflect" "reflect"
"testing" "testing"
"gitea.maximumdirect.net/eric/notarius/internal/core/artifacts"
"gitea.maximumdirect.net/eric/notarius/internal/core/source" "gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts" "gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
validate "gitea.maximumdirect.net/eric/notarius/internal/framework/validate"
) )
func TestRunnerUsesRegistries(t *testing.T) { func TestRunnerUsesRegistries(t *testing.T) {
@@ -33,11 +30,11 @@ func TestRunnerUsesRegistries(t *testing.T) {
if !reflect.DeepEqual(executed, []string{"extract-first:chunk-0", "extract-second:chunk-0"}) { if !reflect.DeepEqual(executed, []string{"extract-first:chunk-0", "extract-second:chunk-0"}) {
t.Fatalf("executed = %#v, want extractor chunk execution", executed) t.Fatalf("executed = %#v, want extractor chunk execution", executed)
} }
if got := artifactKeys(output.Approved); !reflect.DeepEqual(got, []string{"extract-first"}) { if got := normalizeOutputKeys(output.NormalizeOutputs); !reflect.DeepEqual(got, []string{"normalize", "normalize"}) {
t.Fatalf("approved keys = %#v, want [extract-first]", got) t.Fatalf("normalize output keys = %#v, want one output from each lane", got)
} }
if got := rejectedKeys(output.Rejected); !reflect.DeepEqual(got, []string{"extract-second"}) { if len(output.Rejected) != 0 {
t.Fatalf("rejected keys = %#v, want [extract-second]", got) t.Fatalf("len(Rejected) = %d, want none", len(output.Rejected))
} }
} }
@@ -64,12 +61,8 @@ func integrationRegistries(t *testing.T, built, executed *[]string) Registries {
}); err != nil { }); err != nil {
t.Fatalf("register chunker: %v", err) t.Fatalf("register chunker: %v", err)
} }
registerIntegrationExtractor(t, registries.Extractors, "extract-first", built, executed, []contracts.Validator{ registerIntegrationExtractor(t, registries.Extractors, "extract-first", built, executed)
integrationValidator{name: "approve-first", approve: true}, registerIntegrationExtractor(t, registries.Extractors, "extract-second", built, executed)
})
registerIntegrationExtractor(t, registries.Extractors, "extract-second", built, executed, []contracts.Validator{
integrationValidator{name: "reject-second", approve: false},
})
if err := registries.Mergers.Register("merge", func() (contracts.Merger, error) { if err := registries.Mergers.Register("merge", func() (contracts.Merger, error) {
*built = append(*built, "merge") *built = append(*built, "merge")
return integrationMerger{}, nil return integrationMerger{}, nil
@@ -91,12 +84,12 @@ func integrationRegistries(t *testing.T, built, executed *[]string) Registries {
return registries return registries
} }
func registerIntegrationExtractor(t *testing.T, registry *ExtractorRegistry, key string, built, executed *[]string, validators []contracts.Validator) { func registerIntegrationExtractor(t *testing.T, registry *ExtractorRegistry, key string, built, executed *[]string) {
t.Helper() t.Helper()
if err := registry.Register(key, func() (contracts.Extractor, error) { if err := registry.Register(key, func() (contracts.Extractor, error) {
*built = append(*built, key) *built = append(*built, key)
return integrationExtractor{key: key, executed: executed, validators: validators}, nil return integrationExtractor{key: key, executed: executed}, nil
}); err != nil { }); err != nil {
t.Fatalf("Register(%q) error = %v, want nil", key, err) t.Fatalf("Register(%q) error = %v, want nil", key, err)
} }
@@ -126,46 +119,41 @@ func (chunker integrationChunker) Chunk(ctx context.Context, req contracts.Chunk
return contracts.ChunkResult{ return contracts.ChunkResult{
Chunks: []contracts.SourceChunk{ Chunks: []contracts.SourceChunk{
{ {
ID: "chunk-0", ID: "chunk-0",
SourceID: req.Source.ID, SourceID: req.Source.ID,
Index: 0, Index: 0,
Units: req.Source.Units, StartUnitID: req.Source.Units[0].ID,
EndUnitID: req.Source.Units[len(req.Source.Units)-1].ID,
Content: []byte(`{"units":[1]}`),
MediaType: "application/json",
Units: req.Source.Units,
}, },
}, },
}, nil }, nil
} }
type integrationExtractor struct { type integrationExtractor struct {
key string key string
executed *[]string executed *[]string
validators []contracts.Validator
} }
func (extractor integrationExtractor) Key() string { func (extractor integrationExtractor) Key() string {
return extractor.key return extractor.key
} }
func (extractor integrationExtractor) ArtifactType() string {
return "generic-artifact"
}
func (extractor integrationExtractor) SchemaVersion() string {
return "v1"
}
func (extractor integrationExtractor) ReferenceSlots() []contracts.ReferenceSlot { func (extractor integrationExtractor) ReferenceSlots() []contracts.ReferenceSlot {
return nil return nil
} }
func (extractor integrationExtractor) Validators() []contracts.Validator {
return extractor.validators
}
func (extractor integrationExtractor) Extract(ctx context.Context, req contracts.ExtractionRequest) (contracts.ExtractionResult, error) { func (extractor integrationExtractor) Extract(ctx context.Context, req contracts.ExtractionRequest) (contracts.ExtractionResult, error) {
*extractor.executed = append(*extractor.executed, extractor.key+":"+req.Chunk.ID) *extractor.executed = append(*extractor.executed, extractor.key+":"+req.Chunk.ID)
return contracts.ExtractionResult{ return contracts.ExtractionResult{
Candidates: []artifacts.ArtifactCandidate{ Output: contracts.ExtractOutput{
{Payload: []byte(`{"value":true}`)}, Schema: contracts.ResponseSchema{ID: "integration", Name: "integration", Version: "v1"},
Payload: contracts.RawPayload{
Content: []byte(`{"value":true}`),
MediaType: "application/json",
},
}, },
}, nil }, nil
} }
@@ -179,11 +167,20 @@ func (merger integrationMerger) Key() string {
} }
func (merger integrationMerger) Merge(ctx context.Context, req contracts.MergeRequest) (contracts.MergeResult, error) { func (merger integrationMerger) Merge(ctx context.Context, req contracts.MergeRequest) (contracts.MergeResult, error) {
var candidates []artifacts.ArtifactCandidate output := contracts.MergeOutput{
for _, chunkArtifacts := range req.ChunkArtifacts { LaneID: req.LaneID,
candidates = append(candidates, chunkArtifacts.Candidates...) Schema: contracts.ResponseSchema{ID: "integration", Name: "integration", Version: "v1"},
Payload: contracts.RawPayload{
Content: []byte(`{"merged":true}`),
MediaType: "application/json",
},
} }
return contracts.MergeResult{Candidates: candidates}, nil if len(req.ExtractOutputs) > 0 {
output.SourceID = req.ExtractOutputs[0].SourceID
output.Schema = req.ExtractOutputs[0].Schema
output.Payload = req.ExtractOutputs[0].Payload
}
return contracts.MergeResult{Output: output}, nil
} }
func (normalizer integrationNormalizer) Key() string { func (normalizer integrationNormalizer) Key() string {
@@ -195,7 +192,14 @@ func (normalizer integrationNormalizer) ReferenceSlots() []contracts.ReferenceSl
} }
func (normalizer integrationNormalizer) Normalize(ctx context.Context, req contracts.NormalizeRequest) (contracts.NormalizeResult, error) { func (normalizer integrationNormalizer) Normalize(ctx context.Context, req contracts.NormalizeRequest) (contracts.NormalizeResult, error) {
return contracts.NormalizeResult{Candidates: req.Candidates}, nil return contracts.NormalizeResult{
Output: contracts.NormalizeOutput{
LaneID: req.LaneID,
SourceID: req.MergeOutput.SourceID,
Schema: req.MergeOutput.Schema,
Payload: req.MergeOutput.Payload,
},
}, nil
} }
type integrationOutput struct{} type integrationOutput struct{}
@@ -212,30 +216,6 @@ func (output integrationOutput) Encode(ctx context.Context, req contracts.Output
}, nil }, nil
} }
type integrationValidator struct {
name string
approve bool
}
func (validator integrationValidator) Name() string {
return validator.name
}
func (validator integrationValidator) Validate(ctx context.Context, req contracts.ValidationRequest) (contracts.ValidationResult, error) {
decisions := make([]contracts.ValidationDecision, 0, len(req.Candidates))
for _, candidate := range req.Candidates {
if validator.approve {
decisions = append(decisions, validate.Approved(candidate.Index))
} else {
decisions = append(decisions, validate.Rejected(candidate.Index, "invalid", "not accepted"))
}
}
return contracts.ValidationResult{
ValidatorName: validator.name,
Decisions: decisions,
}, nil
}
func integrationPipeline() ResolvedPipeline { func integrationPipeline() ResolvedPipeline {
return ResolvedPipeline{ return ResolvedPipeline{
ID: "pipeline-1", ID: "pipeline-1",
@@ -267,23 +247,15 @@ func integrationSourceDocument() *source.SourceDocument {
Format: "text/plain", Format: "text/plain",
Digest: "sha256:abc123", Digest: "sha256:abc123",
Units: []source.SourceUnit{ Units: []source.SourceUnit{
{ID: "u1", Kind: "unit", Text: "Source unit."}, {ID: 1, Kind: "unit", Text: "Source unit."},
}, },
} }
} }
func artifactKeys(approved []artifacts.Artifact) []string { func normalizeOutputKeys(outputs []contracts.NormalizeOutput) []string {
keys := make([]string, 0, len(approved)) keys := make([]string, 0, len(outputs))
for _, artifact := range approved { for _, output := range outputs {
keys = append(keys, artifact.ExtractorKey) keys = append(keys, output.NormalizerKey)
}
return keys
}
func rejectedKeys(rejected []artifacts.RejectedArtifact) []string {
keys := make([]string, 0, len(rejected))
for _, artifact := range rejected {
keys = append(keys, artifact.Candidate.ExtractorKey)
} }
return keys return keys
} }

View File

@@ -15,17 +15,17 @@ import (
"gitea.maximumdirect.net/eric/notarius/internal/core/artifacts" "gitea.maximumdirect.net/eric/notarius/internal/core/artifacts"
"gitea.maximumdirect.net/eric/notarius/internal/core/source" "gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts" "gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/validate"
) )
type Registries struct { type Registries struct {
Inputs *InputAdapterRegistry Inputs *InputAdapterRegistry
Chunkers *ChunkerRegistry Chunkers *ChunkerRegistry
Extractors *ExtractorRegistry Extractors *ExtractorRegistry
Mergers *MergerRegistry Mergers *MergerRegistry
Normalizers *NormalizerRegistry Normalizers *NormalizerRegistry
Validators *ValidatorRegistry Validators *ValidatorRegistry
Outputs *OutputEncoderRegistry RawValidators *RawValidationRegistry
Outputs *OutputEncoderRegistry
} }
type Runner struct { type Runner struct {
@@ -51,11 +51,11 @@ type RunInput struct {
} }
type RunOutput struct { type RunOutput struct {
Manifest artifacts.RunManifest `json:"manifest"` Manifest artifacts.RunManifest `json:"manifest"`
Approved []artifacts.Artifact `json:"approved,omitempty"` NormalizeOutputs []contracts.NormalizeOutput `json:"normalize_outputs,omitempty"`
Rejected []artifacts.RejectedArtifact `json:"rejected,omitempty"` Rejected []contracts.RejectedOutput `json:"rejected,omitempty"`
Warnings []contracts.Warning `json:"warnings,omitempty"` Warnings []contracts.Warning `json:"warnings,omitempty"`
OutputFiles []contracts.OutputFile `json:"-"` OutputFiles []contracts.OutputFile `json:"-"`
} }
func (r *Runner) Run(ctx context.Context, input RunInput) (output RunOutput, err error) { func (r *Runner) Run(ctx context.Context, input RunInput) (output RunOutput, err error) {
@@ -104,32 +104,51 @@ func (r *Runner) Run(ctx context.Context, input RunInput) (output RunOutput, err
return failOutput(output), fmt.Errorf("build chunker %q: %w", input.Pipeline.Chunk.Module, err) return failOutput(output), fmt.Errorf("build chunker %q: %w", input.Pipeline.Chunk.Module, err)
} }
attachModuleManifestMetadata(&output, "chunker", chunker) attachModuleManifestMetadata(&output, "chunker", chunker)
chunkResult, err := chunker.Chunk(ctx, contracts.ChunkRequest{ var canonicalChunks []contracts.SourceChunk
Source: doc, var chunkWarnings []contracts.Warning
SourceInput: sourceInput.Clone(), chunksAccepted, chunkRejection, err := runWithRetry(ctx, input.Pipeline.Chunk.Retries, func(attempt int) (bool, *contracts.RejectedOutput, error) {
SessionID: sessionID, chunkResult, err := chunker.Chunk(ctx, contracts.ChunkRequest{
References: CloneReferenceSet(input.Pipeline.ChunkReferences.ReferenceSet), Source: doc,
LLMClient: input.LLMClient, SourceInput: sourceInput.Clone(),
LLMProfile: input.Pipeline.Chunk.LLMProfile, SessionID: sessionID,
Options: cloneOptions(input.Pipeline.Chunk.Options), References: CloneReferenceSet(input.Pipeline.ChunkReferences.ReferenceSet),
Metadata: input.Metadata, LLMClient: input.LLMClient,
LLMProfile: input.Pipeline.Chunk.LLMProfile,
Options: cloneOptions(input.Pipeline.Chunk.Options),
Metadata: input.Metadata,
})
if err != nil {
return false, nil, fmt.Errorf("chunk source with chunker %q: %w", chunker.Key(), err)
}
if len(chunkResult.Chunks) == 0 {
return false, nil, fmt.Errorf("chunker %q returned no chunks", chunker.Key())
}
chunks, err := validateAndCanonicalizeChunkResult(doc, chunkResult.Chunks)
if err != nil {
return false, nil, fmt.Errorf("validate chunks from chunker %q: %w", chunker.Key(), err)
}
rejection, err := r.validateChunksRaw(ctx, doc, chunker.Key(), chunks, input.Metadata, attempt)
if err != nil || rejection != nil {
return false, rejection, err
}
canonicalChunks = chunks
chunkWarnings = cloneWarnings(chunkResult.Warnings)
return true, nil, nil
}) })
output.Warnings = append(output.Warnings, chunkResult.Warnings...)
if err != nil { if err != nil {
return failOutput(output), fmt.Errorf("chunk source with chunker %q: %w", chunker.Key(), err) return failOutput(output), err
} }
if len(chunkResult.Chunks) == 0 { if !chunksAccepted {
return failOutput(output), fmt.Errorf("chunker %q returned no chunks", chunker.Key()) output.Rejected = append(output.Rejected, *chunkRejection)
} } else {
canonicalChunks, err := validateAndCanonicalizeChunkResult(doc, chunkResult.Chunks) output.Warnings = append(output.Warnings, chunkWarnings...)
if err != nil {
return failOutput(output), fmt.Errorf("validate chunks from chunker %q: %w", chunker.Key(), err)
} }
nextCandidateIndex := 0 if chunksAccepted {
for _, lane := range input.Pipeline.ArtifactLanes { for _, lane := range input.Pipeline.ArtifactLanes {
if err := r.runLane(ctx, input, doc, sourceInput, sessionID, canonicalChunks, lane, &output, &nextCandidateIndex); err != nil { if err := r.runLane(ctx, input, doc, sourceInput, sessionID, canonicalChunks, lane, &output); err != nil {
return failOutput(output), err return failOutput(output), err
}
} }
} }
@@ -138,6 +157,7 @@ func (r *Runner) Run(ctx context.Context, input RunInput) (output RunOutput, err
} else { } else {
output.Manifest.ValidationStatus = "approved" output.Manifest.ValidationStatus = "approved"
} }
populateRawOutputManifest(&output)
output.Manifest.CompletedAt = timePtr(time.Now().UTC()) output.Manifest.CompletedAt = timePtr(time.Now().UTC())
encoder, err := r.registries.Outputs.Build(input.Pipeline.Output.Module) encoder, err := r.registries.Outputs.Build(input.Pipeline.Output.Module)
@@ -146,13 +166,13 @@ func (r *Runner) Run(ctx context.Context, input RunInput) (output RunOutput, err
} }
attachModuleManifestMetadata(&output, "output", encoder) attachModuleManifestMetadata(&output, "output", encoder)
encoded, err := encoder.Encode(ctx, contracts.OutputRequest{ encoded, err := encoder.Encode(ctx, contracts.OutputRequest{
Manifest: output.Manifest, Manifest: output.Manifest,
Approved: output.Approved, NormalizeOutputs: cloneNormalizeOutputs(output.NormalizeOutputs),
Rejected: output.Rejected, Rejected: cloneRejectedOutputs(output.Rejected),
Warnings: output.Warnings, Warnings: output.Warnings,
LLMProfile: input.Pipeline.Output.LLMProfile, LLMProfile: input.Pipeline.Output.LLMProfile,
Options: cloneOptions(input.Pipeline.Output.Options), Options: cloneOptions(input.Pipeline.Output.Options),
Metadata: input.Metadata, Metadata: input.Metadata,
}) })
output.Warnings = append(output.Warnings, encoded.Warnings...) output.Warnings = append(output.Warnings, encoded.Warnings...)
if err != nil { if err != nil {
@@ -167,7 +187,7 @@ func (r *Runner) Run(ctx context.Context, input RunInput) (output RunOutput, err
return output, nil return output, nil
} }
func (r *Runner) runLane(ctx context.Context, input RunInput, doc *source.SourceDocument, sourceInput contracts.LLMInputMaterial, sessionID string, chunks []contracts.SourceChunk, lane ResolvedArtifactLane, output *RunOutput, nextCandidateIndex *int) error { func (r *Runner) runLane(ctx context.Context, input RunInput, doc *source.SourceDocument, sourceInput contracts.LLMInputMaterial, sessionID string, chunks []contracts.SourceChunk, lane ResolvedArtifactLane, output *RunOutput) error {
extractor, err := r.registries.Extractors.Build(lane.Extract.Module) extractor, err := r.registries.Extractors.Build(lane.Extract.Module)
if err != nil { if err != nil {
return fmt.Errorf("build extractor %q for lane %q: %w", lane.Extract.Module, lane.ID, err) return fmt.Errorf("build extractor %q for lane %q: %w", lane.Extract.Module, lane.ID, err)
@@ -182,91 +202,168 @@ func (r *Runner) runLane(ctx context.Context, input RunInput, doc *source.Source
} }
setLaneManifestMetadata(output, lane.ID, extractor, merger, normalizer) setLaneManifestMetadata(output, lane.ID, extractor, merger, normalizer)
var validators []validatorExecution extractOutputs := make([]contracts.ExtractOutput, 0, len(chunks))
if len(lane.Validators) > 0 {
validators, err = r.buildConfiguredValidators(lane)
if err != nil {
return err
}
} else {
for _, validator := range extractor.Validators() {
validators = append(validators, validatorExecution{validator: validator})
}
}
chunkArtifacts := make([]contracts.ChunkArtifacts, 0, len(chunks))
for index := range chunks { for index := range chunks {
chunk := chunks[index] chunk := chunks[index]
result, err := extractor.Extract(ctx, contracts.ExtractionRequest{ var acceptedOutput contracts.ExtractOutput
Source: doc, var acceptedWarnings []contracts.Warning
Chunk: &chunk, accepted, rejection, err := runWithRetry(ctx, lane.Extract.Retries, func(attempt int) (bool, *contracts.RejectedOutput, error) {
SourceInput: sourceInput.Clone(), result, err := extractor.Extract(ctx, contracts.ExtractionRequest{
SessionID: sessionID, Source: doc,
References: CloneReferenceSet(lane.ExtractReferences.ReferenceSet), Chunk: &chunk,
LLMClient: input.LLMClient, SourceInput: sourceInput.Clone(),
LLMProfile: lane.Extract.LLMProfile, SessionID: sessionID,
Options: cloneOptions(lane.Extract.Options), References: CloneReferenceSet(lane.ExtractReferences.ReferenceSet),
Metadata: input.Metadata, LLMClient: input.LLMClient,
LLMProfile: lane.Extract.LLMProfile,
Options: cloneOptions(lane.Extract.Options),
Metadata: input.Metadata,
})
if err != nil {
return false, nil, fmt.Errorf("extract lane %q chunk %q with extractor %q: %w", lane.ID, chunk.ID, extractor.Key(), err)
}
extractOutput := result.Output
extractOutput.LaneID = lane.ID
extractOutput.ExtractorKey = extractor.Key()
extractOutput.SourceID = doc.ID
extractOutput.ChunkID = chunk.ID
extractOutput.ChunkIndex = chunk.Index
extractOutput.Payload.Warnings = append(extractOutput.Payload.Warnings, result.Warnings...)
validationWarnings, rejection, err := r.validateRaw(ctx, rawValidationTarget{
stage: StageExtract,
laneID: lane.ID,
moduleKey: extractor.Key(),
source: doc,
sourceID: doc.ID,
chunkID: chunk.ID,
chunkIndex: chunk.Index,
schema: extractOutput.Schema,
payload: extractOutput.Payload,
metadata: input.Metadata,
attempt: attempt,
})
if err != nil || rejection != nil {
return false, rejection, err
}
acceptedOutput = cloneExtractOutput(extractOutput)
acceptedWarnings = append(cloneWarnings(result.Warnings), validationWarnings...)
return true, nil, nil
}) })
output.Warnings = append(output.Warnings, result.Warnings...)
if err != nil {
return fmt.Errorf("extract lane %q chunk %q with extractor %q: %w", lane.ID, chunk.ID, extractor.Key(), err)
}
candidates, err := normalizeCandidates(extractor, result.Candidates, nextCandidateIndex)
if err != nil { if err != nil {
return err return err
} }
chunkArtifacts = append(chunkArtifacts, contracts.ChunkArtifacts{ if !accepted {
Chunk: chunk, output.Rejected = append(output.Rejected, *rejection)
Candidates: candidates, continue
}
output.Warnings = append(output.Warnings, acceptedWarnings...)
extractOutputs = append(extractOutputs, acceptedOutput)
}
if len(extractOutputs) == 0 {
return nil
}
var acceptedMerge contracts.MergeOutput
var mergeWarnings []contracts.Warning
mergeAccepted, mergeRejection, err := runWithRetry(ctx, lane.Merge.Retries, func(attempt int) (bool, *contracts.RejectedOutput, error) {
mergeResult, err := merger.Merge(ctx, contracts.MergeRequest{
Source: doc,
LaneID: lane.ID,
ExtractOutputs: cloneExtractOutputs(extractOutputs),
SourceInput: sourceInput.Clone(),
SessionID: sessionID,
References: CloneReferenceSet(lane.MergeReferences.ReferenceSet),
LLMClient: input.LLMClient,
LLMProfile: lane.Merge.LLMProfile,
Options: cloneOptions(lane.Merge.Options),
Metadata: input.Metadata,
}) })
} if err != nil {
return false, nil, fmt.Errorf("merge lane %q with merger %q: %w", lane.ID, merger.Key(), err)
mergeResult, err := merger.Merge(ctx, contracts.MergeRequest{ }
Source: doc, mergeOutput := mergeResult.Output
LaneID: lane.ID, mergeOutput.LaneID = lane.ID
ChunkArtifacts: chunkArtifacts, mergeOutput.MergerKey = merger.Key()
LLMProfile: lane.Merge.LLMProfile, mergeOutput.SourceID = doc.ID
Options: cloneOptions(lane.Merge.Options), mergeOutput.Payload.Warnings = append(mergeOutput.Payload.Warnings, mergeResult.Warnings...)
Metadata: input.Metadata, validationWarnings, rejection, err := r.validateRaw(ctx, rawValidationTarget{
stage: StageMerge,
laneID: lane.ID,
moduleKey: merger.Key(),
source: doc,
sourceID: doc.ID,
schema: mergeOutput.Schema,
payload: mergeOutput.Payload,
metadata: input.Metadata,
attempt: attempt,
})
if err != nil || rejection != nil {
return false, rejection, err
}
acceptedMerge = cloneMergeOutput(mergeOutput)
mergeWarnings = append(cloneWarnings(mergeResult.Warnings), validationWarnings...)
return true, nil, nil
}) })
output.Warnings = append(output.Warnings, mergeResult.Warnings...)
if err != nil {
return fmt.Errorf("merge lane %q with merger %q: %w", lane.ID, merger.Key(), err)
}
normalizeResult, err := normalizer.Normalize(ctx, contracts.NormalizeRequest{
Source: doc,
LaneID: lane.ID,
Candidates: mergeResult.Candidates,
SourceInput: sourceInput.Clone(),
SessionID: sessionID,
References: CloneReferenceSet(lane.NormalizeReferences.ReferenceSet),
LLMClient: input.LLMClient,
LLMProfile: lane.Normalize.LLMProfile,
Options: cloneOptions(lane.Normalize.Options),
Metadata: input.Metadata,
})
output.Warnings = append(output.Warnings, normalizeResult.Warnings...)
if err != nil {
return fmt.Errorf("normalize lane %q with normalizer %q: %w", lane.ID, normalizer.Key(), err)
}
if err := validateCandidateEnvelope(extractor, normalizeResult.Candidates); err != nil {
return fmt.Errorf("validate normalized candidates for lane %q: %w", lane.ID, err)
}
approved, rejected, warnings, err := runValidators(ctx, extractor.Key(), validators, doc, normalizeResult.Candidates, input.Metadata)
output.Warnings = append(output.Warnings, warnings...)
output.Rejected = append(output.Rejected, rejected...)
if err != nil { if err != nil {
return err return err
} }
if !mergeAccepted {
for _, candidate := range approved { output.Rejected = append(output.Rejected, *mergeRejection)
output.Approved = append(output.Approved, artifacts.ArtifactFromCandidate(candidate)) return nil
} }
output.Warnings = append(output.Warnings, mergeWarnings...)
var acceptedNormalize contracts.NormalizeOutput
var normalizeWarnings []contracts.Warning
normalizeAccepted, normalizeRejection, err := runWithRetry(ctx, lane.Normalize.Retries, func(attempt int) (bool, *contracts.RejectedOutput, error) {
normalizeResult, err := normalizer.Normalize(ctx, contracts.NormalizeRequest{
Source: doc,
LaneID: lane.ID,
MergeOutput: cloneMergeOutput(acceptedMerge),
SourceInput: sourceInput.Clone(),
SessionID: sessionID,
References: CloneReferenceSet(lane.NormalizeReferences.ReferenceSet),
LLMClient: input.LLMClient,
LLMProfile: lane.Normalize.LLMProfile,
Options: cloneOptions(lane.Normalize.Options),
Metadata: input.Metadata,
})
if err != nil {
return false, nil, fmt.Errorf("normalize lane %q with normalizer %q: %w", lane.ID, normalizer.Key(), err)
}
normalizeOutput := normalizeResult.Output
normalizeOutput.LaneID = lane.ID
normalizeOutput.NormalizerKey = normalizer.Key()
normalizeOutput.SourceID = doc.ID
normalizeOutput.Payload.Warnings = append(normalizeOutput.Payload.Warnings, normalizeResult.Warnings...)
validationWarnings, rejection, err := r.validateRaw(ctx, rawValidationTarget{
stage: StageNormalize,
laneID: lane.ID,
moduleKey: normalizer.Key(),
source: doc,
sourceID: doc.ID,
schema: normalizeOutput.Schema,
payload: normalizeOutput.Payload,
metadata: input.Metadata,
attempt: attempt,
})
if err != nil || rejection != nil {
return false, rejection, err
}
acceptedNormalize = cloneNormalizeOutput(normalizeOutput)
normalizeWarnings = append(cloneWarnings(normalizeResult.Warnings), validationWarnings...)
return true, nil, nil
})
if err != nil {
return err
}
if !normalizeAccepted {
output.Rejected = append(output.Rejected, *normalizeRejection)
return nil
}
output.Warnings = append(output.Warnings, normalizeWarnings...)
output.NormalizeOutputs = append(output.NormalizeOutputs, acceptedNormalize)
return nil return nil
} }
@@ -275,6 +372,146 @@ type validatorExecution struct {
binding ModuleBinding binding ModuleBinding
} }
type rawValidationTarget struct {
stage ModuleStage
laneID string
moduleKey string
source *source.SourceDocument
sourceID string
chunkID string
chunkIndex int
schema contracts.ResponseSchema
payload contracts.RawPayload
metadata map[string]any
attempt int
}
func runWithRetry(ctx context.Context, retries int, run func(attempt int) (bool, *contracts.RejectedOutput, error)) (bool, *contracts.RejectedOutput, error) {
attempts := 1
if retries > 0 {
attempts += retries
}
var lastRejection *contracts.RejectedOutput
for attempt := 1; attempt <= attempts; attempt++ {
if err := ctx.Err(); err != nil {
return false, nil, err
}
accepted, rejection, err := run(attempt)
if err != nil {
if ctxErr := ctx.Err(); ctxErr != nil {
return false, nil, ctxErr
}
if attempt == attempts {
return false, nil, fmt.Errorf("failed after %d attempt(s): %w", attempt, err)
}
continue
}
if accepted {
return true, nil, nil
}
if rejection != nil {
rejection.AttemptCount = attempt
lastRejection = rejection
}
if ctxErr := ctx.Err(); ctxErr != nil {
return false, nil, ctxErr
}
if attempt == attempts {
if lastRejection == nil {
lastRejection = &contracts.RejectedOutput{
ReasonCode: "raw_output_rejected",
Message: "raw output rejected",
AttemptCount: attempt,
}
}
return false, lastRejection, nil
}
}
return false, lastRejection, nil
}
func (r *Runner) validateChunksRaw(ctx context.Context, doc *source.SourceDocument, moduleKey string, chunks []contracts.SourceChunk, metadata map[string]any, attempt int) (*contracts.RejectedOutput, error) {
for _, chunk := range chunks {
_, rejection, err := r.validateRaw(ctx, rawValidationTarget{
stage: StageChunk,
moduleKey: moduleKey,
source: doc,
sourceID: doc.ID,
chunkID: chunk.ID,
chunkIndex: chunk.Index,
payload: contracts.RawPayload{
Content: append([]byte(nil), chunk.Content...),
MediaType: chunk.MediaType,
Metadata: cloneMetadata(chunk.Metadata),
},
metadata: metadata,
attempt: attempt,
})
if err != nil {
return nil, err
}
if rejection != nil {
return rejection, nil
}
}
return nil, nil
}
func (r *Runner) validateRaw(ctx context.Context, target rawValidationTarget) ([]contracts.Warning, *contracts.RejectedOutput, error) {
validators := r.registries.RawValidators.Validators(target.stage, target.moduleKey)
if len(validators) == 0 {
return nil, nil, nil
}
request := contracts.RawValidationRequest{
Stage: string(target.stage),
LaneID: target.laneID,
ModuleKey: target.moduleKey,
Source: target.source,
SourceID: target.sourceID,
ChunkID: target.chunkID,
ChunkIndex: target.chunkIndex,
Schema: target.schema,
Payload: cloneRawPayload(target.payload),
Metadata: cloneMetadata(target.metadata),
}
var warnings []contracts.Warning
for _, validator := range validators {
result, err := validator.ValidateRaw(ctx, request)
if err != nil {
return nil, nil, fmt.Errorf("validate raw %s output with validator %q: %w", target.stage, validator.Name(), err)
}
if !result.Approved {
reasonCode := strings.TrimSpace(result.ReasonCode)
if reasonCode == "" {
reasonCode = "raw_output_rejected"
}
message := strings.TrimSpace(result.Message)
if message == "" {
message = "raw output rejected"
}
return nil, &contracts.RejectedOutput{
Stage: string(target.stage),
LaneID: target.laneID,
ModuleKey: target.moduleKey,
ChunkID: target.chunkID,
ChunkIndex: target.chunkIndex,
ValidatorName: validator.Name(),
ReasonCode: reasonCode,
Message: message,
AttemptCount: target.attempt,
DiagnosticArtifactPath: result.DiagnosticArtifactPath,
}, nil
}
warnings = append(warnings, result.Warnings...)
}
return warnings, nil, nil
}
func (r *Runner) buildConfiguredValidators(lane ResolvedArtifactLane) ([]validatorExecution, error) { func (r *Runner) buildConfiguredValidators(lane ResolvedArtifactLane) ([]validatorExecution, error) {
validators := make([]validatorExecution, 0, len(lane.Validators)) validators := make([]validatorExecution, 0, len(lane.Validators))
for _, binding := range lane.Validators { for _, binding := range lane.Validators {
@@ -400,12 +637,64 @@ func manifestFromPipeline(input RunInput) artifacts.RunManifest {
func failOutput(output RunOutput) RunOutput { func failOutput(output RunOutput) RunOutput {
if output.Manifest.PipelineID != "" { if output.Manifest.PipelineID != "" {
populateRawOutputManifest(&output)
output.Manifest.ValidationStatus = "failed" output.Manifest.ValidationStatus = "failed"
output.Manifest.CompletedAt = timePtr(time.Now().UTC()) output.Manifest.CompletedAt = timePtr(time.Now().UTC())
} }
return output return output
} }
func populateRawOutputManifest(output *RunOutput) {
if output == nil {
return
}
output.Manifest.NormalizedOutputs = normalizedOutputManifests(output.NormalizeOutputs)
output.Manifest.RejectedOutputs = rejectedOutputManifests(output.Rejected)
}
func normalizedOutputManifests(outputs []contracts.NormalizeOutput) []artifacts.NormalizedOutputManifest {
if len(outputs) == 0 {
return nil
}
manifests := make([]artifacts.NormalizedOutputManifest, 0, len(outputs))
for _, output := range outputs {
manifests = append(manifests, artifacts.NormalizedOutputManifest{
LaneID: output.LaneID,
ModuleKey: output.NormalizerKey,
SourceID: output.SourceID,
MediaType: output.Payload.MediaType,
Schema: artifacts.OutputSchemaProvenance{
ID: output.Schema.ID,
Name: output.Schema.Name,
Version: output.Schema.Version,
},
})
}
return manifests
}
func rejectedOutputManifests(rejected []contracts.RejectedOutput) []artifacts.RejectedOutputManifest {
if len(rejected) == 0 {
return nil
}
manifests := make([]artifacts.RejectedOutputManifest, 0, len(rejected))
for _, output := range rejected {
manifests = append(manifests, artifacts.RejectedOutputManifest{
Stage: output.Stage,
LaneID: output.LaneID,
ModuleKey: output.ModuleKey,
ChunkID: output.ChunkID,
ChunkIndex: output.ChunkIndex,
ValidatorName: output.ValidatorName,
ReasonCode: output.ReasonCode,
Message: output.Message,
AttemptCount: output.AttemptCount,
DiagnosticArtifactPath: output.DiagnosticArtifactPath,
})
}
return manifests
}
func setLaneManifestMetadata(output *RunOutput, laneID string, modules ...any) { func setLaneManifestMetadata(output *RunOutput, laneID string, modules ...any) {
if output == nil { if output == nil {
return return
@@ -628,6 +917,59 @@ func cloneWarnings(warnings []contracts.Warning) []contracts.Warning {
return append([]contracts.Warning(nil), warnings...) return append([]contracts.Warning(nil), warnings...)
} }
func cloneRawPayload(payload contracts.RawPayload) contracts.RawPayload {
return contracts.RawPayload{
Content: append([]byte(nil), payload.Content...),
MediaType: payload.MediaType,
Metadata: cloneMetadata(payload.Metadata),
Warnings: cloneWarnings(payload.Warnings),
}
}
func cloneExtractOutput(output contracts.ExtractOutput) contracts.ExtractOutput {
output.Payload = cloneRawPayload(output.Payload)
return output
}
func cloneExtractOutputs(outputs []contracts.ExtractOutput) []contracts.ExtractOutput {
if len(outputs) == 0 {
return nil
}
out := make([]contracts.ExtractOutput, 0, len(outputs))
for _, output := range outputs {
out = append(out, cloneExtractOutput(output))
}
return out
}
func cloneMergeOutput(output contracts.MergeOutput) contracts.MergeOutput {
output.Payload = cloneRawPayload(output.Payload)
return output
}
func cloneNormalizeOutput(output contracts.NormalizeOutput) contracts.NormalizeOutput {
output.Payload = cloneRawPayload(output.Payload)
return output
}
func cloneNormalizeOutputs(outputs []contracts.NormalizeOutput) []contracts.NormalizeOutput {
if len(outputs) == 0 {
return nil
}
out := make([]contracts.NormalizeOutput, 0, len(outputs))
for _, output := range outputs {
out = append(out, cloneNormalizeOutput(output))
}
return out
}
func cloneRejectedOutputs(rejected []contracts.RejectedOutput) []contracts.RejectedOutput {
if len(rejected) == 0 {
return nil
}
return append([]contracts.RejectedOutput(nil), rejected...)
}
func timePtr(t time.Time) *time.Time { func timePtr(t time.Time) *time.Time {
return &t return &t
} }
@@ -640,115 +982,3 @@ func pipelineUsesConfiguredValidators(pipeline ResolvedPipeline) bool {
} }
return false return false
} }
func normalizeCandidates(extractor contracts.Extractor, candidates []artifacts.ArtifactCandidate, nextIndex *int) ([]artifacts.ArtifactCandidate, error) {
normalized := make([]artifacts.ArtifactCandidate, 0, len(candidates))
for _, candidate := range candidates {
candidate.Index = *nextIndex
*nextIndex = *nextIndex + 1
if candidate.ExtractorKey == "" {
candidate.ExtractorKey = extractor.Key()
} else if candidate.ExtractorKey != extractor.Key() {
return nil, fmt.Errorf("candidate extractor_key %q does not match extractor %q", candidate.ExtractorKey, extractor.Key())
}
if candidate.ArtifactType == "" {
candidate.ArtifactType = extractor.ArtifactType()
} else if candidate.ArtifactType != extractor.ArtifactType() {
return nil, fmt.Errorf("candidate artifact_type %q does not match extractor %q artifact type %q", candidate.ArtifactType, extractor.Key(), extractor.ArtifactType())
}
if candidate.SchemaVersion == "" {
candidate.SchemaVersion = extractor.SchemaVersion()
} else if candidate.SchemaVersion != extractor.SchemaVersion() {
return nil, fmt.Errorf("candidate schema_version %q does not match extractor %q schema version %q", candidate.SchemaVersion, extractor.Key(), extractor.SchemaVersion())
}
normalized = append(normalized, candidate)
}
return normalized, nil
}
func validateCandidateEnvelope(extractor contracts.Extractor, candidates []artifacts.ArtifactCandidate) error {
seen := make(map[int]struct{}, len(candidates))
for _, candidate := range candidates {
if _, ok := seen[candidate.Index]; ok {
return fmt.Errorf("candidate index %d is duplicated", candidate.Index)
}
seen[candidate.Index] = struct{}{}
if candidate.ExtractorKey == "" {
return fmt.Errorf("candidate index %d extractor_key must not be empty", candidate.Index)
}
if candidate.ExtractorKey != extractor.Key() {
return fmt.Errorf("candidate index %d extractor_key %q does not match extractor %q", candidate.Index, candidate.ExtractorKey, extractor.Key())
}
if candidate.ArtifactType == "" {
return fmt.Errorf("candidate index %d artifact_type must not be empty", candidate.Index)
}
if candidate.ArtifactType != extractor.ArtifactType() {
return fmt.Errorf("candidate index %d artifact_type %q does not match extractor %q artifact type %q", candidate.Index, candidate.ArtifactType, extractor.Key(), extractor.ArtifactType())
}
if candidate.SchemaVersion == "" {
return fmt.Errorf("candidate index %d schema_version must not be empty", candidate.Index)
}
if candidate.SchemaVersion != extractor.SchemaVersion() {
return fmt.Errorf("candidate index %d schema_version %q does not match extractor %q schema version %q", candidate.Index, candidate.SchemaVersion, extractor.Key(), extractor.SchemaVersion())
}
}
return nil
}
func runValidators(ctx context.Context, extractorKey string, validators []validatorExecution, doc *source.SourceDocument, candidates []artifacts.ArtifactCandidate, metadata map[string]any) ([]artifacts.ArtifactCandidate, []artifacts.RejectedArtifact, []contracts.Warning, error) {
eligible := candidates
var rejected []artifacts.RejectedArtifact
var warnings []contracts.Warning
for validatorIndex, execution := range validators {
validator := execution.validator
if validator == nil {
return nil, rejected, warnings, fmt.Errorf("extractor %q validator[%d] must not be nil", extractorKey, validatorIndex)
}
result, err := validator.Validate(ctx, contracts.ValidationRequest{
Source: doc,
Candidates: eligible,
LLMProfile: execution.binding.LLMProfile,
Options: cloneOptions(execution.binding.Options),
Metadata: metadata,
})
warnings = append(warnings, result.Warnings...)
if err != nil {
return nil, rejected, warnings, fmt.Errorf("validate extractor %q with validator %q: %w", extractorKey, validator.Name(), err)
}
if result.ValidatorName != validator.Name() {
return nil, rejected, warnings, fmt.Errorf("validator %q returned result for %q", validator.Name(), result.ValidatorName)
}
if err := validate.EnforceDecisionCardinality(eligible, result.Decisions); err != nil {
return nil, rejected, warnings, fmt.Errorf("validate extractor %q with validator %q: %w", extractorKey, validator.Name(), err)
}
decisions := make(map[int]contracts.ValidationDecision, len(result.Decisions))
for _, decision := range result.Decisions {
decisions[decision.CandidateIndex] = decision
}
nextEligible := make([]artifacts.ArtifactCandidate, 0, len(eligible))
for _, candidate := range eligible {
decision := decisions[candidate.Index]
if decision.Approved {
nextEligible = append(nextEligible, candidate)
continue
}
rejected = append(rejected, artifacts.RejectedArtifact{
Candidate: candidate,
ValidatorName: result.ValidatorName,
ReasonCode: decision.ReasonCode,
Message: decision.Message,
})
}
eligible = nextEligible
}
return eligible, rejected, warnings, nil
}

File diff suppressed because it is too large Load Diff

View File

@@ -2,15 +2,15 @@
"id": "fixture-source", "id": "fixture-source",
"units": [ "units": [
{ {
"id": "u1", "id": 1,
"text": "First event." "text": "First event."
}, },
{ {
"id": "u2", "id": 2,
"text": "Second event." "text": "Second event."
}, },
{ {
"id": "u3", "id": 3,
"text": "Third event." "text": "Third event."
} }
] ]

View File

@@ -1,7 +1,7 @@
{ {
"manifest": { "manifest": {
"pipeline_id": "walking-skeleton", "pipeline_id": "walking-skeleton",
"pipeline_digest": "sha256:437d7ff486c336ddba38654ea00b7ef9dd6e49ce09f468698202ad8fc459bdcd", "pipeline_digest": "sha256:75c1f6d64d86666734ccf175cdb82ae90e0ba5be3194be952850d7c00c66e615",
"validation_status": "approved", "validation_status": "approved",
"artifact_lanes": [ "artifact_lanes": [
{ {
@@ -12,40 +12,31 @@
} }
] ]
}, },
"approved": [ "normalize_outputs": [
{ {
"extractor_key": "fake/extract", "lane_id": "events",
"artifact_type": "fake_event", "normalizer_key": "noop",
"schema_version": "v1", "source_id": "fixture-source",
"payload": { "schema": {
"chunk_id": "fixture-source:chunk:0", "id": "fake_event",
"llm_call": 1, "name": "fake_event",
"text": "First event. Second event." "version": "v1"
}, },
"source_refs": [ "media_type": "application/json",
{ "content": {
"source_id": "fixture-source", "outputs": [
"start_unit_id": "u1", {
"end_unit_id": "u2" "chunk_id": "fixture-source:chunk:0",
} "llm_call": 1,
] "text": "First event. Second event."
}, },
{ {
"extractor_key": "fake/extract", "chunk_id": "fixture-source:chunk:1",
"artifact_type": "fake_event", "llm_call": 2,
"schema_version": "v1", "text": "Third event."
"payload": { }
"chunk_id": "fixture-source:chunk:1", ]
"llm_call": 2, }
"text": "Third event."
},
"source_refs": [
{
"source_id": "fixture-source",
"start_unit_id": "u3",
"end_unit_id": "u3"
}
]
} }
] ]
} }

View File

@@ -185,7 +185,7 @@ func (input walkingSkeletonInput) Parse(ctx context.Context, req contracts.Parse
var fixture struct { var fixture struct {
ID string `json:"id"` ID string `json:"id"`
Units []struct { Units []struct {
ID string `json:"id"` ID int `json:"id"`
Text string `json:"text"` Text string `json:"text"`
} `json:"units"` } `json:"units"`
} }
@@ -227,16 +227,24 @@ func (chunker walkingSkeletonChunker) Chunk(ctx context.Context, req contracts.C
return contracts.ChunkResult{ return contracts.ChunkResult{
Chunks: []contracts.SourceChunk{ Chunks: []contracts.SourceChunk{
{ {
ID: req.Source.ID + ":chunk:0", ID: req.Source.ID + ":chunk:0",
SourceID: req.Source.ID, SourceID: req.Source.ID,
Index: 0, Index: 0,
Units: append([]source.SourceUnit(nil), req.Source.Units[:2]...), StartUnitID: req.Source.Units[0].ID,
EndUnitID: req.Source.Units[1].ID,
Content: []byte(`{"units":[1,2]}`),
MediaType: "application/json",
Units: append([]source.SourceUnit(nil), req.Source.Units[:2]...),
}, },
{ {
ID: req.Source.ID + ":chunk:1", ID: req.Source.ID + ":chunk:1",
SourceID: req.Source.ID, SourceID: req.Source.ID,
Index: 1, Index: 1,
Units: append([]source.SourceUnit(nil), req.Source.Units[2:]...), StartUnitID: req.Source.Units[2].ID,
EndUnitID: req.Source.Units[len(req.Source.Units)-1].ID,
Content: []byte(`{"units":[3]}`),
MediaType: "application/json",
Units: append([]source.SourceUnit(nil), req.Source.Units[2:]...),
}, },
}, },
}, nil }, nil
@@ -248,22 +256,10 @@ func (extractor walkingSkeletonExtractor) Key() string {
return "fake/extract" return "fake/extract"
} }
func (extractor walkingSkeletonExtractor) ArtifactType() string {
return "fake_event"
}
func (extractor walkingSkeletonExtractor) SchemaVersion() string {
return "v1"
}
func (extractor walkingSkeletonExtractor) ReferenceSlots() []contracts.ReferenceSlot { func (extractor walkingSkeletonExtractor) ReferenceSlots() []contracts.ReferenceSlot {
return nil return nil
} }
func (extractor walkingSkeletonExtractor) Validators() []contracts.Validator {
return nil
}
func (extractor walkingSkeletonExtractor) Extract(ctx context.Context, req contracts.ExtractionRequest) (contracts.ExtractionResult, error) { func (extractor walkingSkeletonExtractor) Extract(ctx context.Context, req contracts.ExtractionRequest) (contracts.ExtractionResult, error) {
var response struct { var response struct {
Call int `json:"call"` Call int `json:"call"`
@@ -286,16 +282,11 @@ func (extractor walkingSkeletonExtractor) Extract(ctx context.Context, req contr
} }
return contracts.ExtractionResult{ return contracts.ExtractionResult{
Candidates: []artifacts.ArtifactCandidate{ Output: contracts.ExtractOutput{
{ Schema: contracts.ResponseSchema{ID: "fake_event", Name: "fake_event", Version: "v1"},
Payload: payload, Payload: contracts.RawPayload{
SourceRefs: []source.SourceRef{ Content: payload,
{ MediaType: "application/json",
SourceID: req.Source.ID,
StartUnitID: req.Chunk.Units[0].ID,
EndUnitID: req.Chunk.Units[len(req.Chunk.Units)-1].ID,
},
},
}, },
}, },
}, nil }, nil
@@ -328,11 +319,25 @@ func (merger walkingSkeletonMerger) Key() string {
} }
func (merger walkingSkeletonMerger) Merge(ctx context.Context, req contracts.MergeRequest) (contracts.MergeResult, error) { func (merger walkingSkeletonMerger) Merge(ctx context.Context, req contracts.MergeRequest) (contracts.MergeResult, error) {
var candidates []artifacts.ArtifactCandidate outputs := make([]json.RawMessage, 0, len(req.ExtractOutputs))
for _, chunkArtifacts := range req.ChunkArtifacts { for _, output := range req.ExtractOutputs {
candidates = append(candidates, chunkArtifacts.Candidates...) outputs = append(outputs, json.RawMessage(output.Payload.Content))
} }
return contracts.MergeResult{Candidates: candidates}, nil content, err := json.Marshal(map[string]any{"outputs": outputs})
if err != nil {
return contracts.MergeResult{}, err
}
return contracts.MergeResult{
Output: contracts.MergeOutput{
LaneID: req.LaneID,
SourceID: req.Source.ID,
Schema: contracts.ResponseSchema{ID: "fake_event", Name: "fake_event", Version: "v1"},
Payload: contracts.RawPayload{
Content: content,
MediaType: "application/json",
},
},
}, nil
} }
type walkingSkeletonNormalizer struct{} type walkingSkeletonNormalizer struct{}
@@ -356,7 +361,14 @@ func (normalizer walkingSkeletonNormalizer) Normalize(ctx context.Context, req c
}, &response); err != nil { }, &response); err != nil {
return contracts.NormalizeResult{}, err return contracts.NormalizeResult{}, err
} }
return contracts.NormalizeResult{Candidates: req.Candidates}, nil return contracts.NormalizeResult{
Output: contracts.NormalizeOutput{
LaneID: req.LaneID,
SourceID: req.MergeOutput.SourceID,
Schema: req.MergeOutput.Schema,
Payload: req.MergeOutput.Payload,
},
}, nil
} }
type walkingSkeletonOutput struct{} type walkingSkeletonOutput struct{}
@@ -366,9 +378,28 @@ func (output walkingSkeletonOutput) Key() string {
} }
func (output walkingSkeletonOutput) Encode(ctx context.Context, req contracts.OutputRequest) (contracts.OutputResult, error) { func (output walkingSkeletonOutput) Encode(ctx context.Context, req contracts.OutputRequest) (contracts.OutputResult, error) {
type rawOutput struct {
LaneID string `json:"lane_id"`
NormalizerKey string `json:"normalizer_key"`
SourceID string `json:"source_id"`
Schema contracts.ResponseSchema `json:"schema"`
MediaType string `json:"media_type"`
Content json.RawMessage `json:"content"`
}
rawOutputs := make([]rawOutput, 0, len(req.NormalizeOutputs))
for _, output := range req.NormalizeOutputs {
rawOutputs = append(rawOutputs, rawOutput{
LaneID: output.LaneID,
NormalizerKey: output.NormalizerKey,
SourceID: output.SourceID,
Schema: output.Schema,
MediaType: output.Payload.MediaType,
Content: json.RawMessage(output.Payload.Content),
})
}
encoded, err := json.Marshal(struct { encoded, err := json.Marshal(struct {
Manifest artifacts.RunManifest `json:"manifest"` Manifest artifacts.RunManifest `json:"manifest"`
Approved []artifacts.Artifact `json:"approved"` NormalizeOutputs []rawOutput `json:"normalize_outputs"`
}{ }{
Manifest: artifacts.RunManifest{ Manifest: artifacts.RunManifest{
PipelineID: req.Manifest.PipelineID, PipelineID: req.Manifest.PipelineID,
@@ -376,7 +407,7 @@ func (output walkingSkeletonOutput) Encode(ctx context.Context, req contracts.Ou
ArtifactLanes: req.Manifest.ArtifactLanes, ArtifactLanes: req.Manifest.ArtifactLanes,
ValidationStatus: req.Manifest.ValidationStatus, ValidationStatus: req.Manifest.ValidationStatus,
}, },
Approved: req.Approved, NormalizeOutputs: rawOutputs,
}) })
if err != nil { if err != nil {
return contracts.OutputResult{}, err return contracts.OutputResult{}, err

View File

@@ -22,8 +22,7 @@ dnd/scenes boundary policy:
- return sequential scenes with no gaps; - return sequential scenes with no gaps;
- do not overlap scenes; - do not overlap scenes;
- preserve source-unit order; - preserve source-unit order;
- use 1-based integer source-unit numbers from the transcript, where 1 is the - use integer source-unit IDs from the transcript;
first provided source unit;
- each scene must have start_unit_id and end_unit_id; - each scene must have start_unit_id and end_unit_id;
- do not include final chunk IDs or chunk indexes. - do not include final chunk IDs or chunk indexes.

View File

@@ -2,6 +2,7 @@ package scenes
import ( import (
"context" "context"
"encoding/json"
"fmt" "fmt"
"strings" "strings"
@@ -142,7 +143,7 @@ func chunksFromResponse(doc *source.SourceDocument, response chunkResponse) ([]c
return nil, fmt.Errorf("scenes must not be empty") return nil, fmt.Errorf("scenes must not be empty")
} }
unitIndexes := make(map[string]int, len(doc.Units)) unitIndexes := make(map[int]int, len(doc.Units))
for i, unit := range doc.Units { for i, unit := range doc.Units {
unitIndexes[unit.ID] = i unitIndexes[unit.ID] = i
} }
@@ -157,18 +158,18 @@ func chunksFromResponse(doc *source.SourceDocument, response chunkResponse) ([]c
startIndex, ok := unitIndexes[normalized.StartUnitID] startIndex, ok := unitIndexes[normalized.StartUnitID]
if !ok { if !ok {
return nil, fmt.Errorf("scene[%d] start_unit_id %q was not found", i, normalized.StartUnitID) return nil, fmt.Errorf("scene[%d] start_unit_id %d was not found", i, normalized.StartUnitID)
} }
endIndex, ok := unitIndexes[normalized.EndUnitID] endIndex, ok := unitIndexes[normalized.EndUnitID]
if !ok { if !ok {
return nil, fmt.Errorf("scene[%d] end_unit_id %q was not found", i, normalized.EndUnitID) return nil, fmt.Errorf("scene[%d] end_unit_id %d was not found", i, normalized.EndUnitID)
} }
if startIndex > endIndex { if startIndex > endIndex {
return nil, fmt.Errorf("scene[%d] start_unit_id %q appears after end_unit_id %q", i, normalized.StartUnitID, normalized.EndUnitID) return nil, fmt.Errorf("scene[%d] start_unit_id %d appears after end_unit_id %d", i, normalized.StartUnitID, normalized.EndUnitID)
} }
if i == 0 && startIndex != 0 { if i == 0 && startIndex != 0 {
return nil, fmt.Errorf("first scene must start at first source unit %q", doc.Units[0].ID) return nil, fmt.Errorf("first scene must start at first source unit %d", doc.Units[0].ID)
} }
if i > 0 { if i > 0 {
if startIndex <= previousEnd { if startIndex <= previousEnd {
@@ -181,11 +182,19 @@ func chunksFromResponse(doc *source.SourceDocument, response chunkResponse) ([]c
previousEnd = endIndex previousEnd = endIndex
units := cloneUnits(doc.Units[startIndex : endIndex+1]) units := cloneUnits(doc.Units[startIndex : endIndex+1])
content, err := chunkContent(units)
if err != nil {
return nil, err
}
chunks = append(chunks, contracts.SourceChunk{ chunks = append(chunks, contracts.SourceChunk{
ID: fmt.Sprintf("scene-%06d", i+1), ID: fmt.Sprintf("scene-%06d", i+1),
SourceID: doc.ID, SourceID: doc.ID,
Index: i, Index: i,
Units: units, StartUnitID: units[0].ID,
EndUnitID: units[len(units)-1].ID,
Content: content,
MediaType: "application/json",
Units: units,
Metadata: map[string]any{ Metadata: map[string]any{
"scene_title": normalized.ShortTitle, "scene_title": normalized.ShortTitle,
"primary_mode": normalized.PrimaryMode, "primary_mode": normalized.PrimaryMode,
@@ -201,11 +210,23 @@ func chunksFromResponse(doc *source.SourceDocument, response chunkResponse) ([]c
} }
if previousEnd != len(doc.Units)-1 { if previousEnd != len(doc.Units)-1 {
return nil, fmt.Errorf("final scene must end at final source unit %q", doc.Units[len(doc.Units)-1].ID) return nil, fmt.Errorf("final scene must end at final source unit %d", doc.Units[len(doc.Units)-1].ID)
} }
return chunks, nil return chunks, nil
} }
func chunkContent(units []source.SourceUnit) ([]byte, error) {
content, err := json.Marshal(struct {
Units []source.SourceUnit `json:"units"`
}{
Units: units,
})
if err != nil {
return nil, fmt.Errorf("encode chunk content: %w", err)
}
return content, nil
}
func normalizeScene(doc *source.SourceDocument, index int, scene sceneResponse) (normalizedScene, error) { func normalizeScene(doc *source.SourceDocument, index int, scene sceneResponse) (normalizedScene, error) {
startUnitID, err := dnd.ResolveUnitID(doc, "start_unit_id", scene.StartUnitID) startUnitID, err := dnd.ResolveUnitID(doc, "start_unit_id", scene.StartUnitID)
if err != nil { if err != nil {
@@ -226,9 +247,16 @@ func normalizeScene(doc *source.SourceDocument, index int, scene sceneResponse)
BoundaryConfidence: strings.TrimSpace(scene.BoundaryConfidence), BoundaryConfidence: strings.TrimSpace(scene.BoundaryConfidence),
} }
requiredInts := map[string]int{
"start_unit_id": out.StartUnitID,
"end_unit_id": out.EndUnitID,
}
for field, value := range requiredInts {
if value <= 0 {
return normalizedScene{}, fmt.Errorf("scene[%d] %s must be positive", index, field)
}
}
required := map[string]string{ required := map[string]string{
"start_unit_id": out.StartUnitID,
"end_unit_id": out.EndUnitID,
"short_title": out.ShortTitle, "short_title": out.ShortTitle,
"primary_mode": out.PrimaryMode, "primary_mode": out.PrimaryMode,
"summary": out.Summary, "summary": out.Summary,

View File

@@ -170,8 +170,8 @@ func TestChunkReturnsSceneChunksFromStructuredOutput(t *testing.T) {
if got := chunkIDs(result.Chunks); !reflect.DeepEqual(got, []string{"scene-000001", "scene-000002"}) { if got := chunkIDs(result.Chunks); !reflect.DeepEqual(got, []string{"scene-000001", "scene-000002"}) {
t.Fatalf("chunk IDs = %#v, want deterministic scene IDs", got) t.Fatalf("chunk IDs = %#v, want deterministic scene IDs", got)
} }
gotUnits := [][]string{unitIDs(result.Chunks[0].Units), unitIDs(result.Chunks[1].Units)} gotUnits := [][]int{unitIDs(result.Chunks[0].Units), unitIDs(result.Chunks[1].Units)}
wantUnits := [][]string{{"seg-001", "seg-002"}, {"seg-003", "seg-004"}} wantUnits := [][]int{{1, 2}, {3, 4}}
if !reflect.DeepEqual(gotUnits, wantUnits) { if !reflect.DeepEqual(gotUnits, wantUnits) {
t.Fatalf("chunk units = %#v, want %#v", gotUnits, wantUnits) t.Fatalf("chunk units = %#v, want %#v", gotUnits, wantUnits)
} }
@@ -179,13 +179,19 @@ func TestChunkReturnsSceneChunksFromStructuredOutput(t *testing.T) {
if first.SourceID != "session-alpha" || first.Index != 0 { if first.SourceID != "session-alpha" || first.Index != 0 {
t.Fatalf("first chunk = %#v, want source and index fields", first) t.Fatalf("first chunk = %#v, want source and index fields", first)
} }
if first.StartUnitID != 1 || first.EndUnitID != 2 {
t.Fatalf("first boundaries = %d-%d, want 1-2", first.StartUnitID, first.EndUnitID)
}
if first.MediaType != "application/json" || len(first.Content) == 0 {
t.Fatalf("first payload = media type %q length %d, want JSON content", first.MediaType, len(first.Content))
}
if first.Metadata["scene_title"] != "Goblin parley" || if first.Metadata["scene_title"] != "Goblin parley" ||
first.Metadata["primary_mode"] != "Discussion" || first.Metadata["primary_mode"] != "Discussion" ||
first.Metadata["summary"] != "The party negotiates with a scout." || first.Metadata["summary"] != "The party negotiates with a scout." ||
first.Metadata["boundary_note"] != "The scene covers the discussion before fighting starts." || first.Metadata["boundary_note"] != "The scene covers the discussion before fighting starts." ||
first.Metadata["boundary_confidence"] != "High" || first.Metadata["boundary_confidence"] != "High" ||
first.Metadata["start_unit_id"] != "seg-001" || first.Metadata["start_unit_id"] != 1 ||
first.Metadata["end_unit_id"] != "seg-002" || first.Metadata["end_unit_id"] != 2 ||
first.Metadata["unit_count"] != 2 { first.Metadata["unit_count"] != 2 {
t.Fatalf("first metadata = %#v, want scene metadata", first.Metadata) t.Fatalf("first metadata = %#v, want scene metadata", first.Metadata)
} }
@@ -311,11 +317,11 @@ func TestChunkDefensivelyCopiesSourceUnitsAndMetadata(t *testing.T) {
t.Fatalf("Chunk() error = %v, want nil", err) t.Fatalf("Chunk() error = %v, want nil", err)
} }
doc.Units[0].ID = "mutated" doc.Units[0].ID = 99
doc.Units[0].Metadata["speaker"] = "mutated" doc.Units[0].Metadata["speaker"] = "mutated"
client.response.Scenes[0].MainParticipants[0] = "mutated" client.response.Scenes[0].MainParticipants[0] = "mutated"
if result.Chunks[0].Units[0].ID != "seg-001" { if result.Chunks[0].Units[0].ID != 1 {
t.Fatalf("chunk unit ID changed after source mutation: %#v", result.Chunks[0].Units[0]) t.Fatalf("chunk unit ID changed after source mutation: %#v", result.Chunks[0].Units[0])
} }
if result.Chunks[0].Units[0].Metadata["speaker"] != "Alice" { if result.Chunks[0].Units[0].Metadata["speaker"] != "Alice" {
@@ -362,7 +368,7 @@ func TestChunkRejectsInvalidRequests(t *testing.T) {
canceledCtx, cancel := context.WithCancel(context.Background()) canceledCtx, cancel := context.WithCancel(context.Background())
cancel() cancel()
invalidDoc := sceneSourceDocument() invalidDoc := sceneSourceDocument()
invalidDoc.Units[0].ID = "" invalidDoc.Units[0].ID = 0
emptyDoc := sceneSourceDocument() emptyDoc := sceneSourceDocument()
emptyDoc.Units = nil emptyDoc.Units = nil
@@ -407,37 +413,37 @@ func TestChunkRejectsMalformedStructuredOutput(t *testing.T) {
{ {
name: "unknown boundary id", name: "unknown boundary id",
response: replaceScenes(validSceneResponse(), []sceneResponse{ response: replaceScenes(validSceneResponse(), []sceneResponse{
scene("seg-001", "seg-999"), scene(1, 999),
}), }),
want: "was not found", want: "was not found",
}, },
{ {
name: "out of order boundaries", name: "out of order boundaries",
response: replaceScenes(validSceneResponse(), []sceneResponse{ response: replaceScenes(validSceneResponse(), []sceneResponse{
scene("seg-003", "seg-002"), scene(3, 2),
}), }),
want: "appears after", want: "appears after",
}, },
{ {
name: "gap", name: "gap",
response: replaceScenes(validSceneResponse(), []sceneResponse{ response: replaceScenes(validSceneResponse(), []sceneResponse{
scene("seg-001", "seg-001"), scene(1, 1),
scene("seg-003", "seg-004"), scene(3, 4),
}), }),
want: "gap", want: "gap",
}, },
{ {
name: "overlap", name: "overlap",
response: replaceScenes(validSceneResponse(), []sceneResponse{ response: replaceScenes(validSceneResponse(), []sceneResponse{
scene("seg-001", "seg-002"), scene(1, 2),
scene("seg-002", "seg-004"), scene(2, 4),
}), }),
want: "overlap", want: "overlap",
}, },
{ {
name: "incomplete coverage", name: "incomplete coverage",
response: replaceScenes(validSceneResponse(), []sceneResponse{ response: replaceScenes(validSceneResponse(), []sceneResponse{
scene("seg-001", "seg-003"), scene(1, 3),
}), }),
want: "final scene", want: "final scene",
}, },
@@ -445,8 +451,8 @@ func TestChunkRejectsMalformedStructuredOutput(t *testing.T) {
name: "empty metadata field", name: "empty metadata field",
response: replaceScenes(validSceneResponse(), []sceneResponse{ response: replaceScenes(validSceneResponse(), []sceneResponse{
{ {
StartUnitID: dnd.UnitRefFromString("seg-001"), StartUnitID: dnd.UnitRefFromInt(1),
EndUnitID: dnd.UnitRefFromString("seg-004"), EndUnitID: dnd.UnitRefFromInt(4),
ShortTitle: " ", ShortTitle: " ",
PrimaryMode: "Narrative", PrimaryMode: "Narrative",
MainParticipants: []string{"Aria"}, MainParticipants: []string{"Aria"},
@@ -461,8 +467,8 @@ func TestChunkRejectsMalformedStructuredOutput(t *testing.T) {
name: "empty participant", name: "empty participant",
response: replaceScenes(validSceneResponse(), []sceneResponse{ response: replaceScenes(validSceneResponse(), []sceneResponse{
{ {
StartUnitID: dnd.UnitRefFromString("seg-001"), StartUnitID: dnd.UnitRefFromInt(1),
EndUnitID: dnd.UnitRefFromString("seg-004"), EndUnitID: dnd.UnitRefFromInt(4),
ShortTitle: "Title", ShortTitle: "Title",
PrimaryMode: "Narrative", PrimaryMode: "Narrative",
MainParticipants: []string{"Aria", " "}, MainParticipants: []string{"Aria", " "},
@@ -511,7 +517,7 @@ func chunkRequestWithClient(client contracts.StructuredLLMClient) contracts.Chun
} }
} }
const sceneTranscriptJSON = `{"id":"session-alpha","segments":[{"id":"seg-001","text":"Aria asks whether the goblin will parley."}]}` const sceneTranscriptJSON = `{"id":"session-alpha","segments":[{"id":1,"text":"Aria asks whether the goblin will parley."}]}`
func sceneSourceInput() contracts.LLMInputMaterial { func sceneSourceInput() contracts.LLMInputMaterial {
return contracts.NewLLMInputMaterial("source", "application/json", []byte(sceneTranscriptJSON), "sha256:transcript", "file:///session-alpha.json") return contracts.NewLLMInputMaterial("source", "application/json", []byte(sceneTranscriptJSON), "sha256:transcript", "file:///session-alpha.json")
@@ -529,10 +535,10 @@ func sceneSourceDocument() *source.SourceDocument {
Format: "application/vnd.seriatim.minimal+json", Format: "application/vnd.seriatim.minimal+json",
Digest: "sha256:source", Digest: "sha256:source",
Units: []source.SourceUnit{ Units: []source.SourceUnit{
{ID: "seg-001", Kind: "transcript_segment", Text: "Aria asks whether the goblin will parley.", Metadata: map[string]any{"speaker": "Alice"}}, {ID: 1, Kind: "transcript_segment", Text: "Aria asks whether the goblin will parley.", Metadata: map[string]any{"speaker": "Alice"}},
{ID: "seg-002", Kind: "transcript_segment", Text: "The goblin scout describes the gate guards."}, {ID: 2, Kind: "transcript_segment", Text: "The goblin scout describes the gate guards."},
{ID: "seg-003", Kind: "transcript_segment", Text: "The guards rush out with blades drawn."}, {ID: 3, Kind: "transcript_segment", Text: "The guards rush out with blades drawn."},
{ID: "seg-004", Kind: "transcript_segment", Text: "The party defeats the ambushers."}, {ID: 4, Kind: "transcript_segment", Text: "The party defeats the ambushers."},
}, },
} }
} }
@@ -540,7 +546,7 @@ func sceneSourceDocument() *source.SourceDocument {
func validSceneResponse() chunkResponse { func validSceneResponse() chunkResponse {
return chunkResponse{ return chunkResponse{
Scenes: []sceneResponse{ Scenes: []sceneResponse{
scene("seg-001", "seg-004"), scene(1, 4),
}, },
BoundaryCaveats: []string{}, BoundaryCaveats: []string{},
} }
@@ -551,10 +557,10 @@ func replaceScenes(response chunkResponse, scenes []sceneResponse) chunkResponse
return response return response
} }
func scene(startUnitID string, endUnitID string) sceneResponse { func scene(startUnitID int, endUnitID int) sceneResponse {
return sceneResponse{ return sceneResponse{
StartUnitID: dnd.UnitRefFromString(startUnitID), StartUnitID: dnd.UnitRefFromInt(startUnitID),
EndUnitID: dnd.UnitRefFromString(endUnitID), EndUnitID: dnd.UnitRefFromInt(endUnitID),
ShortTitle: "Scene title", ShortTitle: "Scene title",
PrimaryMode: "Narrative", PrimaryMode: "Narrative",
MainParticipants: []string{"Aria"}, MainParticipants: []string{"Aria"},
@@ -572,8 +578,8 @@ func chunkIDs(chunks []contracts.SourceChunk) []string {
return ids return ids
} }
func unitIDs(units []source.SourceUnit) []string { func unitIDs(units []source.SourceUnit) []int {
ids := make([]string, 0, len(units)) ids := make([]int, 0, len(units))
for _, unit := range units { for _, unit := range units {
ids = append(ids, unit.ID) ids = append(ids, unit.ID)
} }

View File

@@ -19,8 +19,8 @@ type sceneResponse struct {
} }
type normalizedScene struct { type normalizedScene struct {
StartUnitID string StartUnitID int
EndUnitID string EndUnitID int
ShortTitle string ShortTitle string
PrimaryMode string PrimaryMode string
MainParticipants []string MainParticipants []string

View File

@@ -68,11 +68,19 @@ func (c *Chunker) Chunk(ctx context.Context, req contracts.ChunkRequest) (contra
end = len(req.Source.Units) end = len(req.Source.Units)
} }
units := cloneUnits(req.Source.Units[start:end]) units := cloneUnits(req.Source.Units[start:end])
content, err := chunkContent(units)
if err != nil {
return contracts.ChunkResult{}, err
}
chunks = append(chunks, contracts.SourceChunk{ chunks = append(chunks, contracts.SourceChunk{
ID: fmt.Sprintf("chunk-%06d", len(chunks)+1), ID: fmt.Sprintf("chunk-%06d", len(chunks)+1),
SourceID: req.Source.ID, SourceID: req.Source.ID,
Index: len(chunks), Index: len(chunks),
Units: units, StartUnitID: units[0].ID,
EndUnitID: units[len(units)-1].ID,
Content: content,
MediaType: "application/json",
Units: units,
Metadata: map[string]any{ Metadata: map[string]any{
"start_unit_id": units[0].ID, "start_unit_id": units[0].ID,
"end_unit_id": units[len(units)-1].ID, "end_unit_id": units[len(units)-1].ID,
@@ -87,6 +95,18 @@ func (c *Chunker) Chunk(ctx context.Context, req contracts.ChunkRequest) (contra
return contracts.ChunkResult{Chunks: chunks}, nil return contracts.ChunkResult{Chunks: chunks}, nil
} }
func chunkContent(units []source.SourceUnit) ([]byte, error) {
content, err := json.Marshal(struct {
Units []source.SourceUnit `json:"units"`
}{
Units: units,
})
if err != nil {
return nil, chunkerErrorf("encode chunk content: %w", err)
}
return content, nil
}
func ModuleSpec() pipeline.ModuleSpec { func ModuleSpec() pipeline.ModuleSpec {
return pipeline.ModuleSpec{ return pipeline.ModuleSpec{
Key: Key, Key: Key,

View File

@@ -59,10 +59,16 @@ func TestChunkUsesDefaultsForSingleChunk(t *testing.T) {
if chunk.Index != 0 || chunk.SourceID != "source-1" { if chunk.Index != 0 || chunk.SourceID != "source-1" {
t.Fatalf("chunk = %#v, want source and index fields", chunk) t.Fatalf("chunk = %#v, want source and index fields", chunk)
} }
if got := unitIDs(chunk.Units); !reflect.DeepEqual(got, []string{"u001", "u002", "u003"}) { if got := unitIDs(chunk.Units); !reflect.DeepEqual(got, []int{1, 2, 3}) {
t.Fatalf("unit IDs = %#v, want all units", got) t.Fatalf("unit IDs = %#v, want all units", got)
} }
if chunk.Metadata["start_unit_id"] != "u001" || chunk.Metadata["end_unit_id"] != "u003" || chunk.Metadata["unit_count"] != 3 { if chunk.StartUnitID != 1 || chunk.EndUnitID != 3 {
t.Fatalf("chunk boundaries = %d-%d, want 1-3", chunk.StartUnitID, chunk.EndUnitID)
}
if chunk.MediaType != "application/json" || len(chunk.Content) == 0 {
t.Fatalf("chunk payload = media type %q length %d, want JSON content", chunk.MediaType, len(chunk.Content))
}
if chunk.Metadata["start_unit_id"] != 1 || chunk.Metadata["end_unit_id"] != 3 || chunk.Metadata["unit_count"] != 3 {
t.Fatalf("metadata = %#v, want chunk bounds", chunk.Metadata) t.Fatalf("metadata = %#v, want chunk bounds", chunk.Metadata)
} }
} }
@@ -79,8 +85,8 @@ func TestChunkExactBoundaries(t *testing.T) {
if got := chunkIDs(result.Chunks); !reflect.DeepEqual(got, []string{"chunk-000001", "chunk-000002", "chunk-000003"}) { if got := chunkIDs(result.Chunks); !reflect.DeepEqual(got, []string{"chunk-000001", "chunk-000002", "chunk-000003"}) {
t.Fatalf("chunk IDs = %#v, want stable IDs", got) t.Fatalf("chunk IDs = %#v, want stable IDs", got)
} }
gotUnits := [][]string{unitIDs(result.Chunks[0].Units), unitIDs(result.Chunks[1].Units), unitIDs(result.Chunks[2].Units)} gotUnits := [][]int{unitIDs(result.Chunks[0].Units), unitIDs(result.Chunks[1].Units), unitIDs(result.Chunks[2].Units)}
wantUnits := [][]string{{"u001", "u002"}, {"u003", "u004"}, {"u005", "u006"}} wantUnits := [][]int{{1, 2}, {3, 4}, {5, 6}}
if !reflect.DeepEqual(gotUnits, wantUnits) { if !reflect.DeepEqual(gotUnits, wantUnits) {
t.Fatalf("chunk units = %#v, want %#v", gotUnits, wantUnits) t.Fatalf("chunk units = %#v, want %#v", gotUnits, wantUnits)
} }
@@ -95,11 +101,11 @@ func TestChunkOverlap(t *testing.T) {
t.Fatalf("Chunk() error = %v, want nil", err) t.Fatalf("Chunk() error = %v, want nil", err)
} }
gotUnits := make([][]string, 0, len(result.Chunks)) gotUnits := make([][]int, 0, len(result.Chunks))
for _, chunk := range result.Chunks { for _, chunk := range result.Chunks {
gotUnits = append(gotUnits, unitIDs(chunk.Units)) gotUnits = append(gotUnits, unitIDs(chunk.Units))
} }
wantUnits := [][]string{{"u001", "u002", "u003"}, {"u003", "u004", "u005"}, {"u005", "u006", "u007"}} wantUnits := [][]int{{1, 2, 3}, {3, 4, 5}, {5, 6, 7}}
if !reflect.DeepEqual(gotUnits, wantUnits) { if !reflect.DeepEqual(gotUnits, wantUnits) {
t.Fatalf("chunk units = %#v, want %#v", gotUnits, wantUnits) t.Fatalf("chunk units = %#v, want %#v", gotUnits, wantUnits)
} }
@@ -162,10 +168,10 @@ func TestChunkDefensivelyCopiesUnits(t *testing.T) {
t.Fatalf("len(Chunks) = %d, want 2", len(result.Chunks)) t.Fatalf("len(Chunks) = %d, want 2", len(result.Chunks))
} }
doc.Units[0].ID = "changed" doc.Units[0].ID = 99
doc.Units[0].Metadata["speaker"] = "changed" doc.Units[0].Metadata["speaker"] = "changed"
if result.Chunks[0].Units[0].ID != "u001" { if result.Chunks[0].Units[0].ID != 1 {
t.Fatalf("chunk unit ID changed after source mutation: %#v", result.Chunks[0].Units[0]) t.Fatalf("chunk unit ID changed after source mutation: %#v", result.Chunks[0].Units[0])
} }
if result.Chunks[0].Units[0].Metadata["speaker"] != "speaker-001" { if result.Chunks[0].Units[0].Metadata["speaker"] != "speaker-001" {
@@ -176,11 +182,10 @@ func TestChunkDefensivelyCopiesUnits(t *testing.T) {
func testSource(count int) *source.SourceDocument { func testSource(count int) *source.SourceDocument {
units := make([]source.SourceUnit, 0, count) units := make([]source.SourceUnit, 0, count)
for i := 1; i <= count; i++ { for i := 1; i <= count; i++ {
id := "u" + zeroPad3(i)
units = append(units, source.SourceUnit{ units = append(units, source.SourceUnit{
ID: id, ID: i,
Kind: "unit", Kind: "unit",
Text: "Text for " + id, Text: "Text for " + zeroPad3(i),
Metadata: map[string]any{ Metadata: map[string]any{
"speaker": "speaker-" + zeroPad3(i), "speaker": "speaker-" + zeroPad3(i),
}, },
@@ -207,8 +212,8 @@ func chunkIDs(chunks []contracts.SourceChunk) []string {
return ids return ids
} }
func unitIDs(units []source.SourceUnit) []string { func unitIDs(units []source.SourceUnit) []int {
ids := make([]string, 0, len(units)) ids := make([]int, 0, len(units))
for _, unit := range units { for _, unit := range units {
ids = append(ids, unit.ID) ids = append(ids, unit.ID)
} }

View File

@@ -1,5 +1,4 @@
Source references must use 1-based integer source-unit numbers from the Source references must use integer source-unit IDs from the transcript.
transcript, where 1 is the first provided source unit.
Return only D&D spell-cast artifacts. For each spell cast, identify the in-world Return only D&D spell-cast artifacts. For each spell cast, identify the in-world
caster, spell name, effect, narrative description, and source references using caster, spell name, effect, narrative description, and source references using

View File

@@ -244,10 +244,14 @@ func (dndSpellsChunker) Chunk(ctx context.Context, req contracts.ChunkRequest) (
return contracts.ChunkResult{ return contracts.ChunkResult{
Chunks: []contracts.SourceChunk{ Chunks: []contracts.SourceChunk{
{ {
ID: req.Source.ID + ":chunk:0", ID: req.Source.ID + ":chunk:0",
SourceID: req.Source.ID, SourceID: req.Source.ID,
Index: 0, Index: 0,
Units: append([]source.SourceUnit(nil), req.Source.Units...), StartUnitID: req.Source.Units[0].ID,
EndUnitID: req.Source.Units[len(req.Source.Units)-1].ID,
Content: []byte(`{"units":[1,2,3]}`),
MediaType: "application/json",
Units: append([]source.SourceUnit(nil), req.Source.Units...),
}, },
}, },
}, nil }, nil

View File

@@ -6,8 +6,6 @@ import (
"fmt" "fmt"
"strings" "strings"
"gitea.maximumdirect.net/eric/notarius/internal/core/artifacts"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts" "gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline" "gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
"gitea.maximumdirect.net/eric/notarius/internal/modules/sharedassets/dnd" "gitea.maximumdirect.net/eric/notarius/internal/modules/sharedassets/dnd"
@@ -45,14 +43,6 @@ func (e *Extractor) Key() string {
return Key return Key
} }
func (e *Extractor) ArtifactType() string {
return ArtifactType
}
func (e *Extractor) SchemaVersion() string {
return SchemaVersion
}
func (e *Extractor) ReferenceSlots() []contracts.ReferenceSlot { func (e *Extractor) ReferenceSlots() []contracts.ReferenceSlot {
return dnd.ReferenceSlots(referenceSlotDescriptions) return dnd.ReferenceSlots(referenceSlotDescriptions)
} }
@@ -77,13 +67,6 @@ func (e *Extractor) ManifestMetadata() map[string]any {
return metadata return metadata
} }
func (e *Extractor) Validators() []contracts.Validator {
return []contracts.Validator{
ShapeValidator{},
SourceRefValidator{},
}
}
func (e *Extractor) Extract(ctx context.Context, req contracts.ExtractionRequest) (contracts.ExtractionResult, error) { func (e *Extractor) Extract(ctx context.Context, req contracts.ExtractionRequest) (contracts.ExtractionResult, error) {
if e == nil { if e == nil {
return contracts.ExtractionResult{}, extractorErrorf("extractor must not be nil") return contracts.ExtractionResult{}, extractorErrorf("extractor must not be nil")
@@ -108,35 +91,41 @@ func (e *Extractor) Extract(ctx context.Context, req contracts.ExtractionRequest
} }
var response extractionResponse var response extractionResponse
if _, err := req.LLMClient.CompleteStructured(ctx, contracts.StructuredCompletionRequest{ completion, err := req.LLMClient.CompleteStructured(ctx, contracts.StructuredCompletionRequest{
StageName: Key, StageName: Key,
PromptID: PromptID, PromptID: PromptID,
PromptVersion: SchemaVersion, PromptVersion: SchemaVersion,
ProfileID: req.LLMProfile, ProfileID: req.LLMProfile,
SessionID: req.SessionID, SessionID: req.SessionID,
Inputs: dnd.PromptInputs(req.SourceInput, req.References), Inputs: dnd.PromptInputs(req.SourceInput, req.References),
}, &response); err != nil { }, &response)
if err != nil {
return contracts.ExtractionResult{}, extractorErrorf("complete structured output: %w", err) return contracts.ExtractionResult{}, extractorErrorf("complete structured output: %w", err)
} }
if response.SpellCasts == nil { content := append([]byte(nil), completion.Content...)
return contracts.ExtractionResult{}, extractorErrorf("malformed structured output: spell_casts must be present") if len(strings.TrimSpace(string(content))) == 0 {
} var err error
if len(response.SpellCasts) == 0 { content, err = json.Marshal(response)
return contracts.ExtractionResult{}, nil
}
candidates := make([]artifacts.ArtifactCandidate, 0, len(response.SpellCasts))
for i, spellCast := range response.SpellCasts {
payload, err := spellCastPayload(spellCast)
if err != nil { if err != nil {
return contracts.ExtractionResult{}, extractorErrorf("marshal spell cast[%d]: %w", i, err) return contracts.ExtractionResult{}, extractorErrorf("marshal raw output: %w", err)
} }
candidates = append(candidates, artifacts.ArtifactCandidate{
Payload: payload,
SourceRefs: sourceRefCandidates(req.Source, spellCast.SourceRefs),
})
} }
return contracts.ExtractionResult{Candidates: candidates}, nil return contracts.ExtractionResult{
Output: contracts.ExtractOutput{
Schema: contracts.ResponseSchema{
ID: ResponseSchemaID,
Name: ResponseSchemaName,
Version: SchemaVersion,
},
Payload: contracts.RawPayload{
Content: content,
MediaType: "application/json",
Metadata: map[string]any{
"spell_cast_count": len(response.SpellCasts),
},
},
},
}, nil
} }
func ModuleSpec() pipeline.ModuleSpec { func ModuleSpec() pipeline.ModuleSpec {
@@ -155,26 +144,6 @@ func Register(registry *pipeline.ExtractorRegistry) error {
}) })
} }
func spellCastPayload(spellCast spellCastResponse) (json.RawMessage, error) {
return json.Marshal(SpellCast{
Caster: strings.TrimSpace(spellCast.Caster),
Spell: strings.TrimSpace(spellCast.Spell),
Effect: strings.TrimSpace(spellCast.Effect),
NarrativeDescription: strings.TrimSpace(spellCast.NarrativeDescription),
})
}
func sourceRefCandidates(doc *source.SourceDocument, refs []dnd.SourceRefResponse) []source.SourceRef {
if len(refs) == 0 {
return nil
}
out := make([]source.SourceRef, 0, len(refs))
for _, ref := range refs {
out = append(out, dnd.SourceRefCandidate(doc, ref))
}
return out
}
func extractorErrorf(format string, args ...any) error { func extractorErrorf(format string, args ...any) error {
return fmt.Errorf("dnd spells extractor: "+format, args...) return fmt.Errorf("dnd spells extractor: "+format, args...)
} }

View File

@@ -7,12 +7,11 @@ import (
"strings" "strings"
"testing" "testing"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts" "gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/modules/sharedassets/dnd" "gitea.maximumdirect.net/eric/notarius/internal/modules/sharedassets/dnd"
) )
func TestExtractReturnsSpellCandidateFromStructuredOutput(t *testing.T) { func TestExtractReturnsRawOutputFromStructuredResponse(t *testing.T) {
client := &fakeSpellsLLMClient{ client := &fakeSpellsLLMClient{
response: extractionResponse{ response: extractionResponse{
SpellCasts: []spellCastResponse{ SpellCasts: []spellCastResponse{
@@ -25,6 +24,7 @@ func TestExtractReturnsSpellCandidateFromStructuredOutput(t *testing.T) {
}, },
}, },
}, },
content: []byte(`{"spell_casts":[{"caster":" Aria ","spell":" Cure Wounds ","effect":" Heals an injured ally. ","narrative_description":" Aria restores the fighter after the fight. ","source_refs":[{"source_id":"session-alpha","start_unit_id":1,"end_unit_id":2}]}],"raw_marker":true}`),
} }
result, err := New().Extract(context.Background(), extractionRequestWithClient(client)) result, err := New().Extract(context.Background(), extractionRequestWithClient(client))
@@ -53,29 +53,22 @@ func TestExtractReturnsSpellCandidateFromStructuredOutput(t *testing.T) {
t.Fatalf("transcript content = %q, want original source input", got) t.Fatalf("transcript content = %q, want original source input", got)
} }
if len(result.Candidates) != 1 { if result.Output.Payload.MediaType != "application/json" {
t.Fatalf("len(Candidates) = %d, want 1", len(result.Candidates)) t.Fatalf("MediaType = %q, want application/json", result.Output.Payload.MediaType)
} }
candidate := result.Candidates[0] if result.Output.Schema.ID != ResponseSchemaID || result.Output.Schema.Name != ResponseSchemaName || result.Output.Schema.Version != SchemaVersion {
if candidate.Index != 0 || candidate.ExtractorKey != "" || candidate.ArtifactType != "" || candidate.SchemaVersion != "" { t.Fatalf("schema = %#v, want response schema provenance", result.Output.Schema)
t.Fatalf("candidate envelope fields = %#v, want runner-normalized zero values", candidate)
} }
var payload SpellCast if got := string(result.Output.Payload.Content); got != string(client.content) {
if err := json.Unmarshal(candidate.Payload, &payload); err != nil { t.Fatalf("content = %q, want exact raw completion content", got)
t.Fatalf("Unmarshal(Payload) error = %v, want nil", err)
} }
wantPayload := SpellCast{
Caster: "Aria", var payload extractionResponse
Spell: "Cure Wounds", if err := json.Unmarshal(result.Output.Payload.Content, &payload); err != nil {
Effect: "Heals an injured ally.", t.Fatalf("Unmarshal(Content) error = %v, want nil", err)
NarrativeDescription: "Aria restores the fighter after the fight.",
} }
if payload != wantPayload { if len(payload.SpellCasts) != 1 || payload.SpellCasts[0].Spell != " Cure Wounds " {
t.Fatalf("payload = %#v, want %#v", payload, wantPayload) t.Fatalf("payload = %#v, want raw structured response", payload)
}
wantRef := source.SourceRef{SourceID: "session-alpha", StartUnitID: "seg-001", EndUnitID: "seg-002"}
if len(candidate.SourceRefs) != 1 || candidate.SourceRefs[0] != wantRef {
t.Fatalf("SourceRefs = %#v, want %#v", candidate.SourceRefs, []source.SourceRef{wantRef})
} }
} }
@@ -174,27 +167,31 @@ func TestPromptInputsMapLegacyRosterReferenceToParty(t *testing.T) {
} }
} }
func TestExtractReturnsNoCandidatesForEmptyResponse(t *testing.T) { func TestExtractReturnsRawOutputForEmptyResponse(t *testing.T) {
client := &fakeSpellsLLMClient{response: extractionResponse{SpellCasts: []spellCastResponse{}}} client := &fakeSpellsLLMClient{response: extractionResponse{SpellCasts: []spellCastResponse{}}}
result, err := New().Extract(context.Background(), extractionRequestWithClient(client)) result, err := New().Extract(context.Background(), extractionRequestWithClient(client))
if err != nil { if err != nil {
t.Fatalf("Extract() error = %v, want nil", err) t.Fatalf("Extract() error = %v, want nil", err)
} }
if len(result.Candidates) != 0 { var payload extractionResponse
t.Fatalf("Candidates = %#v, want none", result.Candidates) if err := json.Unmarshal(result.Output.Payload.Content, &payload); err != nil {
t.Fatalf("Unmarshal(Content) error = %v, want nil", err)
}
if len(payload.SpellCasts) != 0 {
t.Fatalf("SpellCasts = %#v, want none", payload.SpellCasts)
} }
} }
func TestExtractRejectsMissingSpellCasts(t *testing.T) { func TestExtractCarriesMalformedStructuredContentAsRawOutput(t *testing.T) {
client := &fakeSpellsLLMClient{response: extractionResponse{}} client := &fakeSpellsLLMClient{response: extractionResponse{}}
_, err := New().Extract(context.Background(), extractionRequestWithClient(client)) result, err := New().Extract(context.Background(), extractionRequestWithClient(client))
if err == nil { if err != nil {
t.Fatal("Extract() error = nil, want malformed output error") t.Fatalf("Extract() error = %v, want nil", err)
} }
if !strings.Contains(err.Error(), "dnd spells") || !strings.Contains(err.Error(), "spell_casts") { if string(result.Output.Payload.Content) != `{"spell_casts":null}` {
t.Fatalf("Extract() error = %q, want spell_casts context", err.Error()) t.Fatalf("content = %s, want raw structured output", result.Output.Payload.Content)
} }
} }
@@ -254,14 +251,14 @@ func TestExtractPreservesResponseOrder(t *testing.T) {
Spell: "Cure Wounds", Spell: "Cure Wounds",
Effect: "Heals.", Effect: "Heals.",
NarrativeDescription: "First spell.", NarrativeDescription: "First spell.",
SourceRefs: responseSourceRefs("session-alpha", "seg-001", "seg-001"), SourceRefs: responseSourceRefs("session-alpha", 1, 1),
}, },
{ {
Caster: "Bandit Shaman", Caster: "Bandit Shaman",
Spell: "Fire Bolt", Spell: "Fire Bolt",
Effect: "Burns.", Effect: "Burns.",
NarrativeDescription: "Second spell.", NarrativeDescription: "Second spell.",
SourceRefs: responseSourceRefs("session-alpha", "seg-002", "seg-002"), SourceRefs: responseSourceRefs("session-alpha", 2, 2),
}, },
}, },
}, },
@@ -271,23 +268,16 @@ func TestExtractPreservesResponseOrder(t *testing.T) {
if err != nil { if err != nil {
t.Fatalf("Extract() error = %v, want nil", err) t.Fatalf("Extract() error = %v, want nil", err)
} }
if len(result.Candidates) != 2 { var payload extractionResponse
t.Fatalf("len(Candidates) = %d, want 2", len(result.Candidates)) if err := json.Unmarshal(result.Output.Payload.Content, &payload); err != nil {
t.Fatalf("Unmarshal(Content) error = %v, want nil", err)
} }
if len(payload.SpellCasts) != 2 || payload.SpellCasts[0].Spell != "Cure Wounds" || payload.SpellCasts[1].Spell != "Fire Bolt" {
var first, second SpellCast t.Fatalf("spell order = %#v, want response order", payload.SpellCasts)
if err := json.Unmarshal(result.Candidates[0].Payload, &first); err != nil {
t.Fatalf("Unmarshal(first) error = %v, want nil", err)
}
if err := json.Unmarshal(result.Candidates[1].Payload, &second); err != nil {
t.Fatalf("Unmarshal(second) error = %v, want nil", err)
}
if first.Spell != "Cure Wounds" || second.Spell != "Fire Bolt" {
t.Fatalf("candidate order = %q, %q; want response order", first.Spell, second.Spell)
} }
} }
func TestExtractCopiesCandidateSourceRefs(t *testing.T) { func TestExtractDefensivelyCopiesRawContent(t *testing.T) {
client := &fakeSpellsLLMClient{ client := &fakeSpellsLLMClient{
response: extractionResponse{ response: extractionResponse{
SpellCasts: []spellCastResponse{ SpellCasts: []spellCastResponse{
@@ -296,7 +286,7 @@ func TestExtractCopiesCandidateSourceRefs(t *testing.T) {
Spell: "Cure Wounds", Spell: "Cure Wounds",
Effect: "Heals.", Effect: "Heals.",
NarrativeDescription: "Aria heals.", NarrativeDescription: "Aria heals.",
SourceRefs: responseSourceRefs("session-alpha", "seg-001", "seg-002"), SourceRefs: responseSourceRefs("session-alpha", 1, 2),
}, },
}, },
}, },
@@ -306,10 +296,14 @@ func TestExtractCopiesCandidateSourceRefs(t *testing.T) {
if err != nil { if err != nil {
t.Fatalf("Extract() error = %v, want nil", err) t.Fatalf("Extract() error = %v, want nil", err)
} }
client.response.SpellCasts[0].SourceRefs[0].StartUnitID = dnd.UnitRefFromString("mutated") client.response.SpellCasts[0].SourceRefs[0].StartUnitID = dnd.UnitRefFromInt(99)
if got := result.Candidates[0].SourceRefs[0].StartUnitID; got != "seg-001" { var payload extractionResponse
t.Fatalf("candidate source ref start = %q, want copied seg-001", got) if err := json.Unmarshal(result.Output.Payload.Content, &payload); err != nil {
t.Fatalf("Unmarshal(Content) error = %v, want nil", err)
}
if got := payload.SpellCasts[0].SourceRefs[0].StartUnitID.String(); got != "1" {
t.Fatalf("source ref start = %q, want copied 1", got)
} }
} }
@@ -322,7 +316,7 @@ func extractionRequestWithClient(client contracts.StructuredLLMClient) contracts
return req return req
} }
const spellTranscriptJSON = `{"id":"session-alpha","segments":[{"id":"seg-001","text":"Aria raises her hand and casts Cure Wounds."}]}` const spellTranscriptJSON = `{"id":"session-alpha","segments":[{"id":1,"text":"Aria raises her hand and casts Cure Wounds."}]}`
func spellSourceInput() contracts.LLMInputMaterial { func spellSourceInput() contracts.LLMInputMaterial {
return contracts.NewLLMInputMaterial("source", "application/json", []byte(spellTranscriptJSON), "sha256:transcript", "file:///session-alpha.json") return contracts.NewLLMInputMaterial("source", "application/json", []byte(spellTranscriptJSON), "sha256:transcript", "file:///session-alpha.json")
@@ -339,6 +333,7 @@ func emptyChunkRequest(req contracts.ExtractionRequest) contracts.ExtractionRequ
type fakeSpellsLLMClient struct { type fakeSpellsLLMClient struct {
response extractionResponse response extractionResponse
content []byte
err error err error
requests []contracts.StructuredCompletionRequest requests []contracts.StructuredCompletionRequest
} }
@@ -354,9 +349,13 @@ func (client *fakeSpellsLLMClient) CompleteStructured(ctx context.Context, req c
return contracts.StructuredCompletionResponse{}, errors.New("unexpected output target") return contracts.StructuredCompletionResponse{}, errors.New("unexpected output target")
} }
*target = client.response *target = client.response
content, err := json.Marshal(client.response) content := append([]byte(nil), client.content...)
if err != nil { if len(content) == 0 {
return contracts.StructuredCompletionResponse{}, err var err error
content, err = json.Marshal(client.response)
if err != nil {
return contracts.StructuredCompletionResponse{}, err
}
} }
return contracts.StructuredCompletionResponse{Content: content}, nil return contracts.StructuredCompletionResponse{Content: content}, nil
} }

View File

@@ -17,12 +17,6 @@ func TestNewReturnsExtractorWithMetadata(t *testing.T) {
if extractor.Key() != Key { if extractor.Key() != Key {
t.Fatalf("extractor.Key() = %q, want %q", extractor.Key(), Key) t.Fatalf("extractor.Key() = %q, want %q", extractor.Key(), Key)
} }
if extractor.ArtifactType() != ArtifactType {
t.Fatalf("extractor.ArtifactType() = %q, want %q", extractor.ArtifactType(), ArtifactType)
}
if extractor.SchemaVersion() != SchemaVersion {
t.Fatalf("extractor.SchemaVersion() = %q, want %q", extractor.SchemaVersion(), SchemaVersion)
}
} }
func TestModuleSpec(t *testing.T) { func TestModuleSpec(t *testing.T) {

View File

@@ -26,14 +26,14 @@ func TestRunnerProcessesSeriatimInputWithDNDSpellsExtractor(t *testing.T) {
Spell: "Cure Wounds", Spell: "Cure Wounds",
Effect: "Heals an injured ally.", Effect: "Heals an injured ally.",
NarrativeDescription: "Aria restores the fighter after the fight.", NarrativeDescription: "Aria restores the fighter after the fight.",
SourceRefs: responseSourceRefs(expectedDoc.ID, "seg-001", "seg-001"), SourceRefs: responseSourceRefs(expectedDoc.ID, 1, 1),
}, },
{ {
Caster: "Borin", Caster: "Borin",
Spell: "Fire Bolt", Spell: "Fire Bolt",
Effect: "Scorches the wight.", Effect: "Scorches the wight.",
NarrativeDescription: "Borin hurls fire at the wight.", NarrativeDescription: "Borin hurls fire at the wight.",
SourceRefs: responseSourceRefs(expectedDoc.ID, "seg-003", "seg-003"), SourceRefs: responseSourceRefs(expectedDoc.ID, 3, 3),
}, },
}, },
}, },
@@ -48,31 +48,30 @@ func TestRunnerProcessesSeriatimInputWithDNDSpellsExtractor(t *testing.T) {
t.Fatalf("Run() error = %v, want nil", err) t.Fatalf("Run() error = %v, want nil", err)
} }
if len(output.Approved) != 2 { if len(output.NormalizeOutputs) != 1 {
t.Fatalf("len(Approved) = %d, want 2", len(output.Approved)) t.Fatalf("len(NormalizeOutputs) = %d, want 1", len(output.NormalizeOutputs))
} }
var first, second SpellCast rawOutput := output.NormalizeOutputs[0]
if err := json.Unmarshal(output.Approved[0].Payload, &first); err != nil { if rawOutput.LaneID != "spells" || rawOutput.Schema.ID != ResponseSchemaID || rawOutput.Schema.Version != SchemaVersion {
t.Fatalf("Unmarshal(first payload) error = %v, want nil", err) t.Fatalf("raw output envelope = %#v, want dnd spells schema on spells lane", rawOutput)
} }
if err := json.Unmarshal(output.Approved[1].Payload, &second); err != nil { response := decodeRunnerSpellResponse(t, rawOutput.Payload.Content)
t.Fatalf("Unmarshal(second payload) error = %v, want nil", err) if len(response.SpellCasts) != 2 {
t.Fatalf("len(spell_casts) = %d, want 2", len(response.SpellCasts))
} }
first, second := response.SpellCasts[0], response.SpellCasts[1]
if first.Spell != "Cure Wounds" || second.Spell != "Fire Bolt" { if first.Spell != "Cure Wounds" || second.Spell != "Fire Bolt" {
t.Fatalf("approved spell order = %q, %q; want response order", first.Spell, second.Spell) t.Fatalf("spell order = %q, %q; want response order", first.Spell, second.Spell)
} }
if first.Caster != "Aria" || second.Caster != "Borin" { if first.Caster != "Aria" || second.Caster != "Borin" {
t.Fatalf("approved casters = %q, %q; want spell data", first.Caster, second.Caster) t.Fatalf("casters = %q, %q; want spell data", first.Caster, second.Caster)
} }
for _, artifact := range output.Approved { for _, spell := range response.SpellCasts {
if artifact.ExtractorKey != Key || artifact.ArtifactType != ArtifactType || artifact.SchemaVersion != SchemaVersion { if len(spell.SourceRefs) != 1 {
t.Fatalf("approved artifact envelope = %#v, want dnd spells envelope", artifact) t.Fatalf("len(SourceRefs) = %d, want 1", len(spell.SourceRefs))
} }
if len(artifact.SourceRefs) != 1 { if spell.SourceRefs[0].SourceID != expectedDoc.ID {
t.Fatalf("len(SourceRefs) = %d, want 1", len(artifact.SourceRefs)) t.Fatalf("SourceID = %q, want fixture document ID", spell.SourceRefs[0].SourceID)
}
if err := source.ValidateRef(expectedDoc, artifact.SourceRefs[0]); err != nil {
t.Fatalf("ValidateRef() error = %v, want nil", err)
} }
} }
@@ -122,7 +121,7 @@ func TestRunnerPassesPartyAndGlossaryReferencesToDNDSpellsPrompt(t *testing.T) {
Spell: "Fire Bolt", Spell: "Fire Bolt",
Effect: "Scorches the wight.", Effect: "Scorches the wight.",
NarrativeDescription: "Borin hurls fire at the wight.", NarrativeDescription: "Borin hurls fire at the wight.",
SourceRefs: responseSourceRefs(expectedDoc.ID, "seg-003", "seg-003"), SourceRefs: responseSourceRefs(expectedDoc.ID, 3, 3),
}, },
}, },
}, },
@@ -137,8 +136,8 @@ func TestRunnerPassesPartyAndGlossaryReferencesToDNDSpellsPrompt(t *testing.T) {
t.Fatalf("Run() error = %v, want nil", err) t.Fatalf("Run() error = %v, want nil", err)
} }
if len(output.Approved) != 1 { if len(output.NormalizeOutputs) != 1 {
t.Fatalf("len(Approved) = %d, want 1", len(output.Approved)) t.Fatalf("len(NormalizeOutputs) = %d, want 1", len(output.NormalizeOutputs))
} }
if len(output.Manifest.References) != 2 { if len(output.Manifest.References) != 2 {
t.Fatalf("manifest references = %#v, want party and glossary provenance", output.Manifest.References) t.Fatalf("manifest references = %#v, want party and glossary provenance", output.Manifest.References)
@@ -178,8 +177,12 @@ func TestRunnerDoesNotExtractSpellMentionedOnlyInPartyReference(t *testing.T) {
t.Fatalf("Run() error = %v, want nil", err) t.Fatalf("Run() error = %v, want nil", err)
} }
if len(output.Approved) != 0 { if len(output.NormalizeOutputs) != 1 {
t.Fatalf("approved artifacts = %#v, want no party-reference-only spell casts", output.Approved) t.Fatalf("len(NormalizeOutputs) = %d, want empty spell response output", len(output.NormalizeOutputs))
}
response := decodeRunnerSpellResponse(t, output.NormalizeOutputs[0].Payload.Content)
if len(response.SpellCasts) != 0 {
t.Fatalf("spell_casts = %#v, want no party-reference-only spell casts", response.SpellCasts)
} }
if len(llmClient.requests) != 1 { if len(llmClient.requests) != 1 {
t.Fatalf("LLM calls = %d, want 1", len(llmClient.requests)) t.Fatalf("LLM calls = %d, want 1", len(llmClient.requests))
@@ -196,7 +199,7 @@ func TestRunnerDoesNotExtractSpellMentionedOnlyInPartyReference(t *testing.T) {
} }
} }
func TestRunnerRejectsDNDSpellCastWithInvalidSourceRef(t *testing.T) { func TestRunnerCarriesDNDSpellCastWithInvalidSourceRefAsRawOutput(t *testing.T) {
raw := readDNDSpellsFixture(t) raw := readDNDSpellsFixture(t)
resolved := resolveDNDSpellsPipeline(t) resolved := resolveDNDSpellsPipeline(t)
llmClient := &fakeSpellsLLMClient{ llmClient := &fakeSpellsLLMClient{
@@ -207,7 +210,7 @@ func TestRunnerRejectsDNDSpellCastWithInvalidSourceRef(t *testing.T) {
Spell: "Cure Wounds", Spell: "Cure Wounds",
Effect: "Heals an injured ally.", Effect: "Heals an injured ally.",
NarrativeDescription: "Aria restores the fighter after the fight.", NarrativeDescription: "Aria restores the fighter after the fight.",
SourceRefs: responseSourceRefs("spell-session", "seg-999", "seg-999"), SourceRefs: responseSourceRefs("spell-session", 999, 999),
}, },
}, },
}, },
@@ -221,21 +224,21 @@ func TestRunnerRejectsDNDSpellCastWithInvalidSourceRef(t *testing.T) {
if err != nil { if err != nil {
t.Fatalf("Run() error = %v, want nil", err) t.Fatalf("Run() error = %v, want nil", err)
} }
if len(output.Approved) != 0 { if len(output.NormalizeOutputs) != 1 {
t.Fatalf("len(Approved) = %d, want 0", len(output.Approved)) t.Fatalf("len(NormalizeOutputs) = %d, want 1", len(output.NormalizeOutputs))
} }
if len(output.Rejected) != 1 { response := decodeRunnerSpellResponse(t, output.NormalizeOutputs[0].Payload.Content)
t.Fatalf("len(Rejected) = %d, want 1", len(output.Rejected)) if len(response.SpellCasts) != 1 {
t.Fatalf("len(spell_casts) = %d, want 1", len(response.SpellCasts))
} }
rejected := output.Rejected[0] if response.SpellCasts[0].SourceRefs[0].SourceID != "spell-session" {
if rejected.ValidatorName != sourceRefValidatorName { t.Fatalf("SourceID = %q, want raw invalid source ref preserved", response.SpellCasts[0].SourceRefs[0].SourceID)
t.Fatalf("ValidatorName = %q, want %q", rejected.ValidatorName, sourceRefValidatorName)
} }
if rejected.ReasonCode != reasonInvalidSourceRef { if len(output.Rejected) != 0 {
t.Fatalf("ReasonCode = %q, want %q", rejected.ReasonCode, reasonInvalidSourceRef) t.Fatalf("len(Rejected) = %d, want 0", len(output.Rejected))
} }
if output.Manifest.ValidationStatus != "rejected" { if output.Manifest.ValidationStatus != "approved" {
t.Fatalf("ValidationStatus = %q, want rejected", output.Manifest.ValidationStatus) t.Fatalf("ValidationStatus = %q, want approved", output.Manifest.ValidationStatus)
} }
} }
@@ -276,7 +279,7 @@ func dndSpellsReferenceSet(party string, glossary string) contracts.ReferenceSet
return contracts.ReferenceSet{Slots: slots} return contracts.ReferenceSet{Slots: slots}
} }
func TestRunnerFailsWhenDNDSpellsExtractorReturnsMalformedOutput(t *testing.T) { func TestRunnerCarriesMalformedDNDSpellsExtractorOutput(t *testing.T) {
raw := readDNDSpellsFixture(t) raw := readDNDSpellsFixture(t)
resolved := resolveDNDSpellsPipeline(t) resolved := resolveDNDSpellsPipeline(t)
llmClient := &fakeSpellsLLMClient{response: extractionResponse{}} llmClient := &fakeSpellsLLMClient{response: extractionResponse{}}
@@ -286,16 +289,17 @@ func TestRunnerFailsWhenDNDSpellsExtractorReturnsMalformedOutput(t *testing.T) {
RawInput: raw, RawInput: raw,
LLMClient: llmClient, LLMClient: llmClient,
}) })
if err == nil { if err != nil {
t.Fatal("Run() error = nil, want malformed extraction error") t.Fatalf("Run() error = %v, want nil", err)
} }
if !strings.Contains(err.Error(), "extract lane") || if len(output.NormalizeOutputs) != 1 {
!strings.Contains(err.Error(), "dnd spells") || t.Fatalf("len(NormalizeOutputs) = %d, want raw output", len(output.NormalizeOutputs))
!strings.Contains(err.Error(), "spell_casts") {
t.Fatalf("Run() error = %q, want D&D spells extraction context", err.Error())
} }
if output.Manifest.ValidationStatus != "failed" { if string(output.NormalizeOutputs[0].Payload.Content) != `{"spell_casts":null}` {
t.Fatalf("ValidationStatus = %q, want failed", output.Manifest.ValidationStatus) t.Fatalf("content = %s, want raw structured output", output.NormalizeOutputs[0].Payload.Content)
}
if output.Manifest.ValidationStatus != "approved" {
t.Fatalf("ValidationStatus = %q, want approved", output.Manifest.ValidationStatus)
} }
} }
@@ -345,3 +349,13 @@ func parseDNDSpellsFixture(t *testing.T, raw []byte) *source.SourceDocument {
} }
return doc return doc
} }
func decodeRunnerSpellResponse(t *testing.T, raw []byte) extractionResponse {
t.Helper()
var response extractionResponse
if err := json.Unmarshal(raw, &response); err != nil {
t.Fatalf("Unmarshal(raw output) error = %v, want nil", err)
}
return response
}

View File

@@ -12,11 +12,15 @@ import (
func promptExtractionRequest() contracts.ExtractionRequest { func promptExtractionRequest() contracts.ExtractionRequest {
doc := promptSourceDocument() doc := promptSourceDocument()
chunk := &contracts.SourceChunk{ chunk := &contracts.SourceChunk{
ID: "session-alpha:chunk:0", ID: "session-alpha:chunk:0",
SourceID: doc.ID, SourceID: doc.ID,
Index: 0, Index: 0,
Units: append([]source.SourceUnit(nil), doc.Units...), StartUnitID: doc.Units[0].ID,
Metadata: map[string]any{"ignored": "chunk metadata"}, EndUnitID: doc.Units[len(doc.Units)-1].ID,
Content: []byte(`{"units":[1,2]}`),
MediaType: "application/json",
Units: append([]source.SourceUnit(nil), doc.Units...),
Metadata: map[string]any{"ignored": "chunk metadata"},
} }
return contracts.ExtractionRequest{ return contracts.ExtractionRequest{
Source: doc, Source: doc,
@@ -32,7 +36,7 @@ func promptSourceDocument() *source.SourceDocument {
Digest: "sha256:test", Digest: "sha256:test",
Units: []source.SourceUnit{ Units: []source.SourceUnit{
{ {
ID: "seg-001", ID: 1,
Kind: "transcript_segment", Kind: "transcript_segment",
Text: "Aria raises her hand and casts Cure Wounds.", Text: "Aria raises her hand and casts Cure Wounds.",
Metadata: map[string]any{ Metadata: map[string]any{
@@ -43,7 +47,7 @@ func promptSourceDocument() *source.SourceDocument {
}, },
}, },
{ {
ID: "seg-002", ID: 2,
Kind: "transcript_segment", Kind: "transcript_segment",
Text: "The fighter's wounds begin to close.", Text: "The fighter's wounds begin to close.",
Metadata: map[string]any{"ignored": "not rendered"}, Metadata: map[string]any{"ignored": "not rendered"},
@@ -61,12 +65,12 @@ func mustJSON(t *testing.T, value any) string {
return string(encoded) return string(encoded)
} }
func responseSourceRefs(sourceID string, startUnitID string, endUnitID string) []dnd.SourceRefResponse { func responseSourceRefs(sourceID string, startUnitID int, endUnitID int) []dnd.SourceRefResponse {
return []dnd.SourceRefResponse{ return []dnd.SourceRefResponse{
{ {
SourceID: sourceID, SourceID: sourceID,
StartUnitID: dnd.UnitRefFromString(startUnitID), StartUnitID: dnd.UnitRefFromInt(startUnitID),
EndUnitID: dnd.UnitRefFromString(endUnitID), EndUnitID: dnd.UnitRefFromInt(endUnitID),
}, },
} }
} }

View File

@@ -5,21 +5,21 @@
}, },
"segments": [ "segments": [
{ {
"id": "seg-001", "id": 1,
"start": 0, "start": 0,
"end": 4, "end": 4,
"speaker": "Alice", "speaker": "Alice",
"text": "Aria raises her holy symbol and casts Cure Wounds." "text": "Aria raises her holy symbol and casts Cure Wounds."
}, },
{ {
"id": "seg-002", "id": 2,
"start": 4, "start": 4,
"end": 8, "end": 8,
"speaker": "DM", "speaker": "DM",
"text": "The bandit mage casts Shield as the blow lands." "text": "The bandit mage casts Shield as the blow lands."
}, },
{ {
"id": "seg-003", "id": 3,
"start": 8, "start": 8,
"end": 12, "end": 12,
"speaker": "Bob", "speaker": "Bob",

View File

@@ -12,26 +12,6 @@ import (
"gitea.maximumdirect.net/eric/notarius/internal/framework/validate" "gitea.maximumdirect.net/eric/notarius/internal/framework/validate"
) )
func TestExtractorValidatorsReturnsExpectedChain(t *testing.T) {
validators := New().Validators()
if len(validators) != 2 {
t.Fatalf("len(Validators()) = %d, want 2", len(validators))
}
if validators[0].Name() != shapeValidatorName {
t.Fatalf("Validators()[0].Name() = %q, want %q", validators[0].Name(), shapeValidatorName)
}
if validators[1].Name() != sourceRefValidatorName {
t.Fatalf("Validators()[1].Name() = %q, want %q", validators[1].Name(), sourceRefValidatorName)
}
validators[0] = nil
again := New().Validators()
if len(again) != 2 || again[0] == nil || again[0].Name() != shapeValidatorName {
t.Fatalf("Validators() after caller mutation = %#v, want fresh validators", again)
}
}
func TestValidatorsApproveValidCandidate(t *testing.T) { func TestValidatorsApproveValidCandidate(t *testing.T) {
candidate := validSpellCandidate(7) candidate := validSpellCandidate(7)
@@ -130,22 +110,22 @@ func TestSourceRefValidatorRejectsInvalidRefs(t *testing.T) {
}{ }{
{ {
name: "unknown source id", name: "unknown source id",
ref: source.SourceRef{SourceID: "session-beta", StartUnitID: "seg-001", EndUnitID: "seg-002"}, ref: source.SourceRef{SourceID: "session-beta", StartUnitID: 1, EndUnitID: 2},
want: "does not match", want: "does not match",
}, },
{ {
name: "unknown start unit", name: "unknown start unit",
ref: source.SourceRef{SourceID: "session-alpha", StartUnitID: "seg-999", EndUnitID: "seg-002"}, ref: source.SourceRef{SourceID: "session-alpha", StartUnitID: 999, EndUnitID: 2},
want: "start_unit_id", want: "start_unit_id",
}, },
{ {
name: "unknown end unit", name: "unknown end unit",
ref: source.SourceRef{SourceID: "session-alpha", StartUnitID: "seg-001", EndUnitID: "seg-999"}, ref: source.SourceRef{SourceID: "session-alpha", StartUnitID: 1, EndUnitID: 999},
want: "end_unit_id", want: "end_unit_id",
}, },
{ {
name: "reversed unit range", name: "reversed unit range",
ref: source.SourceRef{SourceID: "session-alpha", StartUnitID: "seg-002", EndUnitID: "seg-001"}, ref: source.SourceRef{SourceID: "session-alpha", StartUnitID: 2, EndUnitID: 1},
want: "appears after", want: "appears after",
}, },
} }
@@ -234,7 +214,7 @@ func validSpellCandidate(index int) artifacts.ArtifactCandidate {
Index: index, Index: index,
Payload: spellPayload(validSpellPayload()), Payload: spellPayload(validSpellPayload()),
SourceRefs: []source.SourceRef{ SourceRefs: []source.SourceRef{
{SourceID: "session-alpha", StartUnitID: "seg-001", EndUnitID: "seg-002"}, {SourceID: "session-alpha", StartUnitID: 1, EndUnitID: 2},
}, },
} }
} }

View File

@@ -69,7 +69,7 @@ func (a *Adapter) Parse(ctx context.Context, req contracts.ParseRequest) (*sourc
Metadata: copyMetadata(parsed.Metadata), Metadata: copyMetadata(parsed.Metadata),
} }
seenSegmentIDs := make(map[string]struct{}, len(parsed.Segments)) seenSegmentIDs := make(map[int]struct{}, len(parsed.Segments))
for i, segment := range parsed.Segments { for i, segment := range parsed.Segments {
unit, err := sourceUnit(segment, i, seenSegmentIDs) unit, err := sourceUnit(segment, i, seenSegmentIDs)
if err != nil { if err != nil {
@@ -98,39 +98,35 @@ func Register(registry *pipeline.InputAdapterRegistry) error {
}) })
} }
func sourceUnit(segment segment, index int, seen map[string]struct{}) (source.SourceUnit, error) { func sourceUnit(segment segment, index int, seen map[int]struct{}) (source.SourceUnit, error) {
segmentLabel := fmt.Sprintf("segment[%d]", index) segmentLabel := fmt.Sprintf("segment[%d]", index)
segmentID := strings.TrimSpace(segment.ID) if segment.ID <= 0 {
if segmentID == "" { return source.SourceUnit{}, inputErrorf("%s id must be positive", segmentLabel)
return source.SourceUnit{}, inputErrorf("%s id must not be empty", segmentLabel)
}
if segmentID != segment.ID {
return source.SourceUnit{}, inputErrorf("%s id %q must not contain leading or trailing whitespace", segmentLabel, segment.ID)
} }
if _, ok := seen[segment.ID]; ok { if _, ok := seen[segment.ID]; ok {
return source.SourceUnit{}, inputErrorf("segment id %q is duplicated", segment.ID) return source.SourceUnit{}, inputErrorf("segment id %d is duplicated", segment.ID)
} }
seen[segment.ID] = struct{}{} seen[segment.ID] = struct{}{}
speaker := strings.TrimSpace(segment.Speaker) speaker := strings.TrimSpace(segment.Speaker)
if speaker == "" { if speaker == "" {
return source.SourceUnit{}, inputErrorf("segment %q speaker must not be empty", segment.ID) return source.SourceUnit{}, inputErrorf("segment %d speaker must not be empty", segment.ID)
} }
start, err := validTimestamp(segment.Start, fmt.Sprintf("segment %q start", segment.ID)) start, err := validTimestamp(segment.Start, fmt.Sprintf("segment %d start", segment.ID))
if err != nil { if err != nil {
return source.SourceUnit{}, err return source.SourceUnit{}, err
} }
end, err := validTimestamp(segment.End, fmt.Sprintf("segment %q end", segment.ID)) end, err := validTimestamp(segment.End, fmt.Sprintf("segment %d end", segment.ID))
if err != nil { if err != nil {
return source.SourceUnit{}, err return source.SourceUnit{}, err
} }
if end.Cmp(start) < 0 { if end.Cmp(start) < 0 {
return source.SourceUnit{}, inputErrorf("segment %q end must be greater than or equal to start", segment.ID) return source.SourceUnit{}, inputErrorf("segment %d end must be greater than or equal to start", segment.ID)
} }
if strings.TrimSpace(segment.Text) == "" { if strings.TrimSpace(segment.Text) == "" {
return source.SourceUnit{}, inputErrorf("segment %q text must not be empty", segment.ID) return source.SourceUnit{}, inputErrorf("segment %d text must not be empty", segment.ID)
} }
return source.SourceUnit{ return source.SourceUnit{

View File

@@ -41,8 +41,8 @@ func TestParseValidMinimalTranscript(t *testing.T) {
} }
first := doc.Units[0] first := doc.Units[0]
if first.ID != "seg-001" { if first.ID != 1 {
t.Fatalf("first.ID = %q, want seg-001", first.ID) t.Fatalf("first.ID = %d, want 1", first.ID)
} }
if first.Kind != UnitKind { if first.Kind != UnitKind {
t.Fatalf("first.Kind = %q, want %q", first.Kind, UnitKind) t.Fatalf("first.Kind = %q, want %q", first.Kind, UnitKind)
@@ -86,13 +86,13 @@ func TestParseAcceptsNumericSegmentIDs(t *testing.T) {
if len(doc.Units) != 2 { if len(doc.Units) != 2 {
t.Fatalf("len(doc.Units) = %d, want 2", len(doc.Units)) t.Fatalf("len(doc.Units) = %d, want 2", len(doc.Units))
} }
if doc.Units[0].ID != "1" || doc.Units[1].ID != "2" { if doc.Units[0].ID != 1 || doc.Units[1].ID != 2 {
t.Fatalf("unit IDs = %#v, want numeric IDs normalized to strings", []string{doc.Units[0].ID, doc.Units[1].ID}) t.Fatalf("unit IDs = %#v, want numeric IDs", []int{doc.Units[0].ID, doc.Units[1].ID})
} }
ref := source.SourceRef{ ref := source.SourceRef{
SourceID: doc.ID, SourceID: doc.ID,
StartUnitID: "1", StartUnitID: 1,
EndUnitID: "2", EndUnitID: 2,
} }
if err := source.ValidateRef(doc, ref); err != nil { if err := source.ValidateRef(doc, ref); err != nil {
t.Fatalf("ValidateRef() error = %v, want nil", err) t.Fatalf("ValidateRef() error = %v, want nil", err)
@@ -115,7 +115,7 @@ func TestParseRequestSourceIDOverridesMetadataIDs(t *testing.T) {
} }
func TestParseFallbackDocumentIDIsDeterministic(t *testing.T) { func TestParseFallbackDocumentIDIsDeterministic(t *testing.T) {
raw := []byte(`{"metadata":{},"segments":[{"id":"s1","start":0,"end":1,"speaker":"Narrator","text":"Synthetic text."}]}`) raw := []byte(`{"metadata":{},"segments":[{"id":1,"start":0,"end":1,"speaker":"Narrator","text":"Synthetic text."}]}`)
first, err := New().Parse(context.Background(), contracts.ParseRequest{Raw: raw}) first, err := New().Parse(context.Background(), contracts.ParseRequest{Raw: raw})
if err != nil { if err != nil {
@@ -138,7 +138,7 @@ func TestParseFallbackDocumentIDIsDeterministic(t *testing.T) {
} }
func TestParseUsesMetadataSourceIDWhenMetadataIDIsAbsent(t *testing.T) { func TestParseUsesMetadataSourceIDWhenMetadataIDIsAbsent(t *testing.T) {
raw := []byte(`{"metadata":{"source_id":" source-from-metadata "},"segments":[{"id":"s1","start":0,"end":1,"speaker":"Narrator","text":"Synthetic text."}]}`) raw := []byte(`{"metadata":{"source_id":" source-from-metadata "},"segments":[{"id":1,"start":0,"end":1,"speaker":"Narrator","text":"Synthetic text."}]}`)
doc, err := New().Parse(context.Background(), contracts.ParseRequest{Raw: raw}) doc, err := New().Parse(context.Background(), contracts.ParseRequest{Raw: raw})
if err != nil { if err != nil {
@@ -203,7 +203,7 @@ func TestParseRejectsInvalidInput(t *testing.T) {
{ {
name: "missing segment id", name: "missing segment id",
raw: validJSONWithSegment(`"start":0,"end":1,"speaker":"Narrator","text":"Synthetic text."`), raw: validJSONWithSegment(`"start":0,"end":1,"speaker":"Narrator","text":"Synthetic text."`),
wantErr: []string{"id", "empty"}, wantErr: []string{"id", "positive"},
}, },
{ {
name: "invalid segment id type", name: "invalid segment id type",
@@ -212,52 +212,52 @@ func TestParseRejectsInvalidInput(t *testing.T) {
}, },
{ {
name: "whitespace segment id", name: "whitespace segment id",
raw: validJSONWithSegment(`"id":" s1 ","start":0,"end":1,"speaker":"Narrator","text":"Synthetic text."`), raw: validJSONWithSegment(`"id":" 1 ","start":0,"end":1,"speaker":"Narrator","text":"Synthetic text."`),
wantErr: []string{"id", "whitespace"}, wantErr: []string{"id", "whitespace"},
}, },
{ {
name: "empty text", name: "empty text",
raw: validJSONWithSegment(`"id":"s1","start":0,"end":1,"speaker":"Narrator","text":" "`), raw: validJSONWithSegment(`"id":1,"start":0,"end":1,"speaker":"Narrator","text":" "`),
wantErr: []string{"text", "empty"}, wantErr: []string{"text", "empty"},
}, },
{ {
name: "missing speaker", name: "missing speaker",
raw: validJSONWithSegment(`"id":"s1","start":0,"end":1,"text":"Synthetic text."`), raw: validJSONWithSegment(`"id":1,"start":0,"end":1,"text":"Synthetic text."`),
wantErr: []string{"speaker", "empty"}, wantErr: []string{"speaker", "empty"},
}, },
{ {
name: "missing start", name: "missing start",
raw: validJSONWithSegment(`"id":"s1","end":1,"speaker":"Narrator","text":"Synthetic text."`), raw: validJSONWithSegment(`"id":1,"end":1,"speaker":"Narrator","text":"Synthetic text."`),
wantErr: []string{"start", "empty"}, wantErr: []string{"start", "empty"},
}, },
{ {
name: "missing end", name: "missing end",
raw: validJSONWithSegment(`"id":"s1","start":0,"speaker":"Narrator","text":"Synthetic text."`), raw: validJSONWithSegment(`"id":1,"start":0,"speaker":"Narrator","text":"Synthetic text."`),
wantErr: []string{"end", "empty"}, wantErr: []string{"end", "empty"},
}, },
{ {
name: "negative start", name: "negative start",
raw: validJSONWithSegment(`"id":"s1","start":-1,"end":1,"speaker":"Narrator","text":"Synthetic text."`), raw: validJSONWithSegment(`"id":1,"start":-1,"end":1,"speaker":"Narrator","text":"Synthetic text."`),
wantErr: []string{"start", "negative"}, wantErr: []string{"start", "negative"},
}, },
{ {
name: "non-numeric end", name: "non-numeric end",
raw: validJSONWithSegment(`"id":"s1","start":0,"end":"late","speaker":"Narrator","text":"Synthetic text."`), raw: validJSONWithSegment(`"id":1,"start":0,"end":"late","speaker":"Narrator","text":"Synthetic text."`),
wantErr: []string{"segment[0]", "end", "number"}, wantErr: []string{"segment[0]", "end", "number"},
}, },
{ {
name: "non-finite timestamp", name: "non-finite timestamp",
raw: validJSONWithSegment(`"id":"s1","start":1e10000,"end":1e10000,"speaker":"Narrator","text":"Synthetic text."`), raw: validJSONWithSegment(`"id":1,"start":1e10000,"end":1e10000,"speaker":"Narrator","text":"Synthetic text."`),
wantErr: []string{"start", "valid number"}, wantErr: []string{"start", "valid number"},
}, },
{ {
name: "end before start", name: "end before start",
raw: validJSONWithSegment(`"id":"s1","start":2,"end":1,"speaker":"Narrator","text":"Synthetic text."`), raw: validJSONWithSegment(`"id":1,"start":2,"end":1,"speaker":"Narrator","text":"Synthetic text."`),
wantErr: []string{"end", "start"}, wantErr: []string{"end", "start"},
}, },
{ {
name: "end before start beyond float precision", name: "end before start beyond float precision",
raw: validJSONWithSegment(`"id":"s1","start":9007199254740993,"end":9007199254740992,"speaker":"Narrator","text":"Synthetic text."`), raw: validJSONWithSegment(`"id":1,"start":9007199254740993,"end":9007199254740992,"speaker":"Narrator","text":"Synthetic text."`),
wantErr: []string{"end", "start"}, wantErr: []string{"end", "start"},
}, },
} }
@@ -284,7 +284,7 @@ func TestParseRejectsDuplicateSegmentIDs(t *testing.T) {
if err == nil { if err == nil {
t.Fatal("Parse() error = nil, want duplicate ID error") t.Fatal("Parse() error = nil, want duplicate ID error")
} }
if !strings.Contains(err.Error(), "duplicated") || !strings.Contains(err.Error(), "seg-001") { if !strings.Contains(err.Error(), "duplicated") || !strings.Contains(err.Error(), "1") {
t.Fatalf("Parse() error = %q, want duplicate segment context", err.Error()) t.Fatalf("Parse() error = %q, want duplicate segment context", err.Error())
} }
} }

View File

@@ -219,14 +219,8 @@ type fakeExtractor struct{}
func (fakeExtractor) Key() string { return "fake/extract" } func (fakeExtractor) Key() string { return "fake/extract" }
func (fakeExtractor) ArtifactType() string { return "fake" }
func (fakeExtractor) SchemaVersion() string { return "v1" }
func (fakeExtractor) ReferenceSlots() []contracts.ReferenceSlot { return nil } func (fakeExtractor) ReferenceSlots() []contracts.ReferenceSlot { return nil }
func (fakeExtractor) Validators() []contracts.Validator { return nil }
func (fakeExtractor) Extract(ctx context.Context, req contracts.ExtractionRequest) (contracts.ExtractionResult, error) { func (fakeExtractor) Extract(ctx context.Context, req contracts.ExtractionRequest) (contracts.ExtractionResult, error) {
return contracts.ExtractionResult{}, nil return contracts.ExtractionResult{}, nil
} }

View File

@@ -5,6 +5,8 @@ import (
"encoding/json" "encoding/json"
"fmt" "fmt"
"io" "io"
"strconv"
"strings"
) )
type transcript struct { type transcript struct {
@@ -13,7 +15,7 @@ type transcript struct {
} }
type segment struct { type segment struct {
ID string `json:"id"` ID int `json:"id"`
Start json.Number `json:"start"` Start json.Number `json:"start"`
End json.Number `json:"end"` End json.Number `json:"end"`
Speaker string `json:"speaker"` Speaker string `json:"speaker"`
@@ -78,7 +80,7 @@ func decodeSegment(raw []byte, index int) (segment, error) {
var decoded segment var decoded segment
if err := decodeOptionalSegmentID(fields, "id", &decoded.ID); err != nil { if err := decodeOptionalSegmentID(fields, "id", &decoded.ID); err != nil {
return segment{}, fmt.Errorf("segment[%d] id must be a string or number: %w", index, err) return segment{}, fmt.Errorf("segment[%d] id must be a positive integer string or number: %w", index, err)
} }
if err := decodeOptionalNumber(fields, "start", &decoded.Start); err != nil { if err := decodeOptionalNumber(fields, "start", &decoded.Start); err != nil {
return segment{}, fmt.Errorf("segment[%d] start must be a number: %w", index, err) return segment{}, fmt.Errorf("segment[%d] start must be a number: %w", index, err)
@@ -103,7 +105,7 @@ func decodeOptionalString(fields map[string]json.RawMessage, key string, out *st
return decodeJSON(raw, out) return decodeJSON(raw, out)
} }
func decodeOptionalSegmentID(fields map[string]json.RawMessage, key string, out *string) error { func decodeOptionalSegmentID(fields map[string]json.RawMessage, key string, out *int) error {
raw, ok := fields[key] raw, ok := fields[key]
if !ok { if !ok {
return nil return nil
@@ -111,17 +113,46 @@ func decodeOptionalSegmentID(fields map[string]json.RawMessage, key string, out
var text string var text string
if err := decodeJSON(raw, &text); err == nil { if err := decodeJSON(raw, &text); err == nil {
*out = text parsed, err := parsePositiveInt(text)
if err != nil {
return err
}
*out = parsed
return nil return nil
} }
var number json.Number var number json.Number
if err := decodeJSON(raw, &number); err == nil { if err := decodeJSON(raw, &number); err == nil {
*out = number.String() parsed, err := parsePositiveInt(number.String())
if err != nil {
return err
}
*out = parsed
return nil return nil
} }
return fmt.Errorf("must be a string or number") return fmt.Errorf("must be a positive integer string or number")
}
func parsePositiveInt(value string) (int, error) {
trimmed := strings.TrimSpace(value)
if trimmed == "" {
return 0, fmt.Errorf("must not be empty")
}
if trimmed != value {
return 0, fmt.Errorf("must not contain leading or trailing whitespace")
}
parsed, err := strconv.Atoi(value)
if err != nil {
return 0, fmt.Errorf("must be an integer")
}
if parsed <= 0 {
return 0, fmt.Errorf("must be positive")
}
if strconv.Itoa(parsed) != value {
return 0, fmt.Errorf("must be a canonical positive integer")
}
return parsed, nil
} }
func decodeOptionalNumber(fields map[string]json.RawMessage, key string, out *json.Number) error { func decodeOptionalNumber(fields map[string]json.RawMessage, key string, out *json.Number) error {

View File

@@ -7,7 +7,6 @@ import (
"strings" "strings"
"testing" "testing"
"gitea.maximumdirect.net/eric/notarius/internal/core/artifacts"
"gitea.maximumdirect.net/eric/notarius/internal/core/config" "gitea.maximumdirect.net/eric/notarius/internal/core/config"
"gitea.maximumdirect.net/eric/notarius/internal/core/source" "gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts" "gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
@@ -43,22 +42,29 @@ func TestRunnerProcessesSeriatimInputWithFakeModules(t *testing.T) {
if got := output.Manifest.SourceDigests; len(got) != 1 || got[0] != expectedDoc.Digest { if got := output.Manifest.SourceDigests; len(got) != 1 || got[0] != expectedDoc.Digest {
t.Fatalf("manifest source digests = %#v, want %q", got, expectedDoc.Digest) t.Fatalf("manifest source digests = %#v, want %q", got, expectedDoc.Digest)
} }
if len(output.Approved) != 1 { if len(output.NormalizeOutputs) != 1 {
t.Fatalf("len(Approved) = %d, want 1", len(output.Approved)) t.Fatalf("len(NormalizeOutputs) = %d, want 1", len(output.NormalizeOutputs))
} }
artifact := output.Approved[0] rawOutput := output.NormalizeOutputs[0]
if artifact.ExtractorKey != "fake/extract" || artifact.ArtifactType != "fake.event" || artifact.SchemaVersion != "v1" { if rawOutput.LaneID != "events" || rawOutput.NormalizerKey != pipeline.DefaultNormalizeModule || rawOutput.Schema.ID != "fake.event" || rawOutput.Schema.Version != "v1" {
t.Fatalf("approved artifact envelope = %#v, want fake extractor envelope", artifact) t.Fatalf("raw output envelope = %#v, want fake extractor envelope", rawOutput)
} }
if len(artifact.SourceRefs) != 1 { var payload struct {
t.Fatalf("len(SourceRefs) = %d, want 1", len(artifact.SourceRefs)) Value string `json:"value"`
SourceRefs []source.SourceRef `json:"source_refs"`
} }
if err := source.ValidateRef(expectedDoc, artifact.SourceRefs[0]); err != nil { if err := json.Unmarshal(rawOutput.Payload.Content, &payload); err != nil {
t.Fatalf("Unmarshal(raw output) error = %v, want nil", err)
}
if len(payload.SourceRefs) != 1 {
t.Fatalf("len(SourceRefs) = %d, want 1", len(payload.SourceRefs))
}
if err := source.ValidateRef(expectedDoc, payload.SourceRefs[0]); err != nil {
t.Fatalf("ValidateRef() error = %v, want nil", err) t.Fatalf("ValidateRef() error = %v, want nil", err)
} }
if artifact.SourceRefs[0].StartUnitID != "seg-001" || artifact.SourceRefs[0].EndUnitID != "seg-002" { if payload.SourceRefs[0].StartUnitID != 1 || payload.SourceRefs[0].EndUnitID != 2 {
t.Fatalf("SourceRefs[0] = %#v, want Seriatim unit IDs", artifact.SourceRefs[0]) t.Fatalf("SourceRefs[0] = %#v, want Seriatim unit IDs", payload.SourceRefs[0])
} }
if extractor.calls != 1 { if extractor.calls != 1 {
t.Fatalf("extractor calls = %d, want 1", extractor.calls) t.Fatalf("extractor calls = %d, want 1", extractor.calls)
@@ -165,10 +171,14 @@ func (runnerSeriatimChunker) Chunk(ctx context.Context, req contracts.ChunkReque
return contracts.ChunkResult{ return contracts.ChunkResult{
Chunks: []contracts.SourceChunk{ Chunks: []contracts.SourceChunk{
{ {
ID: req.Source.ID + ":chunk:0", ID: req.Source.ID + ":chunk:0",
SourceID: req.Source.ID, SourceID: req.Source.ID,
Index: 0, Index: 0,
Units: append([]source.SourceUnit(nil), req.Source.Units...), StartUnitID: req.Source.Units[0].ID,
EndUnitID: req.Source.Units[len(req.Source.Units)-1].ID,
Content: []byte(`{"units":[1,2]}`),
MediaType: "application/json",
Units: append([]source.SourceUnit(nil), req.Source.Units...),
}, },
}, },
}, nil }, nil
@@ -182,22 +192,10 @@ func (e *runnerSeriatimExtractor) Key() string {
return "fake/extract" return "fake/extract"
} }
func (e *runnerSeriatimExtractor) ArtifactType() string {
return "fake.event"
}
func (e *runnerSeriatimExtractor) SchemaVersion() string {
return "v1"
}
func (e *runnerSeriatimExtractor) ReferenceSlots() []contracts.ReferenceSlot { func (e *runnerSeriatimExtractor) ReferenceSlots() []contracts.ReferenceSlot {
return nil return nil
} }
func (e *runnerSeriatimExtractor) Validators() []contracts.Validator {
return nil
}
func (e *runnerSeriatimExtractor) Extract(ctx context.Context, req contracts.ExtractionRequest) (contracts.ExtractionResult, error) { func (e *runnerSeriatimExtractor) Extract(ctx context.Context, req contracts.ExtractionRequest) (contracts.ExtractionResult, error) {
e.calls++ e.calls++
if req.Source == nil { if req.Source == nil {
@@ -206,35 +204,47 @@ func (e *runnerSeriatimExtractor) Extract(ctx context.Context, req contracts.Ext
if req.Chunk == nil { if req.Chunk == nil {
return contracts.ExtractionResult{}, fmt.Errorf("chunk must not be nil") return contracts.ExtractionResult{}, fmt.Errorf("chunk must not be nil")
} }
if got := unitIDs(req.Source.Units); !equalStrings(got, []string{"seg-001", "seg-002"}) { if got := unitIDs(req.Source.Units); !equalInts(got, []int{1, 2}) {
return contracts.ExtractionResult{}, fmt.Errorf("source unit IDs = %#v, want Seriatim segment IDs", got) return contracts.ExtractionResult{}, fmt.Errorf("source unit IDs = %#v, want Seriatim segment IDs", got)
} }
if got := unitIDs(req.Chunk.Units); !equalStrings(got, []string{"seg-001", "seg-002"}) { if got := unitIDs(req.Chunk.Units); !equalInts(got, []int{1, 2}) {
return contracts.ExtractionResult{}, fmt.Errorf("chunk unit IDs = %#v, want Seriatim segment IDs", got) return contracts.ExtractionResult{}, fmt.Errorf("chunk unit IDs = %#v, want Seriatim segment IDs", got)
} }
for _, unit := range req.Chunk.Units { for _, unit := range req.Chunk.Units {
if speaker, ok := Speaker(unit); !ok || speaker == "" { if speaker, ok := Speaker(unit); !ok || speaker == "" {
return contracts.ExtractionResult{}, fmt.Errorf("unit %q missing speaker metadata", unit.ID) return contracts.ExtractionResult{}, fmt.Errorf("unit %d missing speaker metadata", unit.ID)
} }
if _, ok := Start(unit); !ok { if _, ok := Start(unit); !ok {
return contracts.ExtractionResult{}, fmt.Errorf("unit %q missing start metadata", unit.ID) return contracts.ExtractionResult{}, fmt.Errorf("unit %d missing start metadata", unit.ID)
} }
if _, ok := End(unit); !ok { if _, ok := End(unit); !ok {
return contracts.ExtractionResult{}, fmt.Errorf("unit %q missing end metadata", unit.ID) return contracts.ExtractionResult{}, fmt.Errorf("unit %d missing end metadata", unit.ID)
} }
} }
return contracts.ExtractionResult{ payload, err := json.Marshal(struct {
Candidates: []artifacts.ArtifactCandidate{ Value string `json:"value"`
SourceRefs []source.SourceRef `json:"source_refs"`
}{
Value: "seriatim-source-ref",
SourceRefs: []source.SourceRef{
{ {
Payload: json.RawMessage(`{"value":"seriatim-source-ref"}`), SourceID: req.Source.ID,
SourceRefs: []source.SourceRef{ StartUnitID: req.Chunk.Units[0].ID,
{ EndUnitID: req.Chunk.Units[len(req.Chunk.Units)-1].ID,
SourceID: req.Source.ID, },
StartUnitID: req.Chunk.Units[0].ID, },
EndUnitID: req.Chunk.Units[len(req.Chunk.Units)-1].ID, })
}, if err != nil {
}, return contracts.ExtractionResult{}, err
}
return contracts.ExtractionResult{
Output: contracts.ExtractOutput{
Schema: contracts.ResponseSchema{ID: "fake.event", Name: "fake_event", Version: "v1"},
Payload: contracts.RawPayload{
Content: payload,
MediaType: "application/json",
}, },
}, },
}, nil }, nil
@@ -254,15 +264,15 @@ func (runnerSeriatimOutput) Encode(ctx context.Context, req contracts.OutputRequ
}, nil }, nil
} }
func unitIDs(units []source.SourceUnit) []string { func unitIDs(units []source.SourceUnit) []int {
ids := make([]string, 0, len(units)) ids := make([]int, 0, len(units))
for _, unit := range units { for _, unit := range units {
ids = append(ids, unit.ID) ids = append(ids, unit.ID)
} }
return ids return ids
} }
func equalStrings(a, b []string) bool { func equalInts(a, b []int) bool {
if len(a) != len(b) { if len(a) != len(b) {
return false return false
} }

View File

@@ -4,14 +4,14 @@
}, },
"segments": [ "segments": [
{ {
"id": "seg-001", "id": 1,
"start": 0, "start": 0,
"end": 1, "end": 1,
"speaker": "Narrator", "speaker": "Narrator",
"text": "First segment." "text": "First segment."
}, },
{ {
"id": "seg-001", "id": 1,
"start": 1, "start": 1,
"end": 2, "end": 2,
"speaker": "Player", "speaker": "Player",

View File

@@ -6,14 +6,14 @@
}, },
"segments": [ "segments": [
{ {
"id": "seg-001", "id": 1,
"start": 0, "start": 0,
"end": 4.5, "end": 4.5,
"speaker": "Narrator", "speaker": "Narrator",
"text": "The stone door opens." "text": "The stone door opens."
}, },
{ {
"id": "seg-002", "id": 2,
"start": 4.5, "start": 4.5,
"end": 8, "end": 8,
"speaker": "Player", "speaker": "Player",

View File

@@ -4,9 +4,10 @@ import (
"context" "context"
"encoding/json" "encoding/json"
"fmt" "fmt"
"mime"
"sort"
"strings"
"gitea.maximumdirect.net/eric/notarius/internal/core/artifacts"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts" "gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline" "gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
) )
@@ -36,11 +37,39 @@ func (m *Merger) Merge(ctx context.Context, req contracts.MergeRequest) (contrac
return contracts.MergeResult{}, mergerErrorf("context error before merge: %w", err) return contracts.MergeResult{}, mergerErrorf("context error before merge: %w", err)
} }
var candidates []artifacts.ArtifactCandidate outputs, err := orderedOutputs(req.ExtractOutputs)
for _, chunkArtifacts := range req.ChunkArtifacts { if err != nil {
candidates = append(candidates, cloneCandidates(chunkArtifacts.Candidates)...) return contracts.MergeResult{}, err
} }
return contracts.MergeResult{Candidates: candidates}, nil if len(outputs) == 1 {
payload := cloneRawPayload(outputs[0].Payload)
return contracts.MergeResult{
Output: contracts.MergeOutput{
LaneID: req.LaneID,
MergerKey: Key,
SourceID: outputs[0].SourceID,
Schema: outputs[0].Schema,
Payload: payload,
},
}, nil
}
content, err := mergedContent(outputs)
if err != nil {
return contracts.MergeResult{}, err
}
return contracts.MergeResult{
Output: contracts.MergeOutput{
LaneID: req.LaneID,
MergerKey: Key,
SourceID: sourceID(outputs),
Schema: commonSchema(outputs),
Payload: contracts.RawPayload{
Content: content,
MediaType: "application/json",
},
},
}, nil
} }
func ModuleSpec() pipeline.ModuleSpec { func ModuleSpec() pipeline.ModuleSpec {
@@ -57,27 +86,127 @@ func Register(registry *pipeline.MergerRegistry) error {
}) })
} }
func cloneCandidates(candidates []artifacts.ArtifactCandidate) []artifacts.ArtifactCandidate { func orderedOutputs(outputs []contracts.ExtractOutput) ([]contracts.ExtractOutput, error) {
if len(candidates) == 0 { ordered := make([]contracts.ExtractOutput, 0, len(outputs))
return nil for _, output := range outputs {
if !isJSONMediaType(output.Payload.MediaType) {
return nil, mergerErrorf("extract output for chunk %q has unsupported media type %q", output.ChunkID, output.Payload.MediaType)
}
if !json.Valid(output.Payload.Content) {
return nil, mergerErrorf("extract output for chunk %q contains invalid JSON", output.ChunkID)
}
ordered = append(ordered, cloneExtractOutput(output))
} }
sort.SliceStable(ordered, func(i, j int) bool {
out := make([]artifacts.ArtifactCandidate, 0, len(candidates)) return ordered[i].ChunkIndex < ordered[j].ChunkIndex
for _, candidate := range candidates { })
out = append(out, cloneCandidate(candidate)) return ordered, nil
}
return out
} }
func cloneCandidate(candidate artifacts.ArtifactCandidate) artifacts.ArtifactCandidate { func mergedContent(outputs []contracts.ExtractOutput) ([]byte, error) {
return artifacts.ArtifactCandidate{ values := make([]any, 0, len(outputs))
Index: candidate.Index, objects := make([]map[string]any, 0, len(outputs))
ExtractorKey: candidate.ExtractorKey, for _, output := range outputs {
ArtifactType: candidate.ArtifactType, var value any
SchemaVersion: candidate.SchemaVersion, if err := json.Unmarshal(output.Payload.Content, &value); err != nil {
Payload: append(json.RawMessage(nil), candidate.Payload...), return nil, mergerErrorf("decode extract output for chunk %q: %w", output.ChunkID, err)
SourceRefs: append([]source.SourceRef(nil), candidate.SourceRefs...), }
Metadata: cloneMetadata(candidate.Metadata), values = append(values, value)
object, ok := value.(map[string]any)
if !ok {
continue
}
objects = append(objects, object)
}
if len(objects) == len(outputs) {
if field, ok := commonArrayField(objects); ok {
merged := make([]any, 0)
for _, object := range objects {
items := object[field].([]any)
merged = append(merged, items...)
}
return marshalMerged(map[string]any{field: merged})
}
}
return marshalMerged(values)
}
func commonArrayField(objects []map[string]any) (string, bool) {
if len(objects) == 0 {
return "", false
}
candidates := map[string]struct{}{}
for key, value := range objects[0] {
if _, ok := value.([]any); ok {
candidates[key] = struct{}{}
}
}
for _, object := range objects[1:] {
for key := range candidates {
if _, ok := object[key].([]any); !ok {
delete(candidates, key)
}
}
}
if len(candidates) != 1 {
return "", false
}
for key := range candidates {
return key, true
}
return "", false
}
func marshalMerged(value any) ([]byte, error) {
content, err := json.Marshal(value)
if err != nil {
return nil, mergerErrorf("encode merged output: %w", err)
}
return content, nil
}
func isJSONMediaType(mediaType string) bool {
base, _, err := mime.ParseMediaType(strings.TrimSpace(mediaType))
if err != nil {
base = strings.TrimSpace(mediaType)
}
return strings.EqualFold(base, "application/json")
}
func sourceID(outputs []contracts.ExtractOutput) string {
for _, output := range outputs {
if output.SourceID != "" {
return output.SourceID
}
}
return ""
}
func commonSchema(outputs []contracts.ExtractOutput) contracts.ResponseSchema {
if len(outputs) == 0 {
return contracts.ResponseSchema{}
}
schema := outputs[0].Schema
for _, output := range outputs[1:] {
if output.Schema != schema {
return contracts.ResponseSchema{}
}
}
return schema
}
func cloneExtractOutput(output contracts.ExtractOutput) contracts.ExtractOutput {
output.Payload = cloneRawPayload(output.Payload)
return output
}
func cloneRawPayload(payload contracts.RawPayload) contracts.RawPayload {
return contracts.RawPayload{
Content: append([]byte(nil), payload.Content...),
MediaType: payload.MediaType,
Metadata: cloneMetadata(payload.Metadata),
Warnings: append([]contracts.Warning(nil), payload.Warnings...),
} }
} }

View File

@@ -4,10 +4,9 @@ import (
"context" "context"
"encoding/json" "encoding/json"
"reflect" "reflect"
"strings"
"testing" "testing"
"gitea.maximumdirect.net/eric/notarius/internal/core/artifacts"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts" "gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline" "gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
) )
@@ -35,113 +34,158 @@ func TestModuleSpecAndRegister(t *testing.T) {
} }
} }
func TestMergePreservesChunkAndCandidateOrder(t *testing.T) { func TestMergePassesThroughSingleExtractOutput(t *testing.T) {
input := extractOutput("chunk-0", 0, `{"name":"original"}`)
result, err := New().Merge(context.Background(), contracts.MergeRequest{ result, err := New().Merge(context.Background(), contracts.MergeRequest{
ChunkArtifacts: []contracts.ChunkArtifacts{ LaneID: "events",
{ ExtractOutputs: []contracts.ExtractOutput{input},
Chunk: sourceChunk(0), })
Candidates: []artifacts.ArtifactCandidate{candidate(2, "first-b"), candidate(1, "first-a")}, if err != nil {
}, t.Fatalf("Merge() error = %v, want nil", err)
{ }
Chunk: sourceChunk(1),
Candidates: []artifacts.ArtifactCandidate{candidate(4, "second-b"), candidate(3, "second-a")}, if result.Output.LaneID != "events" || result.Output.MergerKey != Key {
}, t.Fatalf("output provenance = %#v, want lane and merger", result.Output)
}
if string(result.Output.Payload.Content) != `{"name":"original"}` {
t.Fatalf("content = %s, want original content", result.Output.Payload.Content)
}
if result.Output.Payload.Metadata["name"] != "chunk-0" {
t.Fatalf("metadata = %#v, want original metadata", result.Output.Payload.Metadata)
}
}
func TestMergeDefensivelyCopiesRawPayload(t *testing.T) {
input := extractOutput("chunk-0", 0, `{"name":"original"}`)
result, err := New().Merge(context.Background(), contracts.MergeRequest{
LaneID: "events",
ExtractOutputs: []contracts.ExtractOutput{input},
})
if err != nil {
t.Fatalf("Merge() error = %v, want nil", err)
}
input.Payload.Content[0] = '['
input.Payload.Metadata["name"] = "changed"
if string(result.Output.Payload.Content) != `{"name":"original"}` {
t.Fatalf("content changed after input mutation: %s", result.Output.Payload.Content)
}
if result.Output.Payload.Metadata["name"] != "chunk-0" {
t.Fatalf("metadata changed after input mutation: %#v", result.Output.Payload.Metadata)
}
}
func TestMergeConcatenatesCommonTopLevelArrayFieldInChunkOrder(t *testing.T) {
result, err := New().Merge(context.Background(), contracts.MergeRequest{
LaneID: "events",
ExtractOutputs: []contracts.ExtractOutput{
extractOutput("chunk-1", 1, `{"events":[{"name":"second"}]}`),
extractOutput("chunk-0", 0, `{"events":[{"name":"first"}]}`),
},
})
if err != nil {
t.Fatalf("Merge() error = %v, want nil", err)
}
if result.Output.Payload.MediaType != "application/json" {
t.Fatalf("MediaType = %q, want application/json", result.Output.Payload.MediaType)
}
var decoded struct {
Events []struct {
Name string `json:"name"`
} `json:"events"`
}
if err := json.Unmarshal(result.Output.Payload.Content, &decoded); err != nil {
t.Fatalf("Unmarshal() error = %v, want nil", err)
}
if len(decoded.Events) != 2 || decoded.Events[0].Name != "first" || decoded.Events[1].Name != "second" {
t.Fatalf("events = %#v, want concatenated chunk order", decoded.Events)
}
if result.Output.Schema.ID != "schema-id" {
t.Fatalf("schema = %#v, want common extract schema", result.Output.Schema)
}
}
func TestMergeFallsBackToOrderedJSONValueArrayWhenShapesDiffer(t *testing.T) {
result, err := New().Merge(context.Background(), contracts.MergeRequest{
LaneID: "events",
ExtractOutputs: []contracts.ExtractOutput{
extractOutput("chunk-1", 1, `{"notes":["second"]}`),
extractOutput("chunk-0", 0, `{"events":[{"name":"first"}]}`),
}, },
}) })
if err != nil { if err != nil {
t.Fatalf("Merge() error = %v, want nil", err) t.Fatalf("Merge() error = %v, want nil", err)
} }
got := candidateNames(result.Candidates) var decoded []map[string]any
want := []string{"first-b", "first-a", "second-b", "second-a"} if err := json.Unmarshal(result.Output.Payload.Content, &decoded); err != nil {
if !reflect.DeepEqual(got, want) { t.Fatalf("Unmarshal() error = %v, want nil", err)
t.Fatalf("candidate order = %#v, want %#v", got, want)
} }
if len(result.Warnings) != 0 { if len(decoded) != 2 {
t.Fatalf("Warnings = %#v, want none", result.Warnings) t.Fatalf("len(decoded) = %d, want 2", len(decoded))
}
if _, ok := decoded[0]["events"]; !ok {
t.Fatalf("decoded[0] = %#v, want first chunk value", decoded[0])
}
if _, ok := decoded[1]["notes"]; !ok {
t.Fatalf("decoded[1] = %#v, want second chunk value", decoded[1])
} }
} }
func TestMergeDefensivelyCopiesCandidates(t *testing.T) { func TestMergeRejectsInvalidJSONAndNonJSONMediaTypes(t *testing.T) {
input := []contracts.ChunkArtifacts{ tests := []struct {
name string
output contracts.ExtractOutput
want string
}{
{ {
Chunk: sourceChunk(0), name: "invalid JSON",
Candidates: []artifacts.ArtifactCandidate{candidate(1, "original")}, output: extractOutput("chunk-0", 0, `{"events":[`),
want: "invalid JSON",
},
{
name: "non JSON media type",
output: func() contracts.ExtractOutput {
output := extractOutput("chunk-0", 0, `{"events":[]}`)
output.Payload.MediaType = "text/plain"
return output
}(),
want: "unsupported media type",
}, },
} }
result, err := New().Merge(context.Background(), contracts.MergeRequest{ChunkArtifacts: input}) for _, test := range tests {
if err != nil { t.Run(test.name, func(t *testing.T) {
t.Fatalf("Merge() error = %v, want nil", err) _, err := New().Merge(context.Background(), contracts.MergeRequest{
} LaneID: "events",
if len(result.Candidates) != 1 { ExtractOutputs: []contracts.ExtractOutput{test.output},
t.Fatalf("len(Candidates) = %d, want 1", len(result.Candidates)) })
} if err == nil {
t.Fatal("Merge() error = nil, want error")
input[0].Candidates[0].Index = 99 }
input[0].Candidates[0].Payload[0] = '[' if !strings.Contains(err.Error(), test.want) {
input[0].Candidates[0].SourceRefs[0].StartUnitID = "changed" t.Fatalf("Merge() error = %q, want %q", err.Error(), test.want)
input[0].Candidates[0].Metadata["name"] = "changed" }
})
got := result.Candidates[0]
if got.Index != 1 {
t.Fatalf("Index = %d, want 1", got.Index)
}
if string(got.Payload) != `{"name":"original"}` {
t.Fatalf("Payload = %s, want original payload", got.Payload)
}
if got.SourceRefs[0].StartUnitID != "u1" {
t.Fatalf("SourceRefs = %#v, want original source ref", got.SourceRefs)
}
if got.Metadata["name"] != "original" {
t.Fatalf("Metadata = %#v, want original metadata", got.Metadata)
} }
} }
func TestMergeHandlesEmptyInput(t *testing.T) { func extractOutput(chunkID string, chunkIndex int, content string) contracts.ExtractOutput {
result, err := New().Merge(context.Background(), contracts.MergeRequest{}) return contracts.ExtractOutput{
if err != nil { LaneID: "events",
t.Fatalf("Merge() error = %v, want nil", err) ExtractorKey: "extract",
} SourceID: "source-1",
if len(result.Candidates) != 0 { ChunkID: chunkID,
t.Fatalf("len(Candidates) = %d, want 0", len(result.Candidates)) ChunkIndex: chunkIndex,
} Schema: contracts.ResponseSchema{ID: "schema-id", Name: "schema-name", Version: "v1"},
if len(result.Warnings) != 0 { Payload: contracts.RawPayload{
t.Fatalf("Warnings = %#v, want none", result.Warnings) Content: []byte(content),
} MediaType: "application/json",
} Metadata: map[string]any{"name": chunkID},
func candidate(index int, name string) artifacts.ArtifactCandidate {
return artifacts.ArtifactCandidate{
Index: index,
ExtractorKey: "generic-extractor",
ArtifactType: "generic-artifact",
SchemaVersion: "v1",
Payload: json.RawMessage(`{"name":"` + name + `"}`),
SourceRefs: []source.SourceRef{
{SourceID: "source-1", StartUnitID: "u1", EndUnitID: "u1"},
},
Metadata: map[string]any{
"name": name,
},
}
}
func candidateNames(candidates []artifacts.ArtifactCandidate) []string {
names := make([]string, 0, len(candidates))
for _, candidate := range candidates {
names = append(names, candidate.Metadata["name"].(string))
}
return names
}
func sourceChunk(index int) contracts.SourceChunk {
return contracts.SourceChunk{
ID: "chunk",
SourceID: "source-1",
Index: index,
Units: []source.SourceUnit{
{ID: "u1", Kind: "unit", Text: "Source unit."},
}, },
} }
} }

View File

@@ -2,11 +2,8 @@ package noop
import ( import (
"context" "context"
"encoding/json"
"fmt" "fmt"
"gitea.maximumdirect.net/eric/notarius/internal/core/artifacts"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts" "gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline" "gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
) )
@@ -39,7 +36,15 @@ func (n *Normalizer) Normalize(ctx context.Context, req contracts.NormalizeReque
if err := ctx.Err(); err != nil { if err := ctx.Err(); err != nil {
return contracts.NormalizeResult{}, normalizerErrorf("context error before normalize: %w", err) return contracts.NormalizeResult{}, normalizerErrorf("context error before normalize: %w", err)
} }
return contracts.NormalizeResult{Candidates: cloneCandidates(req.Candidates)}, nil return contracts.NormalizeResult{
Output: contracts.NormalizeOutput{
LaneID: req.LaneID,
NormalizerKey: Key,
SourceID: req.MergeOutput.SourceID,
Schema: req.MergeOutput.Schema,
Payload: cloneRawPayload(req.MergeOutput.Payload),
},
}, nil
} }
func ModuleSpec() pipeline.ModuleSpec { func ModuleSpec() pipeline.ModuleSpec {
@@ -57,24 +62,13 @@ func Register(registry *pipeline.NormalizerRegistry) error {
}) })
} }
func cloneCandidates(candidates []artifacts.ArtifactCandidate) []artifacts.ArtifactCandidate { func cloneRawPayload(payload contracts.RawPayload) contracts.RawPayload {
if len(candidates) == 0 { return contracts.RawPayload{
return nil Content: append([]byte(nil), payload.Content...),
MediaType: payload.MediaType,
Metadata: cloneMetadata(payload.Metadata),
Warnings: append([]contracts.Warning(nil), payload.Warnings...),
} }
out := make([]artifacts.ArtifactCandidate, 0, len(candidates))
for _, candidate := range candidates {
out = append(out, artifacts.ArtifactCandidate{
Index: candidate.Index,
ExtractorKey: candidate.ExtractorKey,
ArtifactType: candidate.ArtifactType,
SchemaVersion: candidate.SchemaVersion,
Payload: append(json.RawMessage(nil), candidate.Payload...),
SourceRefs: append([]source.SourceRef(nil), candidate.SourceRefs...),
Metadata: cloneMetadata(candidate.Metadata),
})
}
return out
} }
func cloneMetadata(metadata map[string]any) map[string]any { func cloneMetadata(metadata map[string]any) map[string]any {

View File

@@ -2,12 +2,9 @@ package noop
import ( import (
"context" "context"
"encoding/json"
"reflect" "reflect"
"testing" "testing"
"gitea.maximumdirect.net/eric/notarius/internal/core/artifacts"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts" "gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline" "gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
) )
@@ -43,98 +40,60 @@ func TestModuleSpecAndRegister(t *testing.T) {
} }
} }
func TestNormalizePassesThroughOrderAndValues(t *testing.T) { func TestNormalizePassesThroughMergeOutput(t *testing.T) {
input := []artifacts.ArtifactCandidate{ input := mergeOutput(`{"name":"original"}`)
candidate(3, "third"),
candidate(1, "first"),
candidate(2, "second"),
}
result, err := New().Normalize(context.Background(), contracts.NormalizeRequest{Candidates: input}) result, err := New().Normalize(context.Background(), contracts.NormalizeRequest{
LaneID: "events",
MergeOutput: input,
})
if err != nil { if err != nil {
t.Fatalf("Normalize() error = %v, want nil", err) t.Fatalf("Normalize() error = %v, want nil", err)
} }
if len(result.Warnings) != 0 {
t.Fatalf("Warnings = %#v, want none", result.Warnings)
}
got := candidateNames(result.Candidates) if result.Output.LaneID != "events" || result.Output.NormalizerKey != Key {
want := []string{"third", "first", "second"} t.Fatalf("output provenance = %#v, want lane and normalizer", result.Output)
if !reflect.DeepEqual(got, want) {
t.Fatalf("candidate order = %#v, want %#v", got, want)
} }
if !reflect.DeepEqual(result.Candidates[0].SourceRefs, input[0].SourceRefs) { if string(result.Output.Payload.Content) != `{"name":"original"}` {
t.Fatalf("SourceRefs = %#v, want %#v", result.Candidates[0].SourceRefs, input[0].SourceRefs) t.Fatalf("content = %s, want original content", result.Output.Payload.Content)
} }
if !reflect.DeepEqual(result.Candidates[0].Metadata, input[0].Metadata) { if result.Output.Payload.Metadata["name"] != "original" {
t.Fatalf("Metadata = %#v, want %#v", result.Candidates[0].Metadata, input[0].Metadata) t.Fatalf("metadata = %#v, want original metadata", result.Output.Payload.Metadata)
} }
} }
func TestNormalizeDefensivelyCopiesCandidates(t *testing.T) { func TestNormalizeDefensivelyCopiesRawPayload(t *testing.T) {
input := []artifacts.ArtifactCandidate{candidate(1, "original")} input := mergeOutput(`{"name":"original"}`)
result, err := New().Normalize(context.Background(), contracts.NormalizeRequest{Candidates: input}) result, err := New().Normalize(context.Background(), contracts.NormalizeRequest{
LaneID: "events",
MergeOutput: input,
})
if err != nil { if err != nil {
t.Fatalf("Normalize() error = %v, want nil", err) t.Fatalf("Normalize() error = %v, want nil", err)
} }
if len(result.Candidates) != 1 {
t.Fatalf("len(Candidates) = %d, want 1", len(result.Candidates))
}
input[0].Index = 99 input.Payload.Content[0] = '['
input[0].Payload[0] = '[' input.Payload.Metadata["name"] = "changed"
input[0].SourceRefs[0].EndUnitID = "changed"
input[0].Metadata["name"] = "changed"
got := result.Candidates[0] if string(result.Output.Payload.Content) != `{"name":"original"}` {
if got.Index != 1 { t.Fatalf("content changed after input mutation: %s", result.Output.Payload.Content)
t.Fatalf("Index = %d, want 1", got.Index)
} }
if string(got.Payload) != `{"name":"original"}` { if result.Output.Payload.Metadata["name"] != "original" {
t.Fatalf("Payload = %s, want original payload", got.Payload) t.Fatalf("metadata changed after input mutation: %#v", result.Output.Payload.Metadata)
}
if got.SourceRefs[0].EndUnitID != "u1" {
t.Fatalf("SourceRefs = %#v, want original source ref", got.SourceRefs)
}
if got.Metadata["name"] != "original" {
t.Fatalf("Metadata = %#v, want original metadata", got.Metadata)
} }
} }
func TestNormalizeHandlesEmptyInput(t *testing.T) { func mergeOutput(content string) contracts.MergeOutput {
result, err := New().Normalize(context.Background(), contracts.NormalizeRequest{}) return contracts.MergeOutput{
if err != nil { LaneID: "events",
t.Fatalf("Normalize() error = %v, want nil", err) MergerKey: "merge",
} SourceID: "source-1",
if len(result.Candidates) != 0 { Schema: contracts.ResponseSchema{ID: "schema-id", Name: "schema-name", Version: "v1"},
t.Fatalf("len(Candidates) = %d, want 0", len(result.Candidates)) Payload: contracts.RawPayload{
} Content: []byte(content),
if len(result.Warnings) != 0 { MediaType: "application/json",
t.Fatalf("Warnings = %#v, want none", result.Warnings) Metadata: map[string]any{"name": "original"},
}
}
func candidate(index int, name string) artifacts.ArtifactCandidate {
return artifacts.ArtifactCandidate{
Index: index,
ExtractorKey: "generic-extractor",
ArtifactType: "generic-artifact",
SchemaVersion: "v1",
Payload: json.RawMessage(`{"name":"` + name + `"}`),
SourceRefs: []source.SourceRef{
{SourceID: "source-1", StartUnitID: "u1", EndUnitID: "u1"},
},
Metadata: map[string]any{
"name": name,
}, },
} }
} }
func candidateNames(candidates []artifacts.ArtifactCandidate) []string {
names := make([]string, 0, len(candidates))
for _, candidate := range candidates {
names = append(names, candidate.Metadata["name"].(string))
}
return names
}

View File

@@ -4,12 +4,11 @@ import (
"context" "context"
stdjson "encoding/json" stdjson "encoding/json"
"fmt" "fmt"
"mime"
"regexp" "regexp"
"sort" "sort"
"strings" "strings"
"gitea.maximumdirect.net/eric/notarius/internal/core/artifacts"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts" "gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline" "gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
) )
@@ -18,7 +17,7 @@ const Key = "json"
const contentTypeJSON = "application/json" const contentTypeJSON = "application/json"
var safeArtifactFileChar = regexp.MustCompile(`[^A-Za-z0-9._-]`) var safeOutputFileChar = regexp.MustCompile(`[^A-Za-z0-9._-]`)
var _ contracts.OutputEncoder = (*Encoder)(nil) var _ contracts.OutputEncoder = (*Encoder)(nil)
@@ -66,24 +65,24 @@ func Register(registry *pipeline.OutputEncoderRegistry) error {
} }
type indexFile struct { type indexFile struct {
ManifestFile string `json:"manifest_file"` ManifestFile string `json:"manifest_file"`
ArtifactFiles []artifactFileIndex `json:"artifact_files"` OutputFiles []outputFileIndex `json:"output_files"`
RejectedFile string `json:"rejected_file"` RejectedFile string `json:"rejected_file"`
WarningsFile string `json:"warnings_file"` WarningsFile string `json:"warnings_file"`
} }
type artifactFileIndex struct { type outputFileIndex struct {
ArtifactType string `json:"artifact_type"` LaneID string `json:"lane_id"`
File string `json:"file"` MediaType string `json:"media_type,omitempty"`
} File string `json:"file"`
ModuleKey string `json:"module_key,omitempty"`
type artifactFile struct { SchemaID string `json:"schema_id,omitempty"`
ArtifactType string `json:"artifact_type"` SchemaName string `json:"schema_name,omitempty"`
Artifacts []artifacts.Artifact `json:"artifacts"` SchemaVer string `json:"schema_version,omitempty"`
} }
type rejectedFile struct { type rejectedFile struct {
Rejected []artifacts.RejectedArtifact `json:"rejected"` Rejected []contracts.RejectedOutput `json:"rejected"`
} }
type warningsFile struct { type warningsFile struct {
@@ -91,43 +90,39 @@ type warningsFile struct {
} }
func logicalFiles(req contracts.OutputRequest) ([]contracts.OutputFile, error) { func logicalFiles(req contracts.OutputRequest) ([]contracts.OutputFile, error) {
artifactsByType := make(map[string][]artifacts.Artifact) outputs := cloneNormalizeOutputs(req.NormalizeOutputs)
for _, artifact := range req.Approved { sort.SliceStable(outputs, func(i, j int) bool {
artifactsByType[artifact.ArtifactType] = append(artifactsByType[artifact.ArtifactType], cloneArtifact(artifact)) return outputs[i].LaneID < outputs[j].LaneID
} })
artifactTypes := make([]string, 0, len(artifactsByType)) outputIndexes := make([]outputFileIndex, 0, len(outputs))
for artifactType := range artifactsByType { files := make([]contracts.OutputFile, 0, len(outputs)+4)
artifactTypes = append(artifactTypes, artifactType)
}
sort.Strings(artifactTypes)
artifactIndexes := make([]artifactFileIndex, 0, len(artifactTypes))
files := make([]contracts.OutputFile, 0, len(artifactTypes)+4)
manifestFile, err := jsonFile("manifest.json", req.Manifest) manifestFile, err := jsonFile("manifest.json", req.Manifest)
if err != nil { if err != nil {
return nil, err return nil, err
} }
files = append(files, manifestFile) files = append(files, manifestFile)
usedArtifactFiles := make(map[string]string, len(artifactTypes)) usedOutputFiles := make(map[string]string, len(outputs))
for _, artifactType := range artifactTypes { for _, output := range outputs {
name, err := artifactFileName(artifactType) name, err := outputFileName(output.LaneID)
if err != nil { if err != nil {
return nil, err return nil, err
} }
if existingType, ok := usedArtifactFiles[name]; ok { if existingLane, ok := usedOutputFiles[name]; ok {
return nil, encoderErrorf("artifact types %q and %q produce duplicate output file %q", existingType, artifactType, name) return nil, encoderErrorf("lanes %q and %q produce duplicate output file %q", existingLane, output.LaneID, name)
} }
usedArtifactFiles[name] = artifactType usedOutputFiles[name] = output.LaneID
artifactIndexes = append(artifactIndexes, artifactFileIndex{ outputIndexes = append(outputIndexes, outputFileIndex{
ArtifactType: artifactType, LaneID: output.LaneID,
File: name, MediaType: output.Payload.MediaType,
}) File: name,
file, err := jsonFile(name, artifactFile{ ModuleKey: output.NormalizerKey,
ArtifactType: artifactType, SchemaID: output.Schema.ID,
Artifacts: artifactsByType[artifactType], SchemaName: output.Schema.Name,
SchemaVer: output.Schema.Version,
}) })
file, err := rawOutputFile(name, output.Payload)
if err != nil { if err != nil {
return nil, err return nil, err
} }
@@ -135,10 +130,10 @@ func logicalFiles(req contracts.OutputRequest) ([]contracts.OutputFile, error) {
} }
index := indexFile{ index := indexFile{
ManifestFile: "manifest.json", ManifestFile: "manifest.json",
ArtifactFiles: artifactIndexes, OutputFiles: outputIndexes,
RejectedFile: "rejected.json", RejectedFile: "rejected.json",
WarningsFile: "warnings.json", WarningsFile: "warnings.json",
} }
indexOutput, err := jsonFile("index.json", index) indexOutput, err := jsonFile("index.json", index)
if err != nil { if err != nil {
@@ -159,6 +154,41 @@ func logicalFiles(req contracts.OutputRequest) ([]contracts.OutputFile, error) {
return files, nil return files, nil
} }
func rawOutputFile(name string, payload contracts.RawPayload) (contracts.OutputFile, error) {
content := append([]byte(nil), payload.Content...)
if len(content) == 0 {
content = []byte("null")
}
mediaType := strings.TrimSpace(payload.MediaType)
if mediaType == "" {
mediaType = "application/octet-stream"
}
if !isJSONMediaType(mediaType) {
return contracts.OutputFile{}, encoderErrorf("normalized output %q has unsupported media type %q", name, mediaType)
}
var decoded any
if err := stdjson.Unmarshal(content, &decoded); err != nil {
return contracts.OutputFile{}, encoderErrorf("normalized output %q contains invalid JSON: %w", name, err)
}
pretty, err := marshalPretty(decoded)
if err != nil {
return contracts.OutputFile{}, err
}
return contracts.OutputFile{
Name: name,
ContentType: mediaType,
Bytes: pretty,
}, nil
}
func isJSONMediaType(mediaType string) bool {
base, _, err := mime.ParseMediaType(strings.TrimSpace(mediaType))
if err != nil {
base = strings.TrimSpace(mediaType)
}
return strings.EqualFold(base, contentTypeJSON)
}
func jsonFile(name string, value any) (contracts.OutputFile, error) { func jsonFile(name string, value any) (contracts.OutputFile, error) {
data, err := marshalPretty(value) data, err := marshalPretty(value)
if err != nil { if err != nil {
@@ -179,57 +209,46 @@ func marshalPretty(value any) ([]byte, error) {
return append(data, '\n'), nil return append(data, '\n'), nil
} }
func artifactFileName(artifactType string) (string, error) { func outputFileName(laneID string) (string, error) {
sanitized := safeArtifactFileChar.ReplaceAllString(strings.TrimSpace(artifactType), "_") sanitized := safeOutputFileChar.ReplaceAllString(strings.TrimSpace(laneID), "_")
for strings.Contains(sanitized, "..") { for strings.Contains(sanitized, "..") {
sanitized = strings.ReplaceAll(sanitized, "..", "__") sanitized = strings.ReplaceAll(sanitized, "..", "__")
} }
sanitized = strings.Trim(sanitized, "._") sanitized = strings.Trim(sanitized, "._")
if sanitized == "" { if sanitized == "" {
return "", encoderErrorf("artifact type %q cannot produce a safe file name", artifactType) return "", encoderErrorf("lane id %q cannot produce a safe file name", laneID)
} }
return "artifacts/" + sanitized + ".json", nil return "lanes/" + sanitized + ".json", nil
} }
func cloneArtifact(artifact artifacts.Artifact) artifacts.Artifact { func cloneNormalizeOutputs(outputs []contracts.NormalizeOutput) []contracts.NormalizeOutput {
return artifacts.Artifact{ if len(outputs) == 0 {
ExtractorKey: artifact.ExtractorKey, return nil
ArtifactType: artifact.ArtifactType,
SchemaVersion: artifact.SchemaVersion,
Payload: append(stdjson.RawMessage(nil), artifact.Payload...),
SourceRefs: append([]source.SourceRef(nil), artifact.SourceRefs...),
Metadata: cloneMetadata(artifact.Metadata),
} }
} out := make([]contracts.NormalizeOutput, 0, len(outputs))
for _, output := range outputs {
func cloneRejected(rejected []artifacts.RejectedArtifact) []artifacts.RejectedArtifact { output.Payload = cloneRawPayload(output.Payload)
if len(rejected) == 0 { out = append(out, output)
return []artifacts.RejectedArtifact{}
}
out := make([]artifacts.RejectedArtifact, 0, len(rejected))
for _, item := range rejected {
out = append(out, artifacts.RejectedArtifact{
Candidate: cloneCandidate(item.Candidate),
ValidatorName: item.ValidatorName,
ReasonCode: item.ReasonCode,
Message: item.Message,
})
} }
return out return out
} }
func cloneCandidate(candidate artifacts.ArtifactCandidate) artifacts.ArtifactCandidate { func cloneRawPayload(payload contracts.RawPayload) contracts.RawPayload {
return artifacts.ArtifactCandidate{ return contracts.RawPayload{
Index: candidate.Index, Content: append([]byte(nil), payload.Content...),
ExtractorKey: candidate.ExtractorKey, MediaType: payload.MediaType,
ArtifactType: candidate.ArtifactType, Metadata: cloneMetadata(payload.Metadata),
SchemaVersion: candidate.SchemaVersion, Warnings: append([]contracts.Warning(nil), payload.Warnings...),
Payload: append(stdjson.RawMessage(nil), candidate.Payload...),
SourceRefs: append([]source.SourceRef(nil), candidate.SourceRefs...),
Metadata: cloneMetadata(candidate.Metadata),
} }
} }
func cloneRejected(rejected []contracts.RejectedOutput) []contracts.RejectedOutput {
if len(rejected) == 0 {
return []contracts.RejectedOutput{}
}
return append([]contracts.RejectedOutput(nil), rejected...)
}
func cloneWarnings(warnings []contracts.Warning) []contracts.Warning { func cloneWarnings(warnings []contracts.Warning) []contracts.Warning {
if len(warnings) == 0 { if len(warnings) == 0 {
return []contracts.Warning{} return []contracts.Warning{}

View File

@@ -8,7 +8,6 @@ import (
"testing" "testing"
"gitea.maximumdirect.net/eric/notarius/internal/core/artifacts" "gitea.maximumdirect.net/eric/notarius/internal/core/artifacts"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts" "gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline" "gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
) )
@@ -37,20 +36,24 @@ func TestModuleSpecAndRegister(t *testing.T) {
} }
} }
func TestEncodeReturnsLogicalFilesGroupedByArtifactType(t *testing.T) { func TestEncodeReturnsLogicalFilesForNormalizedOutputs(t *testing.T) {
req := contracts.OutputRequest{ req := contracts.OutputRequest{
Manifest: artifacts.RunManifest{RunID: "run-1", PipelineID: "pipeline-1"}, Manifest: artifacts.RunManifest{RunID: "run-1", PipelineID: "pipeline-1"},
Approved: []artifacts.Artifact{ NormalizeOutputs: []contracts.NormalizeOutput{
artifact("dnd.spell-cast", "first"), normalizeOutput("spells", `{"spell_casts":[{"spell":"Cure Wounds"}]}`),
artifact("notes/item", "item"), normalizeOutput("notes/items", `{"items":[{"name":"Torch"}]}`),
artifact("dnd.spell-cast", "second"),
}, },
Rejected: []artifacts.RejectedArtifact{ Rejected: []contracts.RejectedOutput{
{ {
Candidate: candidate("bad type", "bad"), Stage: "extract",
ValidatorName: "validator", LaneID: "spells",
ModuleKey: "dnd/spells",
ChunkID: "chunk-1",
ChunkIndex: 1,
ReasonCode: "invalid", ReasonCode: "invalid",
Message: "not accepted", Message: "not accepted",
AttemptCount: 1,
ValidatorName: "validator",
}, },
}, },
Warnings: []contracts.Warning{{ReasonCode: "warning", Message: "check source"}}, Warnings: []contracts.Warning{{ReasonCode: "warning", Message: "check source"}},
@@ -62,9 +65,9 @@ func TestEncodeReturnsLogicalFilesGroupedByArtifactType(t *testing.T) {
} }
wantNames := []string{ wantNames := []string{
"artifacts/dnd.spell-cast.json",
"artifacts/notes_item.json",
"index.json", "index.json",
"lanes/notes_items.json",
"lanes/spells.json",
"manifest.json", "manifest.json",
"rejected.json", "rejected.json",
"warnings.json", "warnings.json",
@@ -72,41 +75,36 @@ func TestEncodeReturnsLogicalFilesGroupedByArtifactType(t *testing.T) {
if got := outputFileNames(result.Files); !reflect.DeepEqual(got, wantNames) { if got := outputFileNames(result.Files); !reflect.DeepEqual(got, wantNames) {
t.Fatalf("file names = %#v, want %#v", got, wantNames) t.Fatalf("file names = %#v, want %#v", got, wantNames)
} }
for _, file := range result.Files { for _, file := range result.Files {
if file.ContentType != contentTypeJSON {
t.Fatalf("%s ContentType = %q, want %q", file.Name, file.ContentType, contentTypeJSON)
}
if !strings.HasSuffix(string(file.Bytes), "\n") { if !strings.HasSuffix(string(file.Bytes), "\n") {
t.Fatalf("%s does not end with newline: %q", file.Name, string(file.Bytes)) t.Fatalf("%s does not end with newline: %q", file.Name, string(file.Bytes))
} }
if !stdjson.Valid(file.Bytes) { if file.ContentType == contentTypeJSON && !stdjson.Valid(file.Bytes) {
t.Fatalf("%s has invalid JSON: %s", file.Name, file.Bytes) t.Fatalf("%s has invalid JSON: %s", file.Name, file.Bytes)
} }
} }
spellFile := decodeObject(t, fileBytes(t, result.Files, "artifacts/dnd.spell-cast.json")) spells := decodeObject(t, fileBytes(t, result.Files, "lanes/spells.json"))
if spellFile["artifact_type"] != "dnd.spell-cast" { spellCasts := spells["spell_casts"].([]any)
t.Fatalf("artifact_type = %#v, want dnd.spell-cast", spellFile["artifact_type"]) if spellCasts[0].(map[string]any)["spell"] != "Cure Wounds" {
} t.Fatalf("spells output = %#v, want raw normalized content", spells)
spells := spellFile["artifacts"].([]any)
if len(spells) != 2 {
t.Fatalf("len(spells) = %d, want 2", len(spells))
}
firstPayload := spells[0].(map[string]any)["payload"].(map[string]any)
secondPayload := spells[1].(map[string]any)["payload"].(map[string]any)
if firstPayload["name"] != "first" || secondPayload["name"] != "second" {
t.Fatalf("spell order payloads = %#v then %#v, want runner order", firstPayload, secondPayload)
} }
index := decodeObject(t, fileBytes(t, result.Files, "index.json")) index := decodeObject(t, fileBytes(t, result.Files, "index.json"))
artifactFiles := index["artifact_files"].([]any) outputFiles := index["output_files"].([]any)
if len(artifactFiles) != 2 { if len(outputFiles) != 2 {
t.Fatalf("len(index artifact_files) = %d, want 2", len(artifactFiles)) t.Fatalf("len(index output_files) = %d, want 2", len(outputFiles))
} }
firstIndex := artifactFiles[0].(map[string]any) firstIndex := outputFiles[0].(map[string]any)
secondIndex := artifactFiles[1].(map[string]any) secondIndex := outputFiles[1].(map[string]any)
if firstIndex["artifact_type"] != "dnd.spell-cast" || secondIndex["artifact_type"] != "notes/item" { if firstIndex["lane_id"] != "notes/items" || secondIndex["lane_id"] != "spells" {
t.Fatalf("artifact_files = %#v, want sorted by artifact type", artifactFiles) t.Fatalf("output_files = %#v, want sorted by lane id", outputFiles)
}
rejected := decodeObject(t, fileBytes(t, result.Files, "rejected.json"))
if got := rejected["rejected"].([]any); len(got) != 1 {
t.Fatalf("rejected = %#v, want one rejected output", got)
} }
} }
@@ -179,12 +177,64 @@ func TestEncodeIncludesManifestReferences(t *testing.T) {
} }
} }
func TestEncodeRejectsArtifactTypeWithoutSafeFileName(t *testing.T) { func TestEncodeIncludesManifestRawOutputProvenance(t *testing.T) {
result, err := New().Encode(context.Background(), contracts.OutputRequest{
Manifest: artifacts.RunManifest{
RunID: "run-1",
NormalizedOutputs: []artifacts.NormalizedOutputManifest{
{
LaneID: "spells",
ModuleKey: "noop",
SourceID: "source-1",
MediaType: contentTypeJSON,
Schema: artifacts.OutputSchemaProvenance{
ID: "schema-id",
Name: "schema-name",
Version: "v1",
},
},
},
RejectedOutputs: []artifacts.RejectedOutputManifest{
{
Stage: "extract",
LaneID: "spells",
ModuleKey: "dnd/spells",
ChunkID: "chunk-0",
ReasonCode: "raw_output_rejected",
AttemptCount: 2,
},
},
},
})
if err != nil {
t.Fatalf("Encode() error = %v, want nil", err)
}
manifest := decodeObject(t, fileBytes(t, result.Files, "manifest.json"))
normalized := manifest["normalized_outputs"].([]any)
if len(normalized) != 1 {
t.Fatalf("normalized_outputs = %#v, want one entry", normalized)
}
normalizedEntry := normalized[0].(map[string]any)
if normalizedEntry["lane_id"] != "spells" || normalizedEntry["media_type"] != contentTypeJSON {
t.Fatalf("normalized output manifest = %#v, want lane and media type", normalizedEntry)
}
rejected := manifest["rejected_outputs"].([]any)
if len(rejected) != 1 {
t.Fatalf("rejected_outputs = %#v, want one entry", rejected)
}
rejectedEntry := rejected[0].(map[string]any)
if rejectedEntry["attempt_count"] != float64(2) || rejectedEntry["chunk_id"] != "chunk-0" {
t.Fatalf("rejected output manifest = %#v, want attempt count and chunk", rejectedEntry)
}
}
func TestEncodeRejectsLaneIDWithoutSafeFileName(t *testing.T) {
_, err := New().Encode(context.Background(), contracts.OutputRequest{ _, err := New().Encode(context.Background(), contracts.OutputRequest{
Approved: []artifacts.Artifact{artifact("///", "unsafe")}, NormalizeOutputs: []contracts.NormalizeOutput{normalizeOutput("///", `{"value":true}`)},
}) })
if err == nil { if err == nil {
t.Fatal("Encode() error = nil, want unsafe artifact type error") t.Fatal("Encode() error = nil, want unsafe lane id error")
} }
if !strings.Contains(err.Error(), "json output encoder") || !strings.Contains(err.Error(), "safe file name") { if !strings.Contains(err.Error(), "json output encoder") || !strings.Contains(err.Error(), "safe file name") {
t.Fatalf("Encode() error = %q, want safe file name context", err.Error()) t.Fatalf("Encode() error = %q, want safe file name context", err.Error())
@@ -193,22 +243,59 @@ func TestEncodeRejectsArtifactTypeWithoutSafeFileName(t *testing.T) {
func TestEncodeSanitizesParentPathSequences(t *testing.T) { func TestEncodeSanitizesParentPathSequences(t *testing.T) {
result, err := New().Encode(context.Background(), contracts.OutputRequest{ result, err := New().Encode(context.Background(), contracts.OutputRequest{
Approved: []artifacts.Artifact{artifact("dnd..spell.", "spell")}, NormalizeOutputs: []contracts.NormalizeOutput{normalizeOutput("dnd..spell.", `{"value":true}`)},
}) })
if err != nil { if err != nil {
t.Fatalf("Encode() error = %v, want nil", err) t.Fatalf("Encode() error = %v, want nil", err)
} }
if got := outputFileNames(result.Files); !containsString(got, "artifacts/dnd__spell.json") { if got := outputFileNames(result.Files); !containsString(got, "lanes/dnd__spell.json") {
t.Fatalf("file names = %#v, want sanitized artifact filename", got) t.Fatalf("file names = %#v, want sanitized output filename", got)
}
}
func TestEncodeRejectsInvalidJSONAndUnsupportedMediaTypes(t *testing.T) {
tests := []struct {
name string
output contracts.NormalizeOutput
want string
}{
{
name: "invalid JSON",
output: normalizeOutput("spells", `{"spell_casts":[`),
want: "invalid JSON",
},
{
name: "unsupported media type",
output: func() contracts.NormalizeOutput {
output := normalizeOutput("spells", `{"spell_casts":[]}`)
output.Payload.MediaType = "text/plain"
return output
}(),
want: "unsupported media type",
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
_, err := New().Encode(context.Background(), contracts.OutputRequest{
NormalizeOutputs: []contracts.NormalizeOutput{test.output},
})
if err == nil {
t.Fatal("Encode() error = nil, want error")
}
if !strings.Contains(err.Error(), test.want) {
t.Fatalf("Encode() error = %q, want %q", err.Error(), test.want)
}
})
} }
} }
func TestEncodeRejectsSanitizedFilenameCollisions(t *testing.T) { func TestEncodeRejectsSanitizedFilenameCollisions(t *testing.T) {
_, err := New().Encode(context.Background(), contracts.OutputRequest{ _, err := New().Encode(context.Background(), contracts.OutputRequest{
Approved: []artifacts.Artifact{ NormalizeOutputs: []contracts.NormalizeOutput{
artifact("a/b", "slash"), normalizeOutput("a/b", `{"value":"slash"}`),
artifact("a?b", "question"), normalizeOutput("a?b", `{"value":"question"}`),
}, },
}) })
if err == nil { if err == nil {
@@ -222,16 +309,11 @@ func TestEncodeRejectsSanitizedFilenameCollisions(t *testing.T) {
func TestEncodeDoesNotMutateInputs(t *testing.T) { func TestEncodeDoesNotMutateInputs(t *testing.T) {
req := contracts.OutputRequest{ req := contracts.OutputRequest{
Manifest: artifacts.RunManifest{RunID: "run-1"}, Manifest: artifacts.RunManifest{RunID: "run-1"},
Approved: []artifacts.Artifact{ NormalizeOutputs: []contracts.NormalizeOutput{
artifact("dnd.spell", "original"), normalizeOutput("spells", `{"name":"original"}`),
}, },
Rejected: []artifacts.RejectedArtifact{ Rejected: []contracts.RejectedOutput{
{ {Stage: "extract", LaneID: "spells", Message: "not accepted"},
Candidate: candidate("bad", "rejected"),
ValidatorName: "validator",
ReasonCode: "invalid",
Message: "not accepted",
},
}, },
Warnings: []contracts.Warning{{ReasonCode: "warning", Message: "message"}}, Warnings: []contracts.Warning{{ReasonCode: "warning", Message: "message"}},
} }
@@ -246,14 +328,13 @@ func TestEncodeDoesNotMutateInputs(t *testing.T) {
t.Fatalf("request mutated:\nbefore: %s\nafter: %s", before, after) t.Fatalf("request mutated:\nbefore: %s\nafter: %s", before, after)
} }
req.Approved[0].Payload[0] = '[' req.NormalizeOutputs[0].Payload.Content[0] = '['
req.Approved[0].SourceRefs[0].StartUnitID = "changed" req.NormalizeOutputs[0].Payload.Metadata["name"] = "changed"
req.Approved[0].Metadata["name"] = "changed" req.Rejected[0].Message = "changed"
req.Rejected[0].Candidate.Payload[0] = '['
req.Warnings[0].Message = "changed" req.Warnings[0].Message = "changed"
if !stdjson.Valid(fileBytes(t, result.Files, "artifacts/dnd.spell.json")) { if !stdjson.Valid(fileBytes(t, result.Files, "lanes/spells.json")) {
t.Fatal("artifact output changed after request mutation") t.Fatal("output changed after request mutation")
} }
warnings := decodeObject(t, fileBytes(t, result.Files, "warnings.json")) warnings := decodeObject(t, fileBytes(t, result.Files, "warnings.json"))
gotWarnings := warnings["warnings"].([]any) gotWarnings := warnings["warnings"].([]any)
@@ -262,9 +343,9 @@ func TestEncodeDoesNotMutateInputs(t *testing.T) {
} }
} }
func TestArtifactFilesDoNotContainWarnings(t *testing.T) { func TestOutputFilesDoNotContainWarnings(t *testing.T) {
result, err := New().Encode(context.Background(), contracts.OutputRequest{ result, err := New().Encode(context.Background(), contracts.OutputRequest{
Approved: []artifacts.Artifact{artifact("dnd.spell", "spell")}, NormalizeOutputs: []contracts.NormalizeOutput{normalizeOutput("spells", `{"spell":"Shield"}`)},
Warnings: []contracts.Warning{ Warnings: []contracts.Warning{
{ReasonCode: "pipeline-warning", Message: "warning"}, {ReasonCode: "pipeline-warning", Message: "warning"},
}, },
@@ -273,36 +354,27 @@ func TestArtifactFilesDoNotContainWarnings(t *testing.T) {
t.Fatalf("Encode() error = %v, want nil", err) t.Fatalf("Encode() error = %v, want nil", err)
} }
artifactFile := decodeObject(t, fileBytes(t, result.Files, "artifacts/dnd.spell.json")) outputFile := decodeObject(t, fileBytes(t, result.Files, "lanes/spells.json"))
if _, ok := artifactFile["warnings"]; ok { if _, ok := outputFile["warnings"]; ok {
t.Fatalf("artifact file contains warnings: %#v", artifactFile) t.Fatalf("output file contains warnings: %#v", outputFile)
} }
} }
func artifact(artifactType, name string) artifacts.Artifact { func normalizeOutput(laneID string, content string) contracts.NormalizeOutput {
return artifacts.Artifact{ return contracts.NormalizeOutput{
ExtractorKey: "extractor", LaneID: laneID,
ArtifactType: artifactType, NormalizerKey: "noop",
SchemaVersion: "v1", SourceID: "source-1",
Payload: stdjson.RawMessage(`{"name":"` + name + `"}`), Schema: contracts.ResponseSchema{
SourceRefs: []source.SourceRef{ ID: "schema-id",
{SourceID: "source-1", StartUnitID: "u1", EndUnitID: "u1"}, Name: "schema-name",
Version: "v1",
}, },
Metadata: map[string]any{"name": name}, Payload: contracts.RawPayload{
} Content: []byte(content),
} MediaType: contentTypeJSON,
Metadata: map[string]any{"name": laneID},
func candidate(artifactType, name string) artifacts.ArtifactCandidate {
return artifacts.ArtifactCandidate{
Index: 1,
ExtractorKey: "extractor",
ArtifactType: artifactType,
SchemaVersion: "v1",
Payload: stdjson.RawMessage(`{"name":"` + name + `"}`),
SourceRefs: []source.SourceRef{
{SourceID: "source-1", StartUnitID: "u1", EndUnitID: "u1"},
}, },
Metadata: map[string]any{"name": name},
} }
} }

View File

@@ -11,9 +11,8 @@ import (
) )
type UnitRef struct { type UnitRef struct {
value string value int
fromNumber bool fromNumber bool
number int
} }
type SourceRefResponse struct { type SourceRefResponse struct {
@@ -23,19 +22,22 @@ type SourceRefResponse struct {
} }
func UnitRefFromString(value string) UnitRef { func UnitRefFromString(value string) UnitRef {
return UnitRef{value: value} parsed, _ := parseUnitRefNumber(value)
return UnitRef{value: parsed}
} }
func UnitRefFromInt(value int) UnitRef { func UnitRefFromInt(value int) UnitRef {
return UnitRef{ return UnitRef{
value: strconv.Itoa(value), value: value,
fromNumber: true, fromNumber: true,
number: value,
} }
} }
func (ref UnitRef) String() string { func (ref UnitRef) String() string {
return ref.value if ref.value == 0 {
return ""
}
return strconv.Itoa(ref.value)
} }
func (ref *UnitRef) UnmarshalJSON(raw []byte) error { func (ref *UnitRef) UnmarshalJSON(raw []byte) error {
@@ -48,13 +50,17 @@ func (ref *UnitRef) UnmarshalJSON(raw []byte) error {
if err := json.Unmarshal(raw, &value); err != nil { if err := json.Unmarshal(raw, &value); err != nil {
return err return err
} }
*ref = UnitRefFromString(value) number, err := parseUnitRefNumber(value)
if err != nil {
return err
}
*ref = UnitRef{value: number}
return nil return nil
} }
number, err := strconv.Atoi(string(raw)) number, err := parseUnitRefNumber(string(raw))
if err != nil { if err != nil {
return fmt.Errorf("unit ref must be a string or integer") return err
} }
*ref = UnitRefFromInt(number) *ref = UnitRefFromInt(number)
return nil return nil
@@ -62,72 +68,47 @@ func (ref *UnitRef) UnmarshalJSON(raw []byte) error {
func (ref UnitRef) MarshalJSON() ([]byte, error) { func (ref UnitRef) MarshalJSON() ([]byte, error) {
if ref.fromNumber { if ref.fromNumber {
return []byte(strconv.Itoa(ref.number)), nil return []byte(strconv.Itoa(ref.value)), nil
} }
return json.Marshal(ref.value) return json.Marshal(ref.String())
} }
func ResolveUnitID(doc *source.SourceDocument, field string, ref UnitRef) (string, error) { func ResolveUnitID(doc *source.SourceDocument, field string, ref UnitRef) (int, error) {
value := strings.TrimSpace(ref.value) if ref.value <= 0 {
if value == "" { return 0, fmt.Errorf("%s must be positive", field)
return "", fmt.Errorf("%s must not be empty", field)
} }
if id, ok := canonicalUnitID(doc, value); ok { if _, ok := source.UnitIndex(doc, ref.value); !ok {
return id, nil return 0, fmt.Errorf("%s %d was not found", field, ref.value)
} }
if number, ok := unitNumber(value); ok { return ref.value, nil
if id, ok := unitIDByNumber(doc, number); ok {
return id, nil
}
return "", fmt.Errorf("%s %d was not found as a source-unit ID or 1-based unit number", field, number)
}
return "", fmt.Errorf("%s %q was not found", field, value)
} }
func SourceRefCandidate(doc *source.SourceDocument, ref SourceRefResponse) source.SourceRef { func SourceRefCandidate(doc *source.SourceDocument, ref SourceRefResponse) source.SourceRef {
return source.SourceRef{ return source.SourceRef{
SourceID: strings.TrimSpace(ref.SourceID), SourceID: strings.TrimSpace(ref.SourceID),
StartUnitID: unitIDCandidate(doc, ref.StartUnitID), StartUnitID: unitIDCandidate(ref.StartUnitID),
EndUnitID: unitIDCandidate(doc, ref.EndUnitID), EndUnitID: unitIDCandidate(ref.EndUnitID),
} }
} }
func unitIDCandidate(doc *source.SourceDocument, ref UnitRef) string { func unitIDCandidate(ref UnitRef) int {
value := strings.TrimSpace(ref.value) return ref.value
if id, ok := canonicalUnitID(doc, value); ok {
return id
}
if number, ok := unitNumber(value); ok {
if id, ok := unitIDByNumber(doc, number); ok {
return id
}
}
return value
} }
func canonicalUnitID(doc *source.SourceDocument, value string) (string, bool) { func parseUnitRefNumber(value string) (int, error) {
if doc == nil { trimmed := strings.TrimSpace(value)
return "", false if trimmed == "" {
return 0, fmt.Errorf("unit ref must not be empty")
} }
for _, unit := range doc.Units { if trimmed != value {
if unit.ID == value { return 0, fmt.Errorf("unit ref must not contain leading or trailing whitespace")
return unit.ID, true
}
} }
return "", false
}
func unitIDByNumber(doc *source.SourceDocument, number int) (string, bool) {
if doc == nil || number < 1 || number > len(doc.Units) {
return "", false
}
return doc.Units[number-1].ID, true
}
func unitNumber(value string) (int, bool) {
number, err := strconv.Atoi(value) number, err := strconv.Atoi(value)
if err != nil { if err != nil {
return 0, false return 0, fmt.Errorf("unit ref must be an integer")
} }
return number, true if number <= 0 {
return 0, fmt.Errorf("unit ref must be positive")
}
return number, nil
} }

View File

@@ -8,7 +8,7 @@ import (
"gitea.maximumdirect.net/eric/notarius/internal/core/source" "gitea.maximumdirect.net/eric/notarius/internal/core/source"
) )
func TestUnitRefUnmarshalAcceptsIntegerAndString(t *testing.T) { func TestUnitRefUnmarshalAcceptsIntegerAndNumericString(t *testing.T) {
var integerRef UnitRef var integerRef UnitRef
if err := json.Unmarshal([]byte(`12`), &integerRef); err != nil { if err := json.Unmarshal([]byte(`12`), &integerRef); err != nil {
t.Fatalf("Unmarshal(integer) error = %v, want nil", err) t.Fatalf("Unmarshal(integer) error = %v, want nil", err)
@@ -18,55 +18,49 @@ func TestUnitRefUnmarshalAcceptsIntegerAndString(t *testing.T) {
} }
var stringRef UnitRef var stringRef UnitRef
if err := json.Unmarshal([]byte(`"seg-001"`), &stringRef); err != nil { if err := json.Unmarshal([]byte(`"12"`), &stringRef); err != nil {
t.Fatalf("Unmarshal(string) error = %v, want nil", err) t.Fatalf("Unmarshal(string) error = %v, want nil", err)
} }
if got := stringRef.String(); got != "seg-001" { if got := stringRef.String(); got != "12" {
t.Fatalf("string ref = %q, want seg-001", got) t.Fatalf("string ref = %q, want 12", got)
} }
} }
func TestUnitRefUnmarshalRejectsNonIntegerTypes(t *testing.T) { func TestUnitRefUnmarshalRejectsNonIntegerValues(t *testing.T) {
for _, raw := range []string{`true`, `null`, `1.5`, `{}`} { for _, raw := range []string{`true`, `null`, `1.5`, `{}`, `"seg-001"`, `" 1 "`, `0`, `-1`} {
t.Run(raw, func(t *testing.T) { t.Run(raw, func(t *testing.T) {
var ref UnitRef var ref UnitRef
err := json.Unmarshal([]byte(raw), &ref) err := json.Unmarshal([]byte(raw), &ref)
if err == nil { if err == nil {
t.Fatal("Unmarshal() error = nil, want error") t.Fatal("Unmarshal() error = nil, want error")
} }
if !strings.Contains(err.Error(), "string or integer") {
t.Fatalf("Unmarshal() error = %q, want type context", err.Error())
}
}) })
} }
} }
func TestResolveUnitIDPrefersExactSourceUnitID(t *testing.T) { func TestResolveUnitIDReturnsExistingIntegerSourceUnitID(t *testing.T) {
doc := unitRefSourceDocument("2", "10") doc := unitRefSourceDocument(2, 10)
got, err := ResolveUnitID(doc, "start_unit_id", UnitRefFromInt(2)) got, err := ResolveUnitID(doc, "start_unit_id", UnitRefFromInt(2))
if err != nil { if err != nil {
t.Fatalf("ResolveUnitID() error = %v, want nil", err) t.Fatalf("ResolveUnitID() error = %v, want nil", err)
} }
if got != "2" { if got != 2 {
t.Fatalf("ResolveUnitID() = %q, want exact source unit ID", got) t.Fatalf("ResolveUnitID() = %d, want exact source unit ID", got)
} }
} }
func TestResolveUnitIDFallsBackToOneBasedUnitNumber(t *testing.T) { func TestResolveUnitIDDoesNotFallbackToOneBasedUnitNumber(t *testing.T) {
doc := unitRefSourceDocument("seg-001", "seg-002") doc := unitRefSourceDocument(10, 20)
got, err := ResolveUnitID(doc, "end_unit_id", UnitRefFromInt(2)) _, err := ResolveUnitID(doc, "end_unit_id", UnitRefFromInt(2))
if err != nil { if err == nil {
t.Fatalf("ResolveUnitID() error = %v, want nil", err) t.Fatal("ResolveUnitID() error = nil, want missing source-unit ID")
}
if got != "seg-002" {
t.Fatalf("ResolveUnitID() = %q, want second source unit ID", got)
} }
} }
func TestResolveUnitIDRejectsMissingUnit(t *testing.T) { func TestResolveUnitIDRejectsMissingUnit(t *testing.T) {
doc := unitRefSourceDocument("seg-001") doc := unitRefSourceDocument(1)
_, err := ResolveUnitID(doc, "start_unit_id", UnitRefFromInt(9)) _, err := ResolveUnitID(doc, "start_unit_id", UnitRefFromInt(9))
if err == nil { if err == nil {
@@ -78,14 +72,14 @@ func TestResolveUnitIDRejectsMissingUnit(t *testing.T) {
} }
func TestSourceRefCandidateCanonicalizesValidRefsAndPreservesInvalidRefs(t *testing.T) { func TestSourceRefCandidateCanonicalizesValidRefsAndPreservesInvalidRefs(t *testing.T) {
doc := unitRefSourceDocument("seg-001", "seg-002") doc := unitRefSourceDocument(1, 2)
valid := SourceRefCandidate(doc, SourceRefResponse{ valid := SourceRefCandidate(doc, SourceRefResponse{
SourceID: " session-alpha ", SourceID: " session-alpha ",
StartUnitID: UnitRefFromInt(1), StartUnitID: UnitRefFromInt(1),
EndUnitID: UnitRefFromInt(2), EndUnitID: UnitRefFromInt(2),
}) })
if valid != (source.SourceRef{SourceID: "session-alpha", StartUnitID: "seg-001", EndUnitID: "seg-002"}) { if valid != (source.SourceRef{SourceID: "session-alpha", StartUnitID: 1, EndUnitID: 2}) {
t.Fatalf("valid candidate = %#v, want canonical source ref", valid) t.Fatalf("valid candidate = %#v, want canonical source ref", valid)
} }
@@ -94,12 +88,12 @@ func TestSourceRefCandidateCanonicalizesValidRefsAndPreservesInvalidRefs(t *test
StartUnitID: UnitRefFromInt(9), StartUnitID: UnitRefFromInt(9),
EndUnitID: UnitRefFromString("missing"), EndUnitID: UnitRefFromString("missing"),
}) })
if invalid != (source.SourceRef{SourceID: "session-alpha", StartUnitID: "9", EndUnitID: "missing"}) { if invalid != (source.SourceRef{SourceID: "session-alpha", StartUnitID: 9, EndUnitID: 0}) {
t.Fatalf("invalid candidate = %#v, want unresolved values for validator", invalid) t.Fatalf("invalid candidate = %#v, want unresolved values for validator", invalid)
} }
} }
func unitRefSourceDocument(ids ...string) *source.SourceDocument { func unitRefSourceDocument(ids ...int) *source.SourceDocument {
doc := &source.SourceDocument{ doc := &source.SourceDocument{
ID: "session-alpha", ID: "session-alpha",
Kind: "transcript", Kind: "transcript",