Compare commits
5 Commits
ad1cba41c2
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
| 916532100d | |||
| bef8ca263b | |||
| f6d037b613 | |||
| 449b506804 | |||
| 071a78ae22 |
@@ -39,6 +39,8 @@ demonstrates all implemented D&D lanes and the supporting campaign references.
|
||||
artifact formats.
|
||||
- [Subprocess consumer guide](docs/consumers/subprocess.md) — invoke Notarius
|
||||
from an orchestrator and consume a published result.
|
||||
- [Complete D&D consumer guide](docs/consumers/dnd-pipeline.md) — run the full
|
||||
D&D pipeline as a subprocess and discover its structured artifacts.
|
||||
- [Internal overview](docs/internal/overview.md) — implemented component map
|
||||
for maintainers.
|
||||
- [Developer guide](docs/development.md) — contributor orientation and
|
||||
|
||||
@@ -319,7 +319,8 @@ Unknown outer or nested option fields are rejected, as are incompatible YAML
|
||||
types. The allowlist remains valid when a run uses lane filtering: a configured
|
||||
lane that is not active for that invocation simply contributes no evidence.
|
||||
Evidence publication is opt-in because it can persist source text and metadata.
|
||||
Its payload contract is [Published Evidence Context](integrations/evidence-context.md).
|
||||
When enabled, it publishes the selected source-unit excerpt defined by the
|
||||
[Published Evidence Context contract](integrations/evidence-context.md).
|
||||
|
||||
## References And Ordered Handoffs
|
||||
|
||||
|
||||
202
docs/consumers/dnd-pipeline.md
Normal file
202
docs/consumers/dnd-pipeline.md
Normal file
@@ -0,0 +1,202 @@
|
||||
# Consuming The Complete D&D Pipeline
|
||||
|
||||
Use this workflow when an orchestrator runs the maintained complete D&D
|
||||
pipeline and consumes its structured JSON artifacts. The generic
|
||||
[subprocess consumer guide](subprocess.md) owns process-level responsibilities;
|
||||
this guide connects that workflow to the complete D&D configuration, its
|
||||
Seriatim input, and its artifact inventory.
|
||||
|
||||
The [CLI reference](../cli.md), [configuration reference](../config.md),
|
||||
[run-result receipt](../integrations/run-result.md), and
|
||||
[published JSON output contract](../integrations/json-output.md) remain the
|
||||
canonical definitions of those public interfaces.
|
||||
|
||||
## Prepare And Validate The Deployment
|
||||
|
||||
Start from the maintained
|
||||
[complete D&D configuration](../../examples/dnd-complete.config.yml). It uses
|
||||
the `dnd-session` pipeline and demonstrates every implemented D&D lane, ordered
|
||||
artifact handoffs, campaign references, chunk-map publication, and evidence
|
||||
context.
|
||||
|
||||
A deployment must provide its own PromptKit profile and campaign reference
|
||||
files. Use absolute paths for service and subprocess deployments. In
|
||||
particular, observe these different resolution rules:
|
||||
|
||||
- reference paths in YAML are resolved relative to the Notarius configuration
|
||||
file; and
|
||||
- `promptkit.profile_file` is resolved relative to the Notarius process working
|
||||
directory.
|
||||
|
||||
Do not copy the repository example's relative profile path into a deployment
|
||||
without also controlling that working directory. The complete path and profile
|
||||
rules are defined in [Configuration](../config.md).
|
||||
|
||||
Preflight the deployed configuration before processing sessions and whenever
|
||||
it changes:
|
||||
|
||||
```sh
|
||||
notarius config validate \
|
||||
--config /absolute/path/to/notarius.yml \
|
||||
--pipeline dnd-session
|
||||
```
|
||||
|
||||
Provide credentials through the environment or the documented configuration
|
||||
mechanism. Do not put credentials in command arguments, generated
|
||||
configuration, or logs.
|
||||
|
||||
## Supply The Transcript
|
||||
|
||||
The complete pipeline consumes a Seriatim JSON document. The
|
||||
[Seriatim input contract](../integrations/seriatim.md) defines its required
|
||||
metadata, segments, and validation rules. Preserve segment IDs: D&D artifact
|
||||
citations use those segment IDs as source-unit ranges.
|
||||
|
||||
When the caller maintains several transcript tiers, use the final trimmed JSON
|
||||
transcript so extraction operates on the same session content presented to
|
||||
later consumers. For example, Narratio identifies this implemented artifact as
|
||||
`narratio.transcript.final_trimmed` and normally stores it at
|
||||
`transcripts/final.trimmed.json`.
|
||||
|
||||
Notarius generates a stable prompt session from the resolved input module and
|
||||
the exact input bytes. An ordinary orchestrator should not pass `--session-id`.
|
||||
Use that override only when intentionally changing the routing relationship
|
||||
between invocations; it is not a credential or output identity.
|
||||
|
||||
## Run Notarius
|
||||
|
||||
Invoke the pipeline with explicit absolute paths and request its
|
||||
machine-readable receipt:
|
||||
|
||||
```sh
|
||||
notarius run dnd-session \
|
||||
--config /absolute/path/to/notarius.yml \
|
||||
--input /absolute/path/to/transcripts/final.trimmed.json \
|
||||
--output-dir /absolute/path/to/notarius-output \
|
||||
--json
|
||||
```
|
||||
|
||||
The caller should:
|
||||
|
||||
- capture stdout and stderr separately;
|
||||
- propagate cancellation and impose an operator-appropriate timeout;
|
||||
- wait for process completion before interpreting stdout; and
|
||||
- retain stderr for diagnosis without copying secrets or transcript content
|
||||
into other logs.
|
||||
|
||||
Only exit status 0 permits decoding stdout as a receipt. Ignore stdout after a
|
||||
nonzero exit because a failed receipt write can leave partial bytes. The
|
||||
[CLI reference](../cli.md#output-streams-and-exit-statuses) defines the complete
|
||||
stream and exit-status contract.
|
||||
|
||||
## Discover The Published Bundle
|
||||
|
||||
Decode the successful stdout document as a supported run-result schema. For
|
||||
the current contract, `schema_version` is `notarius.run-result.v1`. Tolerate
|
||||
unknown fields allowed by that version, but reject an unsupported schema
|
||||
version.
|
||||
|
||||
Use the receipt's absolute `output_directory` as the exact run-specific bundle
|
||||
root. Do not scan the output root for its newest directory, guess a run ID, or
|
||||
construct a bundle path. Resolve `index_file` beneath `output_directory` and
|
||||
reject an absolute logical path or any result that escapes the bundle root.
|
||||
|
||||
Read `index.json` and locate each requested lane in `output_files` by its exact
|
||||
`lane_id`. Do not guess a lane filename. Before decoding a payload:
|
||||
|
||||
1. resolve its descriptor's relative `file` beneath the bundle root with the
|
||||
same confinement check;
|
||||
2. verify the descriptor's media type and schema identity against the linked
|
||||
artifact contract; and
|
||||
3. decode the payload according to that contract.
|
||||
|
||||
The [published JSON output contract](../integrations/json-output.md) defines
|
||||
the index and bundle layout. Treat all paths obtained from a decoded external
|
||||
document as untrusted until confined to their documented root.
|
||||
|
||||
## Complete Artifact Inventory
|
||||
|
||||
When every configured lane is accepted, the complete example publishes these
|
||||
lane artifacts:
|
||||
|
||||
| Lane ID | Purpose | Canonical contract |
|
||||
| --- | --- | --- |
|
||||
| `item-registry` | Canonical registry of encountered items and currency. | [Item registry](../integrations/dnd-item-registry-artifacts.md) |
|
||||
| `npc-registry` | Canonical registry of named NPCs. | [NPC registry](../integrations/dnd-npc-registry-artifacts.md) |
|
||||
| `location-registry` | Canonical registry of named locations. | [Location registry](../integrations/dnd-location-registry-artifacts.md) |
|
||||
| `scene-descriptions` | Classification, title, and summary for each scene. | [Scene descriptions](../integrations/dnd-scene-description-artifacts.md) |
|
||||
| `item-occurrences` | Source-grounded item discovery, acquisition, use, transfer, and loss events. | [Item occurrences](../integrations/dnd-item-occurrence-artifacts.md) |
|
||||
| `spells` | Source-grounded spell casts and casters. | [Spell casts](../integrations/dnd-spell-artifacts.md) |
|
||||
| `combat-turns` | Source-grounded combat turn participation. | [Combat turns](../integrations/dnd-combat-turn-artifacts.md) |
|
||||
| `npc-occurrences` | Source-grounded NPC interaction occurrences. | [NPC occurrences](../integrations/dnd-npc-occurrence-artifacts.md) |
|
||||
| `location-occurrences` | Source-grounded location occurrences. | [Location occurrences](../integrations/dnd-location-occurrence-artifacts.md) |
|
||||
| `enemy-events` | Source-grounded enemy combat events. | [Enemy events](../integrations/dnd-enemy-event-artifacts.md) |
|
||||
|
||||
The JSON encoder always publishes these bundle-management files:
|
||||
|
||||
| File | Purpose |
|
||||
| --- | --- |
|
||||
| `index.json` | Discovery document for lane and pipeline-wide artifacts. |
|
||||
| `manifest.json` | Run provenance and result summaries. |
|
||||
| `rejected.json` | Rejected pipeline outputs. |
|
||||
| `warnings.json` | Accepted-output and run warnings. |
|
||||
|
||||
The complete configuration also requests two pipeline-wide artifacts:
|
||||
|
||||
- [`chunk-map.json`](../integrations/chunk-map.md), the accepted chunk plan and
|
||||
chunk metadata; and
|
||||
- [`evidence-context.json`](../integrations/evidence-context.md), a reading
|
||||
excerpt containing the union of selected cited source units and the
|
||||
configured surrounding window.
|
||||
|
||||
Discover both from their top-level `index.json` descriptors rather than
|
||||
treating them as lanes. Evidence context is convenient reading material, not
|
||||
authoritative provenance; citations in the normalized lane payloads remain the
|
||||
evidence contract.
|
||||
|
||||
Every optional or lane file is published only when its corresponding artifact
|
||||
is available. A successful process does not guarantee that all configured
|
||||
lanes were accepted.
|
||||
|
||||
## Decide What Counts As Consumer Success
|
||||
|
||||
Exit status 0 means Notarius completed the pipeline and published its result
|
||||
bundle. The receipt or bundle may still report warnings, rejected outputs, or
|
||||
missing lane descriptors. A downstream consumer must define its own required
|
||||
artifact set explicitly.
|
||||
|
||||
A caller that claims to consume the complete D&D workflow should normally
|
||||
require all ten lane IDs in the table and verify each descriptor's expected
|
||||
contract. If any required lane is missing, rejected, or incompatible, fail the
|
||||
caller's extraction step while retaining the Notarius bundle for diagnosis. A
|
||||
consumer that needs only a subset may define and document a narrower policy.
|
||||
|
||||
Keep the successful receipt with the complete published bundle. Retain
|
||||
`manifest.json`, `rejected.json`, `warnings.json`, and captured process logs as
|
||||
required by the caller's provenance, diagnosis, and retention policies. Avoid
|
||||
selectively copying payload files without also preserving enough index and
|
||||
manifest information to identify their originating run and contracts.
|
||||
|
||||
The transcript, lane artifacts, evidence context, manifest, debug data, and
|
||||
logs can all contain private campaign information. Apply the same access,
|
||||
publication, and retention controls used for the source transcript.
|
||||
|
||||
## Consumer Checklist
|
||||
|
||||
- Validate the deployed Notarius configuration and `dnd-session` pipeline.
|
||||
- Pass the final trimmed Seriatim JSON transcript with stable segment IDs.
|
||||
- Use absolute configuration, input, output-root, profile, and reference paths
|
||||
in service deployments.
|
||||
- Capture stdout and stderr separately and enforce cancellation and timeout.
|
||||
- Parse stdout only after exit status 0.
|
||||
- Accept only supported receipt, index, and artifact schema versions while
|
||||
tolerating permitted unknown fields.
|
||||
- Use the receipt's `output_directory`; never guess the run directory.
|
||||
- Confine `index_file` and every descriptor path to the published bundle root.
|
||||
- Discover lanes by `lane_id` and verify descriptor compatibility before
|
||||
decoding payloads.
|
||||
- Enforce an explicit required-lane policy and inspect rejections and warnings.
|
||||
- Preserve the receipt and sufficient bundle provenance for every retained
|
||||
artifact.
|
||||
- Protect all transcript-derived files and diagnostic streams as sensitive
|
||||
campaign data.
|
||||
@@ -6,6 +6,10 @@ statuses, while the [run-result receipt](../integrations/run-result.md) and
|
||||
[Published JSON Output contract](../integrations/json-output.md) own the
|
||||
durable result formats.
|
||||
|
||||
For the maintained complete D&D workflow, including its transcript input,
|
||||
configured lane inventory, and downstream acceptance checklist, see
|
||||
[Consuming The Complete D&D Pipeline](dnd-pipeline.md).
|
||||
|
||||
## Run And Check The Process
|
||||
|
||||
Optionally preflight a selected configuration and pipeline before work starts:
|
||||
@@ -55,10 +59,10 @@ contract. The JSON bundle contract links to the available lane contracts.
|
||||
If `index.json` has an `evidence_context` descriptor, treat it as a
|
||||
pipeline-wide artifact rather than a lane entry. Verify its six descriptor
|
||||
fields before decoding the linked file according to the [Published Evidence
|
||||
Context contract](../integrations/evidence-context.md). Use each
|
||||
`evidence_refs` entry as the citation to source material. Its surrounding
|
||||
context range and included units explain the citation, but do not widen or
|
||||
replace the cited source reference.
|
||||
Context contract](../integrations/evidence-context.md). Decode its top-level
|
||||
source-unit array as a reading excerpt. Obtain authoritative citations and lane
|
||||
provenance from the normalized lane artifacts; the excerpt has neither and its
|
||||
nearby units do not widen a lane artifact's cited source reference.
|
||||
|
||||
A zero exit status may still report rejected outputs, warnings, or absent
|
||||
lanes. The caller decides which lane IDs are required for its own work and
|
||||
@@ -73,5 +77,5 @@ them. Treat the input, output bundle, cache, debug bundle, and captured process
|
||||
logs as potentially sensitive data. Apply the caller's access controls and
|
||||
retention policy, and avoid copying secrets into arguments, logs, or
|
||||
provenance records. An evidence-context artifact contains source-unit text and
|
||||
metadata, and selected lanes can cover most of an input; preserve and share it
|
||||
only when that source content is authorized for the recipient.
|
||||
metadata and can cover most of an input; preserve and share it only when that
|
||||
source content is authorized for the recipient.
|
||||
|
||||
@@ -18,7 +18,7 @@ implemented component map.
|
||||
| Any documentation addition or revision | [Documentation Policy](policy/documentation.md) | It defines canonical homes, audiences, current-behavior rules, and maintenance requirements. |
|
||||
| Adding, changing, reviewing, or deleting tests | [Testing Policy](policy/testing.md) | It defines risk-based sufficiency, durable test boundaries, test-double guidance, and criteria for retaining tests. |
|
||||
| CLI composition or command behavior | [CLI Internals](internal/cli.md) and [CLI Reference](cli.md) | The internal guide owns composition and command flow; the reference owns public syntax. |
|
||||
| Building a subprocess caller or changing its result protocol | [Subprocess Consumer Guide](consumers/subprocess.md), [Run Result Receipt](integrations/run-result.md), and [CLI Internals](internal/cli.md) | These separate caller workflow, durable receipt contract, and CLI implementation behavior. |
|
||||
| Building a subprocess caller or changing its result protocol | [Subprocess Consumer Guide](consumers/subprocess.md), [Complete D&D Consumer Guide](consumers/dnd-pipeline.md), [Run Result Receipt](integrations/run-result.md), and [CLI Internals](internal/cli.md) | These separate generic caller workflow, the complete D&D workflow, the durable receipt contract, and CLI implementation behavior. |
|
||||
| Configuration loading, resolution, or user-visible configuration behavior | [Configuration Internals](internal/configuration.md) and [Configuration](config.md) | The internal guide owns loading and resolution mechanics; the reference owns the configuration contract. |
|
||||
| Pipeline resolution or execution | [Pipeline Internals](internal/pipeline.md) | It documents profiles, references, validation, retries, checkpoints, and runner behavior. |
|
||||
| Production modules or validators | [Module Internals](internal/modules.md), [D&D Module Internals](internal/dnd.md), and [D&D integration contracts](integrations/) | The generic guide owns extension mechanics, the D&D guide owns shared family conventions, and the contracts own durable output shapes. |
|
||||
|
||||
@@ -1,9 +1,11 @@
|
||||
# Published Evidence Context
|
||||
|
||||
This contract defines the optional `source/evidence-context` artifact emitted
|
||||
by the production JSON output. Its configuration is owned by
|
||||
[Configuration](../config.md#module-bindings-and-validators); its logical-file
|
||||
discovery is owned by [Published JSON Output](json-output.md).
|
||||
by the production JSON output. It is a selected source-unit excerpt for
|
||||
convenient reading alongside normalized lane artifacts; it is not a second
|
||||
citation or provenance model. Its configuration is owned by
|
||||
[Configuration](../config.md#module-bindings-and-validators), and its
|
||||
logical-file discovery is owned by [Published JSON Output](json-output.md).
|
||||
|
||||
## Identity And Discovery
|
||||
|
||||
@@ -26,91 +28,80 @@ its absence means evidence publication was not enabled for that bundle.
|
||||
|
||||
## Payload
|
||||
|
||||
The v1 payload is a JSON object with required `source_id`, `source_digest`,
|
||||
`window_units`, `selected_lanes`, and `contexts` fields. `selected_lanes` and
|
||||
`contexts` are always arrays; an enabled configuration with no accepted direct
|
||||
evidence publishes `contexts: []`.
|
||||
The v1 payload is a top-level JSON array of generic source units. There is no
|
||||
wrapper, source-level metadata, context grouping, lane identifier, or evidence
|
||||
reference in the payload. An enabled configuration with no contributing
|
||||
accepted evidence publishes `[]`.
|
||||
|
||||
```json
|
||||
{
|
||||
"source_id": "session-alpha",
|
||||
"source_digest": "sha256:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef",
|
||||
"window_units": 1,
|
||||
"selected_lanes": ["npc_registry", "spells"],
|
||||
"contexts": [
|
||||
{
|
||||
"context_ref": {
|
||||
"source_id": "session-alpha",
|
||||
"start_unit_id": 10,
|
||||
"end_unit_id": 20
|
||||
},
|
||||
"evidence_refs": [
|
||||
{
|
||||
"lane_id": "spells",
|
||||
"source_ref": {
|
||||
"source_id": "session-alpha",
|
||||
"start_unit_id": 10,
|
||||
"end_unit_id": 10
|
||||
}
|
||||
}
|
||||
],
|
||||
"units": [
|
||||
{
|
||||
"id": 10,
|
||||
"kind": "transcript_segment",
|
||||
"text": "Aria casts Cure Wounds.",
|
||||
"ref": {
|
||||
"source_id": "session-alpha",
|
||||
"start_unit_id": 10,
|
||||
"end_unit_id": 10
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": 20,
|
||||
"kind": "transcript_segment",
|
||||
"text": "The party regroups.",
|
||||
"ref": {
|
||||
"source_id": "session-alpha",
|
||||
"start_unit_id": 20,
|
||||
"end_unit_id": 20
|
||||
}
|
||||
}
|
||||
]
|
||||
[
|
||||
{
|
||||
"id": 10,
|
||||
"kind": "transcript_segment",
|
||||
"text": "Aria casts Cure Wounds.",
|
||||
"ref": {
|
||||
"source_id": "session-alpha",
|
||||
"start_unit_id": 10,
|
||||
"end_unit_id": 10
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": 20,
|
||||
"kind": "transcript_segment",
|
||||
"text": "The party regroups.",
|
||||
"ref": {
|
||||
"source_id": "session-alpha",
|
||||
"start_unit_id": 20,
|
||||
"end_unit_id": 20
|
||||
}
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
Each context requires a `context_ref` object and `evidence_refs` and `units`
|
||||
arrays. `context_ref` identifies the first and last included unit. Each
|
||||
evidence entry contains a selected `lane_id` and an original `source_ref`. A
|
||||
unit uses the existing source-unit shape: required `id`, `kind`, `text`, and
|
||||
self `ref`, plus optional JSON-object `metadata`. Fixed payload objects reject
|
||||
unknown fields; unit metadata may contain application-defined JSON values.
|
||||
Each source unit has required `id`, `kind`, `text`, and self `ref` fields.
|
||||
`ref` contains `source_id`, `start_unit_id`, and `end_unit_id`, and both unit
|
||||
endpoints identify that unit's `id`. A unit may also contain source-owned
|
||||
`metadata`, an open-ended JSON object. Fixed unit and reference fields are
|
||||
strict: consumers must reject unknown fixed fields, malformed units, invalid
|
||||
self-references, units whose `source_id` differs from other units in the same
|
||||
excerpt, and a payload that is not the array described here.
|
||||
|
||||
## Citations And Context
|
||||
The excerpt preserves each selected unit exactly as represented by the
|
||||
validated generic source document. It does not add evidence-context-specific
|
||||
annotations or reshape source-owned metadata.
|
||||
|
||||
`evidence_refs` are the authoritative citations. They identify the direct
|
||||
references emitted by accepted normalized artifacts. `context_ref` and the
|
||||
units collection include those cited units plus nearby source units selected by
|
||||
the configured window. They are explanatory context, not widened citations.
|
||||
## Selection And Citations
|
||||
|
||||
Only accepted outputs from the configured lane allowlist contribute. Rejected,
|
||||
failed, absent, and lane-filtered outputs do not contribute. The artifact never
|
||||
contains raw input bytes, prompts, model responses, auxiliary reference
|
||||
content, credentials, or filesystem paths.
|
||||
The framework obtains direct source references only through typed evidence
|
||||
projections of accepted normalized artifacts in the configured lane allowlist.
|
||||
It validates each reference against the current source document, expands its
|
||||
range by `window_units` source-unit positions on each side, clamps at document
|
||||
boundaries, and takes the union of all expanded ranges. The output contains
|
||||
each selected source unit once in source-document position order, regardless
|
||||
of numeric unit IDs. Repeated references, overlapping windows, and citations
|
||||
from multiple lanes do not duplicate a unit. Rejected, failed, absent,
|
||||
inactive, and unselected lanes contribute nothing.
|
||||
|
||||
## Ordering And Compatibility
|
||||
Normalized lane artifacts remain authoritative for citations and for which lane
|
||||
cited a range. The excerpt has no lane attribution and must not be used to
|
||||
reconstruct it. Its included nearby units provide reading context only; they
|
||||
do not widen any citation in a lane artifact.
|
||||
|
||||
The selected lane allowlist is lexical. Contexts and units are in source
|
||||
document position order, not numeric unit-ID order. Direct evidence entries
|
||||
are deterministically ordered by lane and source reference. Overlapping or
|
||||
contiguous windows merge, and each source unit appears at most once in the
|
||||
resulting contexts.
|
||||
The excerpt contains at most every generic source unit once. It can therefore
|
||||
equal the complete generic source document when coverage is broad or the
|
||||
window is large. No byte-, token-, or compression-size guarantee is made, and
|
||||
the framework does not truncate the excerpt to meet an arbitrary size limit.
|
||||
|
||||
## Consumer Responsibilities And Data Handling
|
||||
|
||||
The artifact is additive to the JSON bundle and is not a lane payload,
|
||||
normalized-output count, checkpoint, or generated reference. Consumers that
|
||||
do not need it must tolerate the absent optional descriptor. Consumers that do
|
||||
use it should preserve the artifact and its schema identity with the run
|
||||
provenance, and should treat its source text and metadata as sensitive durable
|
||||
content.
|
||||
do not need it must tolerate an absent descriptor. Consumers that do use it
|
||||
should validate the descriptor and payload before use, retain the artifact with
|
||||
its schema identity when needed for a run record, and read citations from the
|
||||
corresponding normalized lane artifacts.
|
||||
|
||||
The excerpt contains source-unit text and source-owned metadata and is durable
|
||||
output. Treat it as sensitive source content, apply appropriate access controls
|
||||
and retention, and do not assume its selected form is materially smaller or
|
||||
less sensitive than the original input.
|
||||
|
||||
@@ -24,7 +24,7 @@ root for the logical discovery described here.
|
||||
| `warnings.json` | Accepted-output and run warnings. |
|
||||
| `lanes/<safe-lane-id>.json` | One normalized artifact payload for each lane. |
|
||||
| `chunk-map.json` | Optional accepted chunk map, when its export is enabled and available. |
|
||||
| `evidence-context.json` | Optional source-context artifact, when evidence publication is enabled. |
|
||||
| `evidence-context.json` | Optional selected source-unit excerpt, when evidence publication is enabled. |
|
||||
|
||||
JSON files are pretty-printed with a trailing newline. Lane payloads are
|
||||
accepted only when their media type is `application/json`.
|
||||
|
||||
@@ -42,7 +42,8 @@ generic source references and must use the codec's exact Go type. It does not
|
||||
interpret surrounding context or publish files; the pipeline validates the
|
||||
capability during preparation and the output boundary owns publication. See
|
||||
the [Published Evidence Context contract](../integrations/evidence-context.md)
|
||||
for the durable result.
|
||||
for the durable source-unit excerpt. Lane artifacts retain citation and lane
|
||||
provenance; the framework does not add either to that published excerpt.
|
||||
|
||||
An artifact family is broader than a module: it owns the cohesive domain
|
||||
feature across its artifact type, codec, stage modules, validators, prompt
|
||||
|
||||
@@ -110,9 +110,9 @@ are defined in [Accepted Chunk Map](integrations/chunk-map.md). An optional
|
||||
[evidence context](integrations/evidence-context.md) contains source-unit text
|
||||
and metadata. It is not a cache or debug artifact: retain it with the output
|
||||
bundle only for as long as consumers need it, and apply source-content access
|
||||
controls to the entire bundle. Selected lanes may collectively cite most of a
|
||||
transcript, so a broad allowlist can make the evidence artifact nearly as
|
||||
sensitive and large as the source itself.
|
||||
controls to the entire bundle. Its selected source-unit excerpt may include
|
||||
every source unit once when coverage is broad or its configured window is
|
||||
large, so do not assume a byte or token reduction or reduced sensitivity.
|
||||
|
||||
## Chunk-Plan Cache
|
||||
|
||||
|
||||
224
docs/roadmap/dnd-subprocess-documentation.md
Normal file
224
docs/roadmap/dnd-subprocess-documentation.md
Normal file
@@ -0,0 +1,224 @@
|
||||
# D&D Subprocess Consumer Documentation
|
||||
|
||||
## Status
|
||||
|
||||
Completed. The target guide is `docs/consumers/dnd-pipeline.md`.
|
||||
|
||||
## Purpose
|
||||
|
||||
Provide one task-oriented guide for applications that run Notarius as a
|
||||
subprocess to execute the maintained complete D&D pipeline and consume its
|
||||
published artifacts. The initial concrete consumer is Narratio, but the guide
|
||||
must describe the public Notarius workflow rather than depend on Narratio
|
||||
internals.
|
||||
|
||||
The guide should make the safe integration path obvious without duplicating
|
||||
the CLI, input, receipt, output-bundle, or individual artifact contracts that
|
||||
already have canonical documentation.
|
||||
|
||||
## Current State
|
||||
|
||||
The public integration surface is documented accurately but is distributed
|
||||
across several documents:
|
||||
|
||||
- `docs/consumers/subprocess.md` defines the generic subprocess workflow;
|
||||
- `docs/cli.md` owns commands, flags, stream behavior, and exit statuses;
|
||||
- `docs/integrations/seriatim.md` owns the accepted transcript input shape;
|
||||
- `docs/integrations/run-result.md` owns the machine-readable successful-run
|
||||
receipt;
|
||||
- `docs/integrations/json-output.md` owns bundle discovery and logical files;
|
||||
- the D&D integration documents own the individual lane payload contracts;
|
||||
- `examples/dnd-complete.config.yml` is the maintained complete pipeline.
|
||||
|
||||
A consumer can reconstruct the full workflow from those documents, but there
|
||||
is no D&D-focused guide that connects the maintained example to its input,
|
||||
invocation, complete artifact inventory, discovery procedure, and downstream
|
||||
acceptance decisions.
|
||||
|
||||
## Target Documentation Set
|
||||
|
||||
### Create `docs/consumers/dnd-pipeline.md`
|
||||
|
||||
This document should own the end-to-end consumer workflow for the maintained
|
||||
complete D&D configuration. It should be useful to Narratio and to another
|
||||
subprocess orchestrator with the same needs.
|
||||
|
||||
The guide should contain the following sections.
|
||||
|
||||
#### Prerequisites And Deployment Configuration
|
||||
|
||||
- Link to `examples/dnd-complete.config.yml` rather than embedding a second
|
||||
complete configuration.
|
||||
- Explain that a deployment must provide the configured PromptKit profile and
|
||||
campaign reference files.
|
||||
- Recommend absolute paths for a service or orchestrator deployment.
|
||||
- Call out the path-resolution distinction explicitly: YAML reference paths
|
||||
are relative to the Notarius configuration file, while
|
||||
`promptkit.profile_file` is relative to the Notarius process working
|
||||
directory.
|
||||
- Recommend validating the selected configuration and `dnd-session` pipeline
|
||||
before processing sessions.
|
||||
|
||||
#### Transcript Input
|
||||
|
||||
- State that the complete pipeline consumes a Seriatim JSON document.
|
||||
- Link to the canonical Seriatim contract for required fields and validation.
|
||||
- Recommend the caller's final trimmed transcript when the caller maintains
|
||||
transcript tiers. For Narratio, identify the implemented source as
|
||||
`narratio.transcript.final_trimmed`, normally stored at
|
||||
`transcripts/final.trimmed.json`.
|
||||
- Explain that segment IDs must remain stable because D&D source references
|
||||
cite those units.
|
||||
- Explain that Notarius derives its default prompt session from the input
|
||||
module and exact input bytes and that ordinary callers should not supply
|
||||
`--session-id`.
|
||||
|
||||
#### Subprocess Invocation
|
||||
|
||||
- Show one concise invocation using `notarius run dnd-session`, explicit
|
||||
absolute `--config`, `--input`, and `--output-dir` paths, and `--json`.
|
||||
- Direct callers to capture stdout and stderr separately, propagate
|
||||
cancellation, impose an operator-appropriate timeout, and wait for process
|
||||
completion before parsing stdout.
|
||||
- State that only exit status zero permits receipt decoding and link to the CLI
|
||||
contract for the complete exit-status definition.
|
||||
- Recommend retaining stderr and the invocation context for diagnosis without
|
||||
logging secrets or transcript content.
|
||||
|
||||
#### Receipt And Bundle Discovery
|
||||
|
||||
- Require callers to accept only supported run-result schema versions while
|
||||
tolerating unknown fields allowed by that version.
|
||||
- Direct callers to obtain the exact run-specific bundle from the receipt's
|
||||
absolute `output_directory`; they must not scan for the newest run directory
|
||||
or construct a run ID.
|
||||
- Require a confinement check when resolving `index_file` beneath the reported
|
||||
bundle root.
|
||||
- Direct callers to discover lane payloads by `lane_id` in `index.json`, then
|
||||
verify descriptor media type and schema identity before decoding them.
|
||||
- Explain that descriptor paths are untrusted relative paths and require the
|
||||
same confinement discipline.
|
||||
|
||||
#### Complete D&D Artifact Inventory
|
||||
|
||||
Include a compact table for the ten lane IDs selected by the maintained
|
||||
complete configuration:
|
||||
|
||||
- `item-registry`;
|
||||
- `npc-registry`;
|
||||
- `location-registry`;
|
||||
- `scene-descriptions`;
|
||||
- `item-occurrences`;
|
||||
- `spells`;
|
||||
- `combat-turns`;
|
||||
- `npc-occurrences`;
|
||||
- `location-occurrences`;
|
||||
- `enemy-events`.
|
||||
|
||||
For each row, give a one-line purpose and link to the corresponding canonical
|
||||
D&D artifact contract. Do not copy its fields or schema rules into the
|
||||
consumer guide.
|
||||
|
||||
Document the four always-published bundle files—`index.json`, `manifest.json`,
|
||||
`rejected.json`, and `warnings.json`—and the complete example's configured
|
||||
`chunk-map.json` and `evidence-context.json` pipeline-wide artifacts. Link to
|
||||
their canonical contracts and distinguish pipeline-wide artifacts from lane
|
||||
outputs.
|
||||
|
||||
The inventory must say that a file is available only when its corresponding
|
||||
artifact was accepted and published. It must not imply that process success
|
||||
guarantees every configured lane.
|
||||
|
||||
#### Downstream Acceptance And Retention
|
||||
|
||||
- Explain that exit status zero can coexist with rejected outputs, warnings,
|
||||
or absent lane descriptors.
|
||||
- Require the consumer to define its required lane set explicitly. Recommend
|
||||
treating all ten lanes as required when the caller claims to consume the
|
||||
complete D&D workflow, while allowing another consumer to adopt a narrower
|
||||
documented policy.
|
||||
- Recommend retaining the receipt, the complete published bundle, and captured
|
||||
diagnostic streams long enough to support provenance and failure analysis.
|
||||
- Explain that `evidence-context.json` is a reading excerpt; authoritative
|
||||
citations remain in lane payloads.
|
||||
- Treat transcripts, lane artifacts, evidence context, manifests, and logs as
|
||||
sensitive campaign data.
|
||||
|
||||
#### Compatibility Checklist
|
||||
|
||||
End with a concise checklist covering process exit, receipt schema, path
|
||||
confinement, pipeline identity, index decoding, required descriptors,
|
||||
descriptor schema/media compatibility, warnings and rejections, checksums or
|
||||
retention, and secure handling. Compatibility should be based on published
|
||||
receipt and artifact contracts rather than parsing a human version string.
|
||||
|
||||
### Update Existing Navigation
|
||||
|
||||
- Add a short link from `docs/consumers/subprocess.md` to the D&D-specific
|
||||
workflow. Keep generic subprocess policy in the existing document.
|
||||
- Add the guide to the documentation links in `README.md`.
|
||||
- Extend the subprocess-consumer row in `docs/development.md` so maintainers
|
||||
working on the D&D workflow are routed to the new guide and the canonical
|
||||
contracts.
|
||||
|
||||
### Verify Canonical Contract Documents
|
||||
|
||||
Review the linked integration documents and the complete example while writing
|
||||
the guide. Correct an integration document only if repository inspection finds
|
||||
an actual stale contract. Do not move schema definitions, field tables, CLI
|
||||
flags, or configuration semantics into the new guide.
|
||||
|
||||
## Narratio Alignment
|
||||
|
||||
The guide may name Narratio as the motivating consumer and identify its current
|
||||
final-trimmed transcript source. It must not claim that Narratio already has a
|
||||
Notarius adapter or extraction stage. Until that feature is implemented,
|
||||
Narratio-specific architecture, configuration, stage behavior, manifest
|
||||
records, and artifact source IDs belong in Narratio's roadmap.
|
||||
|
||||
Once Narratio implements the integration, its own integration documentation
|
||||
should link to this guide and the durable Notarius contracts instead of
|
||||
repeating them.
|
||||
|
||||
## Validation
|
||||
|
||||
Documentation implementation should include:
|
||||
|
||||
```sh
|
||||
go run ./cmd/notarius config validate \
|
||||
--config examples/dnd-complete.config.yml \
|
||||
--pipeline dnd-session
|
||||
go test ./...
|
||||
```
|
||||
|
||||
Also verify all new and changed relative Markdown links, compare the artifact
|
||||
inventory directly with the maintained complete configuration, and confirm
|
||||
that commands and path semantics match the CLI and configuration references.
|
||||
If the repository still has no automated link checker, record that fact and
|
||||
perform a focused manual link review.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- A subprocess integrator can follow one D&D-focused guide from a Seriatim
|
||||
transcript through safe discovery of every artifact configured by the
|
||||
complete example.
|
||||
- The guide makes stdout, stderr, exit-status, receipt, and path-confinement
|
||||
responsibilities unambiguous.
|
||||
- The ten configured D&D lanes and both configured pipeline-wide artifacts are
|
||||
listed and linked to their canonical contracts.
|
||||
- The guide distinguishes process success from the caller's required-artifact
|
||||
policy.
|
||||
- The profile-path and reference-path resolution rules are clearly stated.
|
||||
- Existing navigation makes the guide discoverable.
|
||||
- No volatile contract is defined in two places, and no unimplemented Narratio
|
||||
behavior is presented as current.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- Implementing or documenting Narratio's future adapter or stage as current
|
||||
Notarius behavior.
|
||||
- Adding a new Notarius command, receipt version, output format, or artifact
|
||||
schema.
|
||||
- Duplicating the complete configuration or individual D&D payload schemas in
|
||||
prose.
|
||||
- Defining a universal partial-result policy for every Notarius consumer.
|
||||
@@ -189,10 +189,18 @@ func TestMaintainedCompleteExamplePublishesRegistryBackedEntityOccurrences(t *te
|
||||
}
|
||||
|
||||
evidence := readProductionJSON[evidencecontext.Document](t, filepath.Join(runRoot, "evidence-context.json"))
|
||||
for _, laneID := range []string{"enemy-events", "npc-registry", "npc-occurrences", "item-registry", "item-occurrences", "location-registry", "location-occurrences"} {
|
||||
if !containsString(evidence.SelectedLanes, laneID) || !evidenceHasLane(evidence, laneID) {
|
||||
t.Fatalf("evidence context = %#v, want direct %s evidence", evidence, laneID)
|
||||
if len(evidence) == 0 {
|
||||
t.Fatalf("evidence context = %#v, want selected source-unit evidence", evidence)
|
||||
}
|
||||
seenEvidenceUnits := make(map[int]struct{}, len(evidence))
|
||||
for _, unit := range evidence {
|
||||
if unit.Ref.SourceID != "session-ravenfall" || unit.Ref.StartUnitID != unit.ID || unit.Ref.EndUnitID != unit.ID {
|
||||
t.Fatalf("evidence unit = %#v, want unchanged source-unit self-reference", unit)
|
||||
}
|
||||
if _, exists := seenEvidenceUnits[unit.ID]; exists {
|
||||
t.Fatalf("evidence context = %#v, want each source unit once", evidence)
|
||||
}
|
||||
seenEvidenceUnits[unit.ID] = struct{}{}
|
||||
}
|
||||
|
||||
requests := client.requestsFor(enemyevents.PromptID)
|
||||
@@ -415,17 +423,6 @@ func containsString(values []string, want string) bool {
|
||||
return false
|
||||
}
|
||||
|
||||
func evidenceHasLane(value evidencecontext.Document, laneID string) bool {
|
||||
for _, context := range value.Contexts {
|
||||
for _, reference := range context.EvidenceRefs {
|
||||
if reference.LaneID == laneID {
|
||||
return true
|
||||
}
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func generatedReferenceBinding(bindings []pipeline.ReferenceBinding, slotName string) (pipeline.ReferenceBinding, bool) {
|
||||
for _, binding := range bindings {
|
||||
if binding.SlotName == slotName && binding.Artifact != nil {
|
||||
|
||||
@@ -2,24 +2,8 @@
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"$id": "notarius.source.evidence_context",
|
||||
"title": "notarius_source_evidence_context_v1",
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["source_id", "source_digest", "window_units", "selected_lanes", "contexts"],
|
||||
"properties": {
|
||||
"source_id": {"type": "string", "minLength": 1},
|
||||
"source_digest": {"type": "string", "pattern": "^sha256:[0-9a-f]{64}$"},
|
||||
"window_units": {"type": "integer", "minimum": 0},
|
||||
"selected_lanes": {
|
||||
"type": "array",
|
||||
"minItems": 1,
|
||||
"uniqueItems": true,
|
||||
"items": {"type": "string", "minLength": 1}
|
||||
},
|
||||
"contexts": {
|
||||
"type": "array",
|
||||
"items": {"$ref": "#/$defs/context"}
|
||||
}
|
||||
},
|
||||
"type": "array",
|
||||
"items": {"$ref": "#/$defs/unit"},
|
||||
"$defs": {
|
||||
"source_ref": {
|
||||
"type": "object",
|
||||
@@ -42,25 +26,6 @@
|
||||
"ref": {"$ref": "#/$defs/source_ref"},
|
||||
"metadata": {"type": "object", "additionalProperties": true}
|
||||
}
|
||||
},
|
||||
"evidence_ref": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["lane_id", "source_ref"],
|
||||
"properties": {
|
||||
"lane_id": {"type": "string", "minLength": 1},
|
||||
"source_ref": {"$ref": "#/$defs/source_ref"}
|
||||
}
|
||||
},
|
||||
"context": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["context_ref", "evidence_refs", "units"],
|
||||
"properties": {
|
||||
"context_ref": {"$ref": "#/$defs/source_ref"},
|
||||
"evidence_refs": {"type": "array", "minItems": 1, "items": {"$ref": "#/$defs/evidence_ref"}},
|
||||
"units": {"type": "array", "minItems": 1, "items": {"$ref": "#/$defs/unit"}}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -3,118 +3,62 @@ package evidencecontext
|
||||
import (
|
||||
"fmt"
|
||||
"sort"
|
||||
"strings"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
|
||||
)
|
||||
|
||||
type contribution struct {
|
||||
laneID string
|
||||
ref source.SourceRef
|
||||
type expandedRange struct {
|
||||
startPos int
|
||||
endPos int
|
||||
}
|
||||
|
||||
type expandedRange struct {
|
||||
startPos int
|
||||
endPos int
|
||||
contributions []contribution
|
||||
}
|
||||
|
||||
// Build validates accepted direct references, expands them by source-document
|
||||
// position, and returns their deterministic context union.
|
||||
// Build validates projected source references, expands them by source-document
|
||||
// position, and returns their ordered union as an owned source-unit excerpt.
|
||||
func Build(request BuildRequest) (Document, error) {
|
||||
if request.WindowUnits < 0 {
|
||||
return Document{}, fmt.Errorf("window_units must not be negative")
|
||||
}
|
||||
lanes, err := normalizeSelectedLanes(request.SelectedLanes)
|
||||
if err != nil {
|
||||
return Document{}, err
|
||||
return nil, fmt.Errorf("window_units must not be negative")
|
||||
}
|
||||
if err := source.ValidateDocument(request.Source); err != nil {
|
||||
return Document{}, fmt.Errorf("validate source document: %w", err)
|
||||
return nil, fmt.Errorf("validate source document: %w", err)
|
||||
}
|
||||
digest, err := source.DigestDocument(request.Source)
|
||||
if err != nil {
|
||||
return Document{}, fmt.Errorf("digest source document: %w", err)
|
||||
return nil, fmt.Errorf("digest source document: %w", err)
|
||||
}
|
||||
if digest != request.Source.Digest {
|
||||
return Document{}, fmt.Errorf("source digest does not match source document digest")
|
||||
return nil, fmt.Errorf("source digest does not match source document digest")
|
||||
}
|
||||
|
||||
selected := make(map[string]struct{}, len(lanes))
|
||||
for _, laneID := range lanes {
|
||||
selected[laneID] = struct{}{}
|
||||
}
|
||||
index := source.NewDocumentIndex(request.Source)
|
||||
seen := make(map[evidenceKey]struct{})
|
||||
contributions := make([]contribution, 0)
|
||||
for laneIndex, laneEvidence := range request.LaneEvidence {
|
||||
laneID := strings.TrimSpace(laneEvidence.LaneID)
|
||||
if _, ok := selected[laneID]; !ok {
|
||||
return Document{}, fmt.Errorf("lane evidence[%d] lane %q is not selected", laneIndex, laneID)
|
||||
ranges := make([]expandedRange, 0, len(request.SourceRefs))
|
||||
for refIndex, ref := range request.SourceRefs {
|
||||
if err := index.ValidateRef(ref); err != nil {
|
||||
return nil, fmt.Errorf("source reference[%d]: %w", refIndex, err)
|
||||
}
|
||||
for refIndex, ref := range laneEvidence.SourceRefs {
|
||||
if err := index.ValidateRef(ref); err != nil {
|
||||
return Document{}, fmt.Errorf("lane %q source reference[%d]: %w", laneID, refIndex, err)
|
||||
startPos, _ := index.Position(ref.StartUnitID)
|
||||
endPos, _ := index.Position(ref.EndUnitID)
|
||||
ranges = append(ranges, expandedRange{
|
||||
startPos: expandStart(startPos, request.WindowUnits),
|
||||
endPos: expandEnd(endPos, len(request.Source.Units), request.WindowUnits),
|
||||
})
|
||||
}
|
||||
|
||||
merged := mergeRanges(ranges)
|
||||
unitCount := 0
|
||||
for _, value := range merged {
|
||||
unitCount += value.endPos - value.startPos + 1
|
||||
}
|
||||
document := make(Document, 0, unitCount)
|
||||
for _, value := range merged {
|
||||
for position := value.startPos; position <= value.endPos; position++ {
|
||||
unit, err := cloneSourceUnit(request.Source.Units[position])
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("clone source unit at position %d: %w", position, err)
|
||||
}
|
||||
key := evidenceKey{laneID: laneID, ref: ref}
|
||||
if _, exists := seen[key]; exists {
|
||||
continue
|
||||
}
|
||||
seen[key] = struct{}{}
|
||||
startPos, _ := index.Position(ref.StartUnitID)
|
||||
endPos, _ := index.Position(ref.EndUnitID)
|
||||
contributions = append(contributions, contribution{laneID: laneID, ref: ref, startPos: expandStart(startPos, request.WindowUnits), endPos: expandEnd(endPos, len(request.Source.Units), request.WindowUnits)})
|
||||
document = append(document, unit)
|
||||
}
|
||||
}
|
||||
|
||||
sort.Slice(contributions, func(i, j int) bool { return lessContribution(contributions[i], contributions[j]) })
|
||||
document := Document{
|
||||
SourceID: request.Source.ID,
|
||||
SourceDigest: digest,
|
||||
WindowUnits: request.WindowUnits,
|
||||
SelectedLanes: lanes,
|
||||
Contexts: make([]Context, 0),
|
||||
}
|
||||
for _, rangeValue := range mergeRanges(contributions) {
|
||||
context, err := buildContext(request.Source, rangeValue)
|
||||
if err != nil {
|
||||
return Document{}, err
|
||||
}
|
||||
document.Contexts = append(document.Contexts, context)
|
||||
}
|
||||
canonical, err := canonicalizeOwned(document)
|
||||
if err != nil {
|
||||
return Document{}, fmt.Errorf("validate evidence context: %w", err)
|
||||
}
|
||||
return canonical, nil
|
||||
}
|
||||
|
||||
type evidenceKey struct {
|
||||
laneID string
|
||||
ref source.SourceRef
|
||||
}
|
||||
|
||||
func normalizeSelectedLanes(values []string) ([]string, error) {
|
||||
if len(values) == 0 {
|
||||
return nil, fmt.Errorf("selected_lanes must not be empty")
|
||||
}
|
||||
seen := make(map[string]struct{}, len(values))
|
||||
lanes := make([]string, 0, len(values))
|
||||
for index, raw := range values {
|
||||
laneID := strings.TrimSpace(raw)
|
||||
if laneID == "" {
|
||||
return nil, fmt.Errorf("selected_lanes[%d] must not be empty", index)
|
||||
}
|
||||
if _, exists := seen[laneID]; exists {
|
||||
return nil, fmt.Errorf("selected_lanes lane %q is duplicated", laneID)
|
||||
}
|
||||
seen[laneID] = struct{}{}
|
||||
lanes = append(lanes, laneID)
|
||||
}
|
||||
sort.Strings(lanes)
|
||||
return lanes, nil
|
||||
return document, nil
|
||||
}
|
||||
|
||||
func expandStart(position, window int) int {
|
||||
@@ -132,65 +76,25 @@ func expandEnd(position, length, window int) int {
|
||||
return position + window
|
||||
}
|
||||
|
||||
func lessContribution(left, right contribution) bool {
|
||||
if left.startPos != right.startPos {
|
||||
return left.startPos < right.startPos
|
||||
}
|
||||
if left.endPos != right.endPos {
|
||||
return left.endPos < right.endPos
|
||||
}
|
||||
return lessEvidenceRef(EvidenceRef{LaneID: left.laneID, SourceRef: left.ref}, EvidenceRef{LaneID: right.laneID, SourceRef: right.ref})
|
||||
}
|
||||
|
||||
func mergeRanges(values []contribution) []expandedRange {
|
||||
func mergeRanges(values []expandedRange) []expandedRange {
|
||||
if len(values) == 0 {
|
||||
return nil
|
||||
}
|
||||
ranges := make([]expandedRange, 0, len(values))
|
||||
sort.Slice(values, func(i, j int) bool {
|
||||
if values[i].startPos != values[j].startPos {
|
||||
return values[i].startPos < values[j].startPos
|
||||
}
|
||||
return values[i].endPos < values[j].endPos
|
||||
})
|
||||
merged := make([]expandedRange, 0, len(values))
|
||||
for _, value := range values {
|
||||
if len(ranges) == 0 || value.startPos > ranges[len(ranges)-1].endPos+1 {
|
||||
ranges = append(ranges, expandedRange{startPos: value.startPos, endPos: value.endPos, contributions: []contribution{value}})
|
||||
if len(merged) == 0 || value.startPos > merged[len(merged)-1].endPos+1 {
|
||||
merged = append(merged, value)
|
||||
continue
|
||||
}
|
||||
current := &ranges[len(ranges)-1]
|
||||
if value.endPos > current.endPos {
|
||||
current.endPos = value.endPos
|
||||
if value.endPos > merged[len(merged)-1].endPos {
|
||||
merged[len(merged)-1].endPos = value.endPos
|
||||
}
|
||||
current.contributions = append(current.contributions, value)
|
||||
}
|
||||
return ranges
|
||||
}
|
||||
|
||||
func buildContext(document *source.SourceDocument, value expandedRange) (Context, error) {
|
||||
evidenceRefs := make([]EvidenceRef, 0, len(value.contributions))
|
||||
for _, contribution := range value.contributions {
|
||||
evidenceRefs = append(evidenceRefs, EvidenceRef{LaneID: contribution.laneID, SourceRef: contribution.ref})
|
||||
}
|
||||
sort.Slice(evidenceRefs, func(i, j int) bool { return lessEvidenceRef(evidenceRefs[i], evidenceRefs[j]) })
|
||||
units := make([]source.SourceUnit, 0, value.endPos-value.startPos+1)
|
||||
for position := value.startPos; position <= value.endPos; position++ {
|
||||
unit, err := cloneSourceUnit(document.Units[position])
|
||||
if err != nil {
|
||||
return Context{}, fmt.Errorf("clone source unit at position %d: %w", position, err)
|
||||
}
|
||||
units = append(units, unit)
|
||||
}
|
||||
return Context{
|
||||
ContextRef: source.SourceRef{SourceID: document.ID, StartUnitID: units[0].ID, EndUnitID: units[len(units)-1].ID},
|
||||
EvidenceRefs: evidenceRefs,
|
||||
Units: units,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func lessEvidenceRef(left, right EvidenceRef) bool {
|
||||
if left.LaneID != right.LaneID {
|
||||
return left.LaneID < right.LaneID
|
||||
}
|
||||
if left.SourceRef.SourceID != right.SourceRef.SourceID {
|
||||
return left.SourceRef.SourceID < right.SourceRef.SourceID
|
||||
}
|
||||
if left.SourceRef.StartUnitID != right.SourceRef.StartUnitID {
|
||||
return left.SourceRef.StartUnitID < right.SourceRef.StartUnitID
|
||||
}
|
||||
return left.SourceRef.EndUnitID < right.SourceRef.EndUnitID
|
||||
return merged
|
||||
}
|
||||
|
||||
@@ -6,7 +6,6 @@ import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"regexp"
|
||||
"strings"
|
||||
"sync"
|
||||
|
||||
@@ -18,8 +17,6 @@ import (
|
||||
//go:embed assets/schemas/source_evidence_context.v1.json
|
||||
var schemaAssets embed.FS
|
||||
|
||||
var digestPattern = regexp.MustCompile(`^sha256:[0-9a-f]{64}$`)
|
||||
|
||||
var (
|
||||
loadSchemaOnce sync.Once
|
||||
loadedSchema []byte
|
||||
@@ -78,24 +75,24 @@ func (c *Codec) Encode(value Document) ([]byte, error) {
|
||||
|
||||
func (c *Codec) Decode(content []byte) (Document, error) {
|
||||
if _, err := c.schemaBytes(); err != nil {
|
||||
return Document{}, err
|
||||
return nil, err
|
||||
}
|
||||
if err := validateSchemaInstance(content); err != nil {
|
||||
return Document{}, fmt.Errorf("decode evidence context: %w", err)
|
||||
return nil, fmt.Errorf("decode evidence context: %w", err)
|
||||
}
|
||||
decoder := json.NewDecoder(bytes.NewReader(content))
|
||||
decoder.DisallowUnknownFields()
|
||||
var value Document
|
||||
if err := decoder.Decode(&value); err != nil {
|
||||
return Document{}, fmt.Errorf("decode evidence context: %w", err)
|
||||
return nil, fmt.Errorf("decode evidence context: %w", err)
|
||||
}
|
||||
var trailing any
|
||||
if err := decoder.Decode(&trailing); err != io.EOF {
|
||||
return Document{}, fmt.Errorf("decode evidence context: multiple JSON values")
|
||||
return nil, fmt.Errorf("decode evidence context: multiple JSON values")
|
||||
}
|
||||
canonical, err := canonicalizeOwned(value)
|
||||
if err != nil {
|
||||
return Document{}, fmt.Errorf("decode evidence context: %w", err)
|
||||
return nil, fmt.Errorf("decode evidence context: %w", err)
|
||||
}
|
||||
return canonical, nil
|
||||
}
|
||||
@@ -115,17 +112,16 @@ func loadAndCompileSchema() {
|
||||
return
|
||||
}
|
||||
var identity struct {
|
||||
ID string `json:"$id"`
|
||||
Title string `json:"title"`
|
||||
Type string `json:"type"`
|
||||
Required []string `json:"required"`
|
||||
ID string `json:"$id"`
|
||||
Title string `json:"title"`
|
||||
Type string `json:"type"`
|
||||
}
|
||||
if err := json.Unmarshal(raw, &identity); err != nil {
|
||||
loadSchemaErr = fmt.Errorf("decode source evidence context schema: %w", err)
|
||||
return
|
||||
}
|
||||
if identity.ID != SchemaID || identity.Title != SchemaName || identity.Type != "object" || !hasRequiredFields(identity.Required) {
|
||||
loadSchemaErr = fmt.Errorf("source evidence context schema identity or required fields are invalid")
|
||||
if identity.ID != SchemaID || identity.Title != SchemaName || identity.Type != "array" {
|
||||
loadSchemaErr = fmt.Errorf("source evidence context schema identity is invalid")
|
||||
return
|
||||
}
|
||||
schemaDocument, err := jsonschema.UnmarshalJSON(bytes.NewReader(raw))
|
||||
@@ -158,164 +154,65 @@ func validateSchemaInstance(content []byte) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
func hasRequiredFields(required []string) bool {
|
||||
want := map[string]bool{"source_id": true, "source_digest": true, "window_units": true, "selected_lanes": true, "contexts": true}
|
||||
for _, field := range required {
|
||||
delete(want, field)
|
||||
}
|
||||
return len(want) == 0
|
||||
}
|
||||
|
||||
func canonicalize(value Document) (Document, error) {
|
||||
owned, err := clone(value)
|
||||
if err != nil {
|
||||
return Document{}, err
|
||||
return nil, err
|
||||
}
|
||||
return canonicalizeOwned(owned)
|
||||
}
|
||||
|
||||
func canonicalizeOwned(value Document) (Document, error) {
|
||||
if err := requireIdentity("source_id", value.SourceID); err != nil {
|
||||
return Document{}, err
|
||||
if value == nil {
|
||||
return nil, fmt.Errorf("document must be a JSON array")
|
||||
}
|
||||
if !digestPattern.MatchString(value.SourceDigest) {
|
||||
return Document{}, fmt.Errorf("source_digest must be a sha256 digest")
|
||||
}
|
||||
if value.WindowUnits < 0 {
|
||||
return Document{}, fmt.Errorf("window_units must not be negative")
|
||||
}
|
||||
if err := validateSelectedLanes(value.SelectedLanes); err != nil {
|
||||
return Document{}, err
|
||||
}
|
||||
if value.Contexts == nil {
|
||||
value.Contexts = make([]Context, 0)
|
||||
}
|
||||
selected := make(map[string]struct{}, len(value.SelectedLanes))
|
||||
for _, laneID := range value.SelectedLanes {
|
||||
selected[laneID] = struct{}{}
|
||||
}
|
||||
seenUnits := make(map[int]struct{})
|
||||
for contextIndex := range value.Contexts {
|
||||
context, err := canonicalizeContext(value.SourceID, selected, seenUnits, value.Contexts[contextIndex], contextIndex)
|
||||
if err != nil {
|
||||
return Document{}, err
|
||||
}
|
||||
value.Contexts[contextIndex] = context
|
||||
}
|
||||
return value, nil
|
||||
}
|
||||
|
||||
func validateSelectedLanes(lanes []string) error {
|
||||
if len(lanes) == 0 {
|
||||
return fmt.Errorf("selected_lanes must not be empty")
|
||||
}
|
||||
for index, laneID := range lanes {
|
||||
if err := requireIdentity(fmt.Sprintf("selected_lanes[%d]", index), laneID); err != nil {
|
||||
return err
|
||||
}
|
||||
if index > 0 && lanes[index-1] >= laneID {
|
||||
return fmt.Errorf("selected_lanes must be unique and in lexical order")
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func canonicalizeContext(sourceID string, selected map[string]struct{}, seenUnits map[int]struct{}, value Context, contextIndex int) (Context, error) {
|
||||
prefix := fmt.Sprintf("contexts[%d]", contextIndex)
|
||||
if len(value.EvidenceRefs) == 0 {
|
||||
return Context{}, fmt.Errorf("%s.evidence_refs must not be empty", prefix)
|
||||
}
|
||||
if len(value.Units) == 0 {
|
||||
return Context{}, fmt.Errorf("%s.units must not be empty", prefix)
|
||||
}
|
||||
if err := validateRefIdentity(sourceID, value.ContextRef, prefix+".context_ref"); err != nil {
|
||||
return Context{}, err
|
||||
}
|
||||
positions := make(map[int]int, len(value.Units))
|
||||
for unitIndex := range value.Units {
|
||||
unit := value.Units[unitIndex]
|
||||
seenUnitIDs := make(map[int]struct{}, len(value))
|
||||
sourceID := ""
|
||||
for unitIndex := range value {
|
||||
unit := value[unitIndex]
|
||||
if unit.ID <= 0 || strings.TrimSpace(unit.Kind) == "" || strings.TrimSpace(unit.Text) == "" {
|
||||
return Context{}, fmt.Errorf("%s.units[%d] has invalid required fields", prefix, unitIndex)
|
||||
return nil, fmt.Errorf("units[%d] has invalid required fields", unitIndex)
|
||||
}
|
||||
if err := validateRefIdentity(sourceID, unit.Ref, fmt.Sprintf("%s.units[%d].ref", prefix, unitIndex)); err != nil {
|
||||
return Context{}, err
|
||||
if err := validateUnitRef(unit, unitIndex); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if unit.Ref.StartUnitID != unit.ID || unit.Ref.EndUnitID != unit.ID {
|
||||
return Context{}, fmt.Errorf("%s.units[%d].ref must identify unit id %d", prefix, unitIndex, unit.ID)
|
||||
if sourceID == "" {
|
||||
sourceID = unit.Ref.SourceID
|
||||
} else if unit.Ref.SourceID != sourceID {
|
||||
return nil, fmt.Errorf("units[%d].ref.source_id must match units[0].ref.source_id", unitIndex)
|
||||
}
|
||||
if _, exists := positions[unit.ID]; exists {
|
||||
return Context{}, fmt.Errorf("%s.units contains duplicate unit id %d", prefix, unit.ID)
|
||||
}
|
||||
if _, exists := seenUnits[unit.ID]; exists {
|
||||
return Context{}, fmt.Errorf("contexts contain duplicate unit id %d", unit.ID)
|
||||
}
|
||||
positions[unit.ID] = unitIndex
|
||||
seenUnits[unit.ID] = struct{}{}
|
||||
}
|
||||
if value.ContextRef.StartUnitID != value.Units[0].ID || value.ContextRef.EndUnitID != value.Units[len(value.Units)-1].ID {
|
||||
return Context{}, fmt.Errorf("%s.context_ref must identify the first and last units", prefix)
|
||||
}
|
||||
for evidenceIndex := range value.EvidenceRefs {
|
||||
evidence := value.EvidenceRefs[evidenceIndex]
|
||||
if _, ok := selected[evidence.LaneID]; !ok {
|
||||
return Context{}, fmt.Errorf("%s.evidence_refs[%d].lane_id is not selected", prefix, evidenceIndex)
|
||||
}
|
||||
if err := requireIdentity(fmt.Sprintf("%s.evidence_refs[%d].lane_id", prefix, evidenceIndex), evidence.LaneID); err != nil {
|
||||
return Context{}, err
|
||||
}
|
||||
if err := validateRefIdentity(sourceID, evidence.SourceRef, fmt.Sprintf("%s.evidence_refs[%d].source_ref", prefix, evidenceIndex)); err != nil {
|
||||
return Context{}, err
|
||||
}
|
||||
start, startOK := positions[evidence.SourceRef.StartUnitID]
|
||||
end, endOK := positions[evidence.SourceRef.EndUnitID]
|
||||
if !startOK || !endOK || start > end {
|
||||
return Context{}, fmt.Errorf("%s.evidence_refs[%d].source_ref is outside context units", prefix, evidenceIndex)
|
||||
}
|
||||
if evidenceIndex > 0 && !lessEvidenceRef(value.EvidenceRefs[evidenceIndex-1], evidence) {
|
||||
return Context{}, fmt.Errorf("%s.evidence_refs must be unique and in deterministic order", prefix)
|
||||
if _, exists := seenUnitIDs[unit.ID]; exists {
|
||||
return nil, fmt.Errorf("units contains duplicate unit id %d", unit.ID)
|
||||
}
|
||||
seenUnitIDs[unit.ID] = struct{}{}
|
||||
}
|
||||
return value, nil
|
||||
}
|
||||
|
||||
func validateRefIdentity(sourceID string, ref source.SourceRef, field string) error {
|
||||
if ref.SourceID != sourceID {
|
||||
return fmt.Errorf("%s.source_id does not match source_id", field)
|
||||
func validateUnitRef(unit source.SourceUnit, unitIndex int) error {
|
||||
prefix := fmt.Sprintf("units[%d].ref", unitIndex)
|
||||
if strings.TrimSpace(unit.Ref.SourceID) == "" || strings.TrimSpace(unit.Ref.SourceID) != unit.Ref.SourceID {
|
||||
return fmt.Errorf("%s.source_id must be a non-empty trimmed string", prefix)
|
||||
}
|
||||
if ref.StartUnitID <= 0 || ref.EndUnitID <= 0 {
|
||||
return fmt.Errorf("%s endpoints must be positive", field)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func requireIdentity(field, value string) error {
|
||||
if strings.TrimSpace(value) == "" || strings.TrimSpace(value) != value {
|
||||
return fmt.Errorf("%s must be a non-empty trimmed string", field)
|
||||
if unit.Ref.StartUnitID != unit.ID || unit.Ref.EndUnitID != unit.ID {
|
||||
return fmt.Errorf("%s must identify unit id %d", prefix, unit.ID)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func clone(value Document) (Document, error) {
|
||||
value.SelectedLanes = append([]string(nil), value.SelectedLanes...)
|
||||
if value.Contexts == nil {
|
||||
value.Contexts = make([]Context, 0)
|
||||
} else {
|
||||
contexts := make([]Context, len(value.Contexts))
|
||||
for contextIndex, context := range value.Contexts {
|
||||
contexts[contextIndex].ContextRef = context.ContextRef
|
||||
contexts[contextIndex].EvidenceRefs = append([]EvidenceRef(nil), context.EvidenceRefs...)
|
||||
contexts[contextIndex].Units = make([]source.SourceUnit, len(context.Units))
|
||||
for unitIndex, unit := range context.Units {
|
||||
cloned, err := cloneSourceUnit(unit)
|
||||
if err != nil {
|
||||
return Document{}, fmt.Errorf("clone contexts[%d].units[%d]: %w", contextIndex, unitIndex, err)
|
||||
}
|
||||
contexts[contextIndex].Units[unitIndex] = cloned
|
||||
}
|
||||
}
|
||||
value.Contexts = contexts
|
||||
if value == nil {
|
||||
return nil, nil
|
||||
}
|
||||
return value, nil
|
||||
cloned := make(Document, len(value))
|
||||
for unitIndex, unit := range value {
|
||||
owned, err := cloneSourceUnit(unit)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("clone units[%d]: %w", unitIndex, err)
|
||||
}
|
||||
cloned[unitIndex] = owned
|
||||
}
|
||||
return cloned, nil
|
||||
}
|
||||
|
||||
func cloneSourceUnit(unit source.SourceUnit) (source.SourceUnit, error) {
|
||||
|
||||
@@ -2,7 +2,6 @@ package evidencecontext
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"math"
|
||||
"os"
|
||||
"reflect"
|
||||
@@ -12,126 +11,57 @@ import (
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
|
||||
)
|
||||
|
||||
func TestBuildExpandsAndMergesEvidenceByDocumentPosition(t *testing.T) {
|
||||
func TestBuildSelectsExpandedSourceUnitUnion(t *testing.T) {
|
||||
for _, test := range []struct {
|
||||
name string
|
||||
window int
|
||||
evidence []LaneEvidence
|
||||
wantUnits [][]int
|
||||
wantRefs [][]EvidenceRef
|
||||
name string
|
||||
window int
|
||||
refs []source.SourceRef
|
||||
wantIDs []int
|
||||
}{
|
||||
{
|
||||
name: "zero window",
|
||||
evidence: []LaneEvidence{{LaneID: "npcs", SourceRefs: []source.SourceRef{ref(3, 3)}}},
|
||||
wantUnits: [][]int{{3}},
|
||||
wantRefs: [][]EvidenceRef{{{LaneID: "npcs", SourceRef: ref(3, 3)}}},
|
||||
},
|
||||
{
|
||||
name: "non monotonic ids use positions and clip boundaries",
|
||||
window: 1,
|
||||
evidence: []LaneEvidence{{LaneID: "npcs", SourceRefs: []source.SourceRef{ref(3, 3)}}},
|
||||
wantUnits: [][]int{{10, 3, 30}},
|
||||
wantRefs: [][]EvidenceRef{{{LaneID: "npcs", SourceRef: ref(3, 3)}}},
|
||||
},
|
||||
{
|
||||
name: "separate gaps stay separate",
|
||||
evidence: []LaneEvidence{{LaneID: "npcs", SourceRefs: []source.SourceRef{ref(10, 10), ref(50, 50)}}},
|
||||
wantUnits: [][]int{{10}, {50}},
|
||||
wantRefs: [][]EvidenceRef{{{LaneID: "npcs", SourceRef: ref(10, 10)}}, {{LaneID: "npcs", SourceRef: ref(50, 50)}}},
|
||||
},
|
||||
{
|
||||
name: "overlapping windows merge",
|
||||
window: 1,
|
||||
evidence: []LaneEvidence{{LaneID: "npcs", SourceRefs: []source.SourceRef{ref(3, 3), ref(30, 30)}}},
|
||||
wantUnits: [][]int{{10, 3, 30, 7}},
|
||||
wantRefs: [][]EvidenceRef{{{LaneID: "npcs", SourceRef: ref(3, 3)}, {LaneID: "npcs", SourceRef: ref(30, 30)}}},
|
||||
},
|
||||
{
|
||||
name: "contiguous windows merge",
|
||||
window: 1,
|
||||
evidence: []LaneEvidence{{LaneID: "npcs", SourceRefs: []source.SourceRef{ref(10, 10), ref(7, 7)}}},
|
||||
wantUnits: [][]int{{10, 3, 30, 7, 50}},
|
||||
wantRefs: [][]EvidenceRef{{{LaneID: "npcs", SourceRef: ref(7, 7)}, {LaneID: "npcs", SourceRef: ref(10, 10)}}},
|
||||
},
|
||||
{
|
||||
name: "duplicate contributions retain unique lane attribution",
|
||||
evidence: []LaneEvidence{
|
||||
{LaneID: "spells", SourceRefs: []source.SourceRef{ref(30, 30), ref(30, 30)}},
|
||||
{LaneID: "npcs", SourceRefs: []source.SourceRef{ref(30, 30)}},
|
||||
},
|
||||
wantUnits: [][]int{{30}},
|
||||
wantRefs: [][]EvidenceRef{{{LaneID: "npcs", SourceRef: ref(30, 30)}, {LaneID: "spells", SourceRef: ref(30, 30)}}},
|
||||
},
|
||||
{
|
||||
name: "empty contributions retain explicit empty contexts",
|
||||
evidence: []LaneEvidence{{LaneID: "npcs"}},
|
||||
wantUnits: [][]int{},
|
||||
wantRefs: [][]EvidenceRef{},
|
||||
},
|
||||
{
|
||||
name: "largest window clips without overflow",
|
||||
window: math.MaxInt,
|
||||
evidence: []LaneEvidence{{LaneID: "npcs", SourceRefs: []source.SourceRef{ref(30, 30)}}},
|
||||
wantUnits: [][]int{{10, 3, 30, 7, 50}},
|
||||
wantRefs: [][]EvidenceRef{{{LaneID: "npcs", SourceRef: ref(30, 30)}}},
|
||||
},
|
||||
{name: "zero window", refs: []source.SourceRef{ref(3, 3)}, wantIDs: []int{3}},
|
||||
{name: "multi unit citation includes complete range", refs: []source.SourceRef{ref(3, 7)}, wantIDs: []int{3, 30, 7}},
|
||||
{name: "non monotonic IDs use document positions", window: 1, refs: []source.SourceRef{ref(3, 3)}, wantIDs: []int{10, 3, 30}},
|
||||
{name: "boundary clamping", window: 1, refs: []source.SourceRef{ref(10, 10), ref(50, 50)}, wantIDs: []int{10, 3, 7, 50}},
|
||||
{name: "overlapping and adjacent windows merge", window: 1, refs: []source.SourceRef{ref(3, 3), ref(30, 30), ref(30, 30)}, wantIDs: []int{10, 3, 30, 7}},
|
||||
{name: "adjacent expanded ranges merge", window: 1, refs: []source.SourceRef{ref(10, 10), ref(7, 7)}, wantIDs: []int{10, 3, 30, 7, 50}},
|
||||
{name: "largest window clips without overflow", window: math.MaxInt, refs: []source.SourceRef{ref(30, 30)}, wantIDs: []int{10, 3, 30, 7, 50}},
|
||||
{name: "no references returns an initialized empty document", wantIDs: []int{}},
|
||||
} {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
document := testDocument(t)
|
||||
got, err := Build(BuildRequest{Source: document, WindowUnits: test.window, SelectedLanes: []string{"spells", "npcs"}, LaneEvidence: test.evidence})
|
||||
got, err := Build(BuildRequest{Source: testDocument(t), WindowUnits: test.window, SourceRefs: test.refs})
|
||||
if err != nil {
|
||||
t.Fatalf("Build() error = %v", err)
|
||||
}
|
||||
if want := []string{"npcs", "spells"}; !reflect.DeepEqual(got.SelectedLanes, want) {
|
||||
t.Fatalf("SelectedLanes = %#v, want %#v", got.SelectedLanes, want)
|
||||
if got == nil {
|
||||
t.Fatal("Build() returned a nil document")
|
||||
}
|
||||
if got.WindowUnits != test.window || got.SourceID != document.ID || got.SourceDigest != document.Digest {
|
||||
t.Fatalf("Build() identity = %#v, want source and window identity", got)
|
||||
}
|
||||
if actual := contextUnitIDs(got.Contexts); !reflect.DeepEqual(actual, test.wantUnits) {
|
||||
t.Fatalf("context unit ids = %#v, want %#v", actual, test.wantUnits)
|
||||
}
|
||||
if actual := contextEvidenceRefs(got.Contexts); !reflect.DeepEqual(actual, test.wantRefs) {
|
||||
t.Fatalf("context evidence refs = %#v, want %#v", actual, test.wantRefs)
|
||||
if actual := unitIDs(got); !reflect.DeepEqual(actual, test.wantIDs) {
|
||||
t.Fatalf("unit IDs = %#v, want %#v", actual, test.wantIDs)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestBuildIsStableAndOwnsSourceAndInputs(t *testing.T) {
|
||||
func TestBuildCopiesSelectedUnitsAndMetadata(t *testing.T) {
|
||||
document := testDocument(t)
|
||||
refs := []source.SourceRef{ref(30, 30), ref(3, 3)}
|
||||
request := BuildRequest{
|
||||
Source: document,
|
||||
WindowUnits: 1,
|
||||
SelectedLanes: []string{"spells", "npcs"},
|
||||
LaneEvidence: []LaneEvidence{{LaneID: "spells", SourceRefs: refs}, {LaneID: "npcs", SourceRefs: []source.SourceRef{ref(3, 3)}}},
|
||||
}
|
||||
first, err := Build(request)
|
||||
first, err := Build(BuildRequest{Source: document, WindowUnits: 1, SourceRefs: []source.SourceRef{ref(3, 3)}})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
secondRequest := request
|
||||
secondRequest.LaneEvidence = []LaneEvidence{{LaneID: "npcs", SourceRefs: []source.SourceRef{ref(3, 3)}}, {LaneID: "spells", SourceRefs: []source.SourceRef{ref(3, 3), ref(30, 30)}}}
|
||||
second, err := Build(secondRequest)
|
||||
second, err := Build(BuildRequest{Source: document, WindowUnits: 1, SourceRefs: []source.SourceRef{ref(3, 3)}})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !reflect.DeepEqual(first, second) {
|
||||
t.Fatalf("Build() order differs:\nfirst: %#v\nsecond: %#v", first, second)
|
||||
if !reflect.DeepEqual(first[0], document.Units[0]) {
|
||||
t.Fatalf("first unit = %#v, want unchanged source unit %#v", first[0], document.Units[0])
|
||||
}
|
||||
first.SelectedLanes[0] = "changed"
|
||||
first.Contexts[0].Units[0].Metadata["nested"].(map[string]any)["value"] = "changed"
|
||||
first[0].Metadata["nested"].(map[string]any)["value"] = "changed"
|
||||
if document.Units[0].Metadata["nested"].(map[string]any)["value"] != "original" {
|
||||
t.Fatal("Build() returned metadata aliases to source document")
|
||||
t.Fatal("Build() returned metadata aliases to the source document")
|
||||
}
|
||||
document.Units[0].Metadata["nested"].(map[string]any)["value"] = "later"
|
||||
if second.Contexts[0].Units[0].Metadata["nested"].(map[string]any)["value"] != "original" {
|
||||
t.Fatal("Build() retained metadata aliases to source document")
|
||||
}
|
||||
refs[0].StartUnitID = 999
|
||||
if !containsEvidenceRef(second.Contexts[0].EvidenceRefs, ref(30, 30)) {
|
||||
t.Fatal("Build() retained source-reference input aliases")
|
||||
if second[0].Metadata["nested"].(map[string]any)["value"] != "original" {
|
||||
t.Fatal("Build() retained metadata aliases to the source document")
|
||||
}
|
||||
}
|
||||
|
||||
@@ -142,18 +72,11 @@ func TestBuildRejectsInvalidInputs(t *testing.T) {
|
||||
want string
|
||||
}{
|
||||
{name: "negative window", mutate: func(request *BuildRequest) { request.WindowUnits = -1 }, want: "window_units"},
|
||||
{name: "blank selected lane", mutate: func(request *BuildRequest) { request.SelectedLanes = []string{" "} }, want: "selected_lanes"},
|
||||
{name: "duplicate selected lane", mutate: func(request *BuildRequest) { request.SelectedLanes = []string{"npcs", " npcs "} }, want: "duplicated"},
|
||||
{name: "unselected contribution", mutate: func(request *BuildRequest) {
|
||||
request.LaneEvidence = []LaneEvidence{{LaneID: "other", SourceRefs: []source.SourceRef{ref(3, 3)}}}
|
||||
}, want: "not selected"},
|
||||
{name: "source digest mismatch", mutate: func(request *BuildRequest) { request.Source.Digest = "sha256:" + strings.Repeat("0", 64) }, want: "does not match"},
|
||||
{name: "invalid reference", mutate: func(request *BuildRequest) {
|
||||
request.LaneEvidence = []LaneEvidence{{LaneID: "npcs", SourceRefs: []source.SourceRef{ref(99, 99)}}}
|
||||
}, want: "source reference[0]"},
|
||||
{name: "invalid reference", mutate: func(request *BuildRequest) { request.SourceRefs = []source.SourceRef{ref(99, 99)} }, want: "source reference[0]"},
|
||||
} {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
request := BuildRequest{Source: testDocument(t), SelectedLanes: []string{"npcs"}, LaneEvidence: []LaneEvidence{{LaneID: "npcs", SourceRefs: []source.SourceRef{ref(3, 3)}}}}
|
||||
request := BuildRequest{Source: testDocument(t), SourceRefs: []source.SourceRef{ref(3, 3)}}
|
||||
test.mutate(&request)
|
||||
if _, err := Build(request); err == nil || !strings.Contains(err.Error(), test.want) {
|
||||
t.Fatalf("Build() error = %v, want %q", err, test.want)
|
||||
@@ -162,7 +85,7 @@ func TestBuildRejectsInvalidInputs(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestCodecRoundTripsCompactFixtureAndOwnsDecodedValues(t *testing.T) {
|
||||
func TestCodecRoundTripsFixtureAndOwnsValues(t *testing.T) {
|
||||
fixture, err := os.ReadFile("testdata/source_evidence_context.v1.json")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
@@ -179,15 +102,16 @@ func TestCodecRoundTripsCompactFixtureAndOwnsDecodedValues(t *testing.T) {
|
||||
if !bytes.Equal(encoded, bytes.TrimSpace(fixture)) {
|
||||
t.Fatalf("fixture does not use canonical encoding\nwant: %s\n got: %s", fixture, encoded)
|
||||
}
|
||||
value.Contexts[0].Units[0].Text = "changed"
|
||||
value[0].Text = "changed"
|
||||
decoded, err := codec.Decode(encoded)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if decoded.Contexts[0].Units[0].Text != "The party meets Rowan." {
|
||||
if decoded[0].Text != "The party meets Rowan." {
|
||||
t.Fatal("Encode() retained mutable document storage")
|
||||
}
|
||||
built, err := Build(BuildRequest{Source: testDocument(t), WindowUnits: 1, SelectedLanes: []string{"npcs"}, LaneEvidence: []LaneEvidence{{LaneID: "npcs", SourceRefs: []source.SourceRef{ref(3, 3)}}}})
|
||||
|
||||
built, err := Build(BuildRequest{Source: testDocument(t), WindowUnits: 1, SourceRefs: []source.SourceRef{ref(3, 3)}})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -203,82 +127,52 @@ func TestCodecRoundTripsCompactFixtureAndOwnsDecodedValues(t *testing.T) {
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
first.Contexts[0].Units[0].Metadata["nested"].(map[string]any)["value"] = "changed"
|
||||
if second.Contexts[0].Units[0].Metadata["nested"].(map[string]any)["value"] != "original" {
|
||||
first[0].Metadata["nested"].(map[string]any)["value"] = "changed"
|
||||
if second[0].Metadata["nested"].(map[string]any)["value"] != "original" {
|
||||
t.Fatal("Decode() returned metadata aliases")
|
||||
}
|
||||
}
|
||||
|
||||
func TestCodecRejectsInvalidDurableBoundaries(t *testing.T) {
|
||||
value, err := Build(BuildRequest{Source: testDocument(t), SelectedLanes: []string{"npcs"}, LaneEvidence: []LaneEvidence{{LaneID: "npcs", SourceRefs: []source.SourceRef{ref(3, 3)}}}})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
func TestCodecRejectsInvalidDurablePayloads(t *testing.T) {
|
||||
for _, test := range []struct {
|
||||
name string
|
||||
mutate func(*Document)
|
||||
name string
|
||||
content string
|
||||
}{
|
||||
{name: "unsorted lanes", mutate: func(value *Document) { value.SelectedLanes = []string{"z", "a"} }},
|
||||
{name: "context range mismatch", mutate: func(value *Document) { value.Contexts[0].ContextRef.EndUnitID = 999 }},
|
||||
{name: "mismatched evidence source", mutate: func(value *Document) { value.Contexts[0].EvidenceRefs[0].SourceRef.SourceID = "other" }},
|
||||
{name: "invalid evidence range", mutate: func(value *Document) {
|
||||
value.Contexts[0].EvidenceRefs[0].SourceRef.StartUnitID = 10
|
||||
}},
|
||||
{name: "duplicate context unit", mutate: func(value *Document) { value.Contexts = append(value.Contexts, value.Contexts[0]) }},
|
||||
{name: "null", content: "null"},
|
||||
{name: "wrapper object", content: `{"units":[]}`},
|
||||
{name: "missing required unit field", content: `[{"id":1,"kind":"segment","ref":{"source_id":"session","start_unit_id":1,"end_unit_id":1}}]`},
|
||||
{name: "unknown unit field", content: `[{"id":1,"kind":"segment","text":"text","ref":{"source_id":"session","start_unit_id":1,"end_unit_id":1},"unknown":true}]`},
|
||||
{name: "unknown reference field", content: `[{"id":1,"kind":"segment","text":"text","ref":{"source_id":"session","start_unit_id":1,"end_unit_id":1,"unknown":true}}]`},
|
||||
{name: "invalid self reference", content: `[{"id":1,"kind":"segment","text":"text","ref":{"source_id":"session","start_unit_id":1,"end_unit_id":2}}]`},
|
||||
{name: "mixed source documents", content: `[{"id":1,"kind":"segment","text":"one","ref":{"source_id":"session-one","start_unit_id":1,"end_unit_id":1}},{"id":2,"kind":"segment","text":"two","ref":{"source_id":"session-two","start_unit_id":2,"end_unit_id":2}}]`},
|
||||
{name: "duplicate units", content: `[{"id":1,"kind":"segment","text":"one","ref":{"source_id":"session","start_unit_id":1,"end_unit_id":1}},{"id":1,"kind":"segment","text":"two","ref":{"source_id":"session","start_unit_id":1,"end_unit_id":1}}]`},
|
||||
{name: "multiple JSON values", content: `[] []`},
|
||||
} {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
candidate, err := clone(value)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
test.mutate(&candidate)
|
||||
if _, err := New().Encode(candidate); err == nil {
|
||||
t.Fatal("Encode() error = nil, want durable model rejection")
|
||||
}
|
||||
})
|
||||
}
|
||||
content, err := New().Encode(value)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, test := range []struct {
|
||||
name string
|
||||
mutate func(map[string]any)
|
||||
}{
|
||||
{name: "missing contexts", mutate: func(value map[string]any) { delete(value, "contexts") }},
|
||||
{name: "null contexts", mutate: func(value map[string]any) { value["contexts"] = nil }},
|
||||
{name: "unknown fixed field", mutate: func(value map[string]any) { value["unknown"] = true }},
|
||||
{name: "missing units", mutate: func(value map[string]any) { delete(contextObject(value, 0), "units") }},
|
||||
{name: "null evidence refs", mutate: func(value map[string]any) { contextObject(value, 0)["evidence_refs"] = nil }},
|
||||
} {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
raw := decodeJSON(t, content)
|
||||
test.mutate(raw)
|
||||
mutated, err := json.Marshal(raw)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := New().Decode(mutated); err == nil {
|
||||
if _, err := New().Decode([]byte(test.content)); err == nil {
|
||||
t.Fatal("Decode() error = nil, want strict payload rejection")
|
||||
}
|
||||
})
|
||||
}
|
||||
if _, err := New().Decode(append(content, []byte(" {}")...)); err == nil {
|
||||
t.Fatal("Decode() error = nil, want trailing JSON rejection")
|
||||
if _, err := New().Encode(nil); err == nil {
|
||||
t.Fatal("Encode(nil) error = nil, want array rejection")
|
||||
}
|
||||
}
|
||||
|
||||
func TestSerializeUsesFixedArtifactIdentity(t *testing.T) {
|
||||
artifact, err := Serialize(BuildRequest{Source: testDocument(t), SelectedLanes: []string{"npcs"}})
|
||||
func TestSerializeUsesFixedArtifactIdentityAndEmptyArray(t *testing.T) {
|
||||
artifact, err := Serialize(BuildRequest{Source: testDocument(t)})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if artifact.Kind != ArtifactKind || artifact.MediaType != MediaType || artifact.Schema.ID != SchemaID || artifact.Schema.Name != SchemaName || artifact.Schema.Version != SchemaVersion {
|
||||
t.Fatalf("Serialize() = %#v, want fixed artifact identity", artifact)
|
||||
}
|
||||
if string(artifact.Content) != "[]" {
|
||||
t.Fatalf("Serialize() content = %s, want []", artifact.Content)
|
||||
}
|
||||
decoded, err := New().Decode(artifact.Content)
|
||||
if err != nil || len(decoded.Contexts) != 0 || decoded.Contexts == nil {
|
||||
t.Fatalf("Decode(Serialize()) = %#v, %v; want explicit empty contexts", decoded, err)
|
||||
if err != nil || decoded == nil || len(decoded) != 0 {
|
||||
t.Fatalf("Decode(Serialize()) = %#v, %v; want explicit empty array", decoded, err)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -306,45 +200,10 @@ func ref(start, end int) source.SourceRef {
|
||||
return source.SourceRef{SourceID: "session", StartUnitID: start, EndUnitID: end}
|
||||
}
|
||||
|
||||
func contextUnitIDs(contexts []Context) [][]int {
|
||||
values := make([][]int, len(contexts))
|
||||
for index, context := range contexts {
|
||||
values[index] = make([]int, len(context.Units))
|
||||
for unitIndex, unit := range context.Units {
|
||||
values[index][unitIndex] = unit.ID
|
||||
}
|
||||
func unitIDs(units Document) []int {
|
||||
values := make([]int, len(units))
|
||||
for index, unit := range units {
|
||||
values[index] = unit.ID
|
||||
}
|
||||
return values
|
||||
}
|
||||
|
||||
func contextEvidenceRefs(contexts []Context) [][]EvidenceRef {
|
||||
values := make([][]EvidenceRef, len(contexts))
|
||||
for index, context := range contexts {
|
||||
values[index] = append([]EvidenceRef(nil), context.EvidenceRefs...)
|
||||
}
|
||||
return values
|
||||
}
|
||||
|
||||
func containsEvidenceRef(values []EvidenceRef, want source.SourceRef) bool {
|
||||
for _, value := range values {
|
||||
if value.SourceRef == want {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func decodeJSON(t *testing.T, content []byte) map[string]any {
|
||||
t.Helper()
|
||||
decoder := json.NewDecoder(bytes.NewReader(content))
|
||||
decoder.UseNumber()
|
||||
var value map[string]any
|
||||
if err := decoder.Decode(&value); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return value
|
||||
}
|
||||
|
||||
func contextObject(value map[string]any, index int) map[string]any {
|
||||
return value["contexts"].([]any)[index].(map[string]any)
|
||||
}
|
||||
|
||||
@@ -14,37 +14,12 @@ const (
|
||||
MediaType = "application/json"
|
||||
)
|
||||
|
||||
// Document is the durable union of direct evidence and surrounding source
|
||||
// context selected for one accepted source document.
|
||||
type Document struct {
|
||||
SourceID string `json:"source_id"`
|
||||
SourceDigest string `json:"source_digest"`
|
||||
WindowUnits int `json:"window_units"`
|
||||
SelectedLanes []string `json:"selected_lanes"`
|
||||
Contexts []Context `json:"contexts"`
|
||||
}
|
||||
// Document is the durable selected source-unit excerpt.
|
||||
type Document []source.SourceUnit
|
||||
|
||||
type Context struct {
|
||||
ContextRef source.SourceRef `json:"context_ref"`
|
||||
EvidenceRefs []EvidenceRef `json:"evidence_refs"`
|
||||
Units []source.SourceUnit `json:"units"`
|
||||
}
|
||||
|
||||
type EvidenceRef struct {
|
||||
LaneID string `json:"lane_id"`
|
||||
SourceRef source.SourceRef `json:"source_ref"`
|
||||
}
|
||||
|
||||
// LaneEvidence attributes direct source references to one selected lane.
|
||||
type LaneEvidence struct {
|
||||
LaneID string `json:"lane_id"`
|
||||
SourceRefs []source.SourceRef `json:"source_refs"`
|
||||
}
|
||||
|
||||
// BuildRequest supplies accepted source material and direct lane evidence.
|
||||
// BuildRequest supplies accepted source material and projected source references.
|
||||
type BuildRequest struct {
|
||||
Source *source.SourceDocument
|
||||
WindowUnits int
|
||||
SelectedLanes []string
|
||||
LaneEvidence []LaneEvidence
|
||||
Source *source.SourceDocument
|
||||
WindowUnits int
|
||||
SourceRefs []source.SourceRef
|
||||
}
|
||||
|
||||
@@ -1 +1 @@
|
||||
{"source_id":"session-alpha","source_digest":"sha256:0000000000000000000000000000000000000000000000000000000000000000","window_units":0,"selected_lanes":["npcs"],"contexts":[{"context_ref":{"source_id":"session-alpha","start_unit_id":13,"end_unit_id":13},"evidence_refs":[{"lane_id":"npcs","source_ref":{"source_id":"session-alpha","start_unit_id":13,"end_unit_id":13}}],"units":[{"id":13,"kind":"transcript_segment","text":"The party meets Rowan.","ref":{"source_id":"session-alpha","start_unit_id":13,"end_unit_id":13}}]}]}
|
||||
[{"id":13,"kind":"transcript_segment","text":"The party meets Rowan.","ref":{"source_id":"session-alpha","start_unit_id":13,"end_unit_id":13}}]
|
||||
|
||||
@@ -20,7 +20,6 @@ type debugEvidenceContextSummary struct {
|
||||
SchemaVersion string `json:"schema_version"`
|
||||
SelectedLanes []string `json:"selected_lanes"`
|
||||
WindowUnits int `json:"window_units"`
|
||||
ContextCount int `json:"context_count"`
|
||||
UnitCount int `json:"unit_count"`
|
||||
SourceDigest string `json:"source_digest"`
|
||||
}
|
||||
@@ -49,10 +48,9 @@ func buildOutputEvidenceContext(prepared *PreparedPipeline, doc *source.SourceDo
|
||||
}
|
||||
|
||||
request := evidencecontext.BuildRequest{
|
||||
Source: doc,
|
||||
WindowUnits: prepared.evidencePlan.policy.WindowUnits,
|
||||
SelectedLanes: append([]string(nil), prepared.evidencePlan.policy.LaneIDs...),
|
||||
LaneEvidence: make([]evidencecontext.LaneEvidence, 0, len(prepared.evidencePlan.lanes)),
|
||||
Source: doc,
|
||||
WindowUnits: prepared.evidencePlan.policy.WindowUnits,
|
||||
SourceRefs: make([]source.SourceRef, 0),
|
||||
}
|
||||
for _, lane := range prepared.evidencePlan.lanes {
|
||||
output, ok := byLane[lane.laneID]
|
||||
@@ -73,10 +71,7 @@ func buildOutputEvidenceContext(prepared *PreparedPipeline, doc *source.SourceDo
|
||||
if err != nil {
|
||||
return nil, nil, fmt.Errorf("evidence context output lane %q: accepted normalized artifact cannot be projected", lane.laneID)
|
||||
}
|
||||
request.LaneEvidence = append(request.LaneEvidence, evidencecontext.LaneEvidence{
|
||||
LaneID: lane.laneID,
|
||||
SourceRefs: append([]source.SourceRef(nil), references...),
|
||||
})
|
||||
request.SourceRefs = append(request.SourceRefs, references...)
|
||||
}
|
||||
|
||||
document, err := evidencecontext.Build(request)
|
||||
@@ -99,13 +94,10 @@ func buildOutputEvidenceContext(prepared *PreparedPipeline, doc *source.SourceDo
|
||||
SchemaID: artifact.Schema.ID,
|
||||
SchemaName: artifact.Schema.Name,
|
||||
SchemaVersion: artifact.Schema.Version,
|
||||
SelectedLanes: append([]string(nil), document.SelectedLanes...),
|
||||
WindowUnits: document.WindowUnits,
|
||||
ContextCount: len(document.Contexts),
|
||||
SourceDigest: document.SourceDigest,
|
||||
}
|
||||
for _, context := range document.Contexts {
|
||||
summary.UnitCount += len(context.Units)
|
||||
SelectedLanes: append([]string(nil), prepared.evidencePlan.policy.LaneIDs...),
|
||||
WindowUnits: prepared.evidencePlan.policy.WindowUnits,
|
||||
SourceDigest: doc.Digest,
|
||||
UnitCount: len(document),
|
||||
}
|
||||
return contracts.CloneSerializedArtifactPointer(artifact), &summary, nil
|
||||
}
|
||||
|
||||
@@ -97,14 +97,11 @@ func TestRunnerBuildsEvidenceContextFromSelectedNormalizedOutputs(t *testing.T)
|
||||
}
|
||||
|
||||
value := decodeCapturedEvidence(t, encoder)
|
||||
if !reflect.DeepEqual(value.SelectedLanes, []string{"alpha", "beta", "inactive"}) || len(value.Contexts) != 1 || len(value.Contexts[0].Units) != 3 {
|
||||
if actual := []int{value[0].ID, value[1].ID, value[2].ID}; !reflect.DeepEqual(actual, []int{1, 2, 3}) {
|
||||
t.Fatalf("evidence context = %#v, want selected union", value)
|
||||
}
|
||||
if got := value.Contexts[0].EvidenceRefs; len(got) != 2 || got[0].LaneID != "alpha" || got[1].LaneID != "beta" {
|
||||
t.Fatalf("evidence refs = %#v, want both selected lanes", got)
|
||||
}
|
||||
debugJSON := string(debug.json["output/evidence-context.json"])
|
||||
if strings.Contains(debugJSON, "text-1") || strings.Contains(debugJSON, "metadata") || !strings.Contains(debugJSON, `"artifact_kind":"source/evidence-context"`) || !strings.Contains(debugJSON, `"schema_id":"notarius.source.evidence_context"`) || !strings.Contains(debugJSON, `"context_count":1`) || !strings.Contains(debugJSON, `"unit_count":3`) {
|
||||
if strings.Contains(debugJSON, "text-1") || strings.Contains(debugJSON, "metadata") || !strings.Contains(debugJSON, `"artifact_kind":"source/evidence-context"`) || !strings.Contains(debugJSON, `"schema_id":"notarius.source.evidence_context"`) || strings.Contains(debugJSON, "context_count") || !strings.Contains(debugJSON, `"unit_count":3`) {
|
||||
t.Fatalf("evidence debug envelope = %s, want only allowlisted summary", debugJSON)
|
||||
}
|
||||
}
|
||||
@@ -134,7 +131,7 @@ func TestRunnerEvidenceContextOmitsAbsentAndRejectedLanes(t *testing.T) {
|
||||
t.Fatalf("rejections = %#v, want rejected lane unchanged", result.Rejected)
|
||||
}
|
||||
value := decodeCapturedEvidence(t, encoder)
|
||||
if len(value.Contexts) != 1 || len(value.Contexts[0].EvidenceRefs) != 1 || value.Contexts[0].EvidenceRefs[0].LaneID != "present" {
|
||||
if len(value) != 1 || value[0].ID != 1 {
|
||||
t.Fatalf("evidence context = %#v, want present lane only", value)
|
||||
}
|
||||
}
|
||||
@@ -232,7 +229,7 @@ func TestRunnerEvidenceContextRebuildsFromAcceptedCheckpoint(t *testing.T) {
|
||||
t.Fatalf("Run() error = %v", err)
|
||||
}
|
||||
value := decodeCapturedEvidence(t, encoder)
|
||||
if len(value.Contexts) != 1 || value.Contexts[0].EvidenceRefs[0].LaneID != "notes" {
|
||||
if len(value) != 1 || value[0].ID != 1 {
|
||||
t.Fatalf("evidence context = %#v, want checkpointed normalized output", value)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -353,8 +353,8 @@ func TestEncodeIncludesValidatedEvidenceContext(t *testing.T) {
|
||||
if err != nil {
|
||||
t.Fatalf("Decode(evidence context file) error = %v", err)
|
||||
}
|
||||
if len(value.Contexts) != 0 {
|
||||
t.Fatalf("evidence context = %#v, want explicit empty contexts", value)
|
||||
if len(value) != 1 || value[0].ID != 7 || value[0].Text != "Source content retained only in the evidence artifact." {
|
||||
t.Fatalf("evidence context = %#v, want the published source-unit array", value)
|
||||
}
|
||||
index := decodeObject(t, fileBytes(t, result.Files, "index.json"))
|
||||
if got, want := index["evidence_context"], map[string]any{
|
||||
@@ -786,7 +786,8 @@ func acceptedEvidenceContextArtifact(t *testing.T) contracts.SerializedArtifact
|
||||
}
|
||||
document.Digest = digest
|
||||
artifact, err := evidencecontext.Serialize(evidencecontext.BuildRequest{
|
||||
Source: document, WindowUnits: 3, SelectedLanes: []string{"spells"},
|
||||
Source: document, WindowUnits: 3,
|
||||
SourceRefs: []source.SourceRef{{SourceID: "source-1", StartUnitID: 7, EndUnitID: 7}},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
|
||||
@@ -4,6 +4,7 @@ import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"reflect"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
@@ -71,20 +72,17 @@ func TestLocationRegistryHandoffProducesOccurrencesAndEvidence(t *testing.T) {
|
||||
if err != nil {
|
||||
t.Fatalf("Decode(evidence context) error = %v", err)
|
||||
}
|
||||
if !locationEvidenceHasLane(evidence, "locations") || !locationEvidenceHasLane(evidence, "occurrences") {
|
||||
t.Fatalf("evidence context = %#v, want registry and occurrence evidence from their own artifacts", evidence)
|
||||
if actual := locationEvidenceUnitIDs(evidence); !reflect.DeepEqual(actual, []int{1, 2, 3, 4, 5}) {
|
||||
t.Fatalf("evidence context = %#v, want deduplicated registry and occurrence source-unit evidence", evidence)
|
||||
}
|
||||
}
|
||||
|
||||
func locationEvidenceHasLane(document evidencecontext.Document, laneID string) bool {
|
||||
for _, context := range document.Contexts {
|
||||
for _, reference := range context.EvidenceRefs {
|
||||
if reference.LaneID == laneID {
|
||||
return true
|
||||
}
|
||||
}
|
||||
func locationEvidenceUnitIDs(document evidencecontext.Document) []int {
|
||||
ids := make([]int, len(document))
|
||||
for index, unit := range document {
|
||||
ids[index] = unit.ID
|
||||
}
|
||||
return false
|
||||
return ids
|
||||
}
|
||||
|
||||
func TestLocationOccurrenceConsumerDoesNotRunAfterRejectedRegistry(t *testing.T) {
|
||||
|
||||
@@ -279,22 +279,8 @@ func TestProductionDNDOutputPublishesSelectedEvidenceContext(t *testing.T) {
|
||||
if err != nil {
|
||||
t.Fatalf("Decode(evidence context) error = %v", err)
|
||||
}
|
||||
if !reflect.DeepEqual(value.SelectedLanes, []string{"combat", "npc_registry", "spells"}) {
|
||||
t.Fatalf("selected lanes = %#v, want configured production lanes without scene descriptions", value.SelectedLanes)
|
||||
}
|
||||
if len(value.Contexts) != 2 || len(value.Contexts[0].Units) != 1 || len(value.Contexts[1].Units) != 1 || value.Contexts[0].Units[0].ID != 10 || value.Contexts[1].Units[0].ID != 20 {
|
||||
t.Fatalf("evidence contexts = %#v, want source-position union with non-monotonic unit IDs", value.Contexts)
|
||||
}
|
||||
firstRefs := value.Contexts[0].EvidenceRefs
|
||||
if len(firstRefs) != 3 || firstRefs[0].LaneID != "combat" || firstRefs[1].LaneID != "npc_registry" || firstRefs[2].LaneID != "spells" {
|
||||
t.Fatalf("first context evidence = %#v, want overlapping selected lane references", firstRefs)
|
||||
}
|
||||
for _, context := range value.Contexts {
|
||||
for _, reference := range context.EvidenceRefs {
|
||||
if reference.LaneID == "scene-descriptions" {
|
||||
t.Fatalf("evidence refs = %#v, want scene descriptions excluded by allowlist", value.Contexts)
|
||||
}
|
||||
}
|
||||
if actual := evidenceUnitIDs(value); !reflect.DeepEqual(actual, []int{10, 20}) {
|
||||
t.Fatalf("evidence units = %#v, want deduplicated source-position union without scene descriptions", actual)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -312,11 +298,19 @@ func TestProductionDNDOutputCanExplicitlySelectSceneDescriptionEvidence(t *testi
|
||||
if err != nil {
|
||||
t.Fatalf("Decode(evidence context) error = %v", err)
|
||||
}
|
||||
if !reflect.DeepEqual(value.SelectedLanes, []string{"scene-descriptions"}) || len(value.Contexts) == 0 || len(value.Contexts[0].EvidenceRefs) == 0 || value.Contexts[0].EvidenceRefs[0].LaneID != "scene-descriptions" {
|
||||
t.Fatalf("evidence context = %#v, want explicitly selected scene-description evidence", value)
|
||||
if len(value) == 0 {
|
||||
t.Fatalf("evidence context = %#v, want explicitly selected scene-description source units", value)
|
||||
}
|
||||
}
|
||||
|
||||
func evidenceUnitIDs(value evidencecontext.Document) []int {
|
||||
ids := make([]int, len(value))
|
||||
for index, unit := range value {
|
||||
ids[index] = unit.ID
|
||||
}
|
||||
return ids
|
||||
}
|
||||
|
||||
func TestGroundedPipelineSkipsCombatForExactNarrativeScene(t *testing.T) {
|
||||
registries := productionNPCRegistries(t)
|
||||
configValue := loadGroundedPipelineConfig(t)
|
||||
|
||||
Reference in New Issue
Block a user