5 Commits

762 changed files with 16379 additions and 60801 deletions

3
.codebase-memory/.gitattributes vendored Normal file
View File

@@ -0,0 +1,3 @@
# Auto-generated by codebase-memory-mcp
# Prevent merge conflicts on compressed artifact
graph.db.zst merge=ours binary

View File

@@ -0,0 +1,11 @@
{
"schema_version": 2,
"commit": "9e4b989e53d65efa614b5eedd8530caed12f60b5",
"indexed_at": "2026-07-27T18:27:55Z",
"project": "home-eric-Workspace-notarius",
"nodes": 6356,
"edges": 35132,
"original_size": 26017792,
"compressed_size": 4386397,
"compression_level": 3
}

Binary file not shown.

2
.gitignore vendored
View File

@@ -2,7 +2,6 @@
notarius notarius
notarius-output notarius-output
workspace/ workspace/
.codebase-memory/
# ---> Go # ---> Go
# If you prefer the allow list template instead of the deny list, see community template: # If you prefer the allow list template instead of the deny list, see community template:
@@ -74,3 +73,4 @@ Icon
Network Trash Folder Network Trash Folder
Temporary Items Temporary Items
.apdisk .apdisk

View File

@@ -1,9 +0,0 @@
# Advisory validation after source publication, never a release-completion gate.
when:
- event: tag
steps:
- name: validate-release
image: golang:1.25.5
commands:
- ./scripts/check-release-source.sh "$CI_COMMIT_TAG"

View File

@@ -1,5 +1,2 @@
Please review `docs/development.md` for initial orientation in this repository Please review `docs/development.md` for initial orientation in this repository
and follow its task-specific reading guide. and follow its task-specific reading guide.
Review and follow `docs/policy/agents.md` before recommending, planning, or
implementing any changes to this repository.

View File

@@ -2,9 +2,8 @@
Notarius is a Go CLI for turning source material into structured artifacts with Notarius is a Go CLI for turning source material into structured artifacts with
configured extraction pipelines. The implemented D&D workflow reads Seriatim configured extraction pipelines. The implemented D&D workflow reads Seriatim
transcript JSON and can produce NPC, location, and item registries; their transcript JSON and can produce scene descriptions, item and currency events,
source-grounded occurrences; scene descriptions, combat turns, enemy events, NPC identities, combat turns, NPC interactions, and spell casts.
and spell casts.
## Quickstart ## Quickstart
@@ -28,20 +27,6 @@ For the complete ordered D&D workflow, use
[its synthetic transcript](examples/dnd-complete-transcript.json). It [its synthetic transcript](examples/dnd-complete-transcript.json). It
demonstrates all implemented D&D lanes and the supporting campaign references. demonstrates all implemented D&D lanes and the supporting campaign references.
## Install A Source Release
Install a pinned source release with Go:
~~~
GOWORK=off go install \
gitea.maximumdirect.net/eric/notarius/cmd/notarius@<tag>
~~~
Replace `<tag>` with a stable release tag such as `vMAJOR.MINOR.PATCH`. The
installed command's diagnostic version is described in the [CLI
reference](docs/cli.md); maintainers preparing a release should follow [Source
Releases](docs/release.md).
## Documentation ## Documentation
- [CLI reference](docs/cli.md) — commands, flags, output streams, and exits. - [CLI reference](docs/cli.md) — commands, flags, output streams, and exits.
@@ -53,8 +38,6 @@ Releases](docs/release.md).
artifact formats. artifact formats.
- [Subprocess consumer guide](docs/consumers/subprocess.md) — invoke Notarius - [Subprocess consumer guide](docs/consumers/subprocess.md) — invoke Notarius
from an orchestrator and consume a published result. from an orchestrator and consume a published result.
- [Complete D&D consumer guide](docs/consumers/dnd-pipeline.md) — run the full
D&D pipeline as a subprocess and discover its structured artifacts.
- [Internal overview](docs/internal/overview.md) — implemented component map - [Internal overview](docs/internal/overview.md) — implemented component map
for maintainers. for maintainers.
- [Developer guide](docs/development.md) — contributor orientation and - [Developer guide](docs/development.md) — contributor orientation and

View File

@@ -1,18 +0,0 @@
Extract Dungeons & Dragons combat-turn artifacts from the supplied transcript.
Include a record only when the transcript establishes that an in-world
participant takes a combat turn or performs a discrete interrupting combat
event. Keep events in transcript chronology; place an interrupting event where
it occurs.
Exclude initiative setup without a turn or combat event, tactical planning,
table talk, rules lookup, hypothetical events, abandoned intentions, recaps
outside the current passage, and downstream consequences. Do not infer combat
events from Dungeons & Dragons rules knowledge. Preserve the session as played
and attribute relevant nonstandard rulings to the GM or table. Unmatched actors
remain permitted.
Treat each record as one turn-level event and keep its supporting transcript
evidence together. Use `turn` for a regular combat turn, `reaction` for an
off-turn reaction, `legendary_action` for a legendary action,
`lair_action` for a lair action, and `other` for another discrete combat
event that does not fit those categories.

View File

@@ -1,12 +0,0 @@
Compact combat grounding is supplied below. It can guide attention and
disambiguation, but it is not evidence. Do not derive an event, subject,
outcome, or source range from either list. The current transcript alone must
directly establish every returned event.
Combat-turn grounding:
{{ input "combat_turns" }}
Named combat-opponent grounding:
{{ input "npc_occurrences" }}

View File

@@ -1,25 +0,0 @@
Extract Dungeons & Dragons enemy events from the supplied combat transcript.
An `engaged` event requires direct establishment that a subject is actively
opposing the party in combat. A `killed`, `fled`, `captured`, or
`incapacitated` event requires explicit establishment of that outcome. An
outcome may share evidence with an engagement, and a later engagement or
outcome for the same subject remains a separate observation. Emit at most one
`engaged` observation for the same subject in this combat scene.
For `killed`, direct death or killing is required. For `fled`, the subject
must explicitly escape, retreat, or leave combat to avoid continued engagement.
For `captured`, the subject must be explicitly taken prisoner or secured
under the party's control. For `incapacitated`, the subject must be explicitly
unable to continue acting without being established as killed or captured.
When the transcript identifies a named NPC, use its normalized registry
spelling. A hostile creature without a registry entry is allowed. For unnamed
individuals or groups, use only the narrowest transcript-grounded label, such
as `Orcs`, `One orc`, or `Remaining orcs`; never invent member names, IDs,
or quantities.
Exclude party members, allies, neutral observers, mentioned-but-absent enemies,
hazards, traps, environmental effects, uncertain allegiance, table talk,
planning, hypotheses, recaps outside this passage, and downstream inference.
Do not infer an engagement or outcome from initiative, turn absence, damage,
defeat, movement, or a scene ending.

View File

@@ -1,53 +0,0 @@
id: dnd.enemy_events
version: "v1"
default_profile: dnd-extraction
inputs:
- name: transcript
required: true
content_type: application/json
- name: players
required: false
content_type: text/plain
- name: party
required: false
content_type: text/plain
- name: glossary
required: false
content_type: text/plain
- name: npc_registry
required: true
content_type: application/json
- name: combat_turns
required: true
content_type: application/json
- name: npc_occurrences
required: true
content_type: application/json
messages:
- role: system
content_file: ./sharedassets/common-dnd-system.md
- role: user
content_file: ./sharedassets/common-dnd-identity.md
- role: user
content_file: ./sharedassets/common-dnd-references.md
cache_control:
type: ephemeral
- role: user
content_file: ./sharedassets/common-dnd-transcript-chunk.md
cache_control:
type: ephemeral
- role: user
content_file: ./sharedassets/common-dnd-extraction-evidence.md
- role: user
content_file: ./sharedassets/common-dnd-npc-registry.md
- role: user
content_file: ./combat-grounding.md
- role: user
content_file: ./instructions.md
cache_control:
type: ephemeral
output:
format: json
validation_mode: json_schema
schema_path: dnd_enemy_events_llm.v1.json
repair_attempts: 1

View File

@@ -1,38 +0,0 @@
Extract meaningful Dungeons & Dragons item and currency occurrences: discoveries and changes
in party possession established by the transcript. This is an occurrence history,
not an inventory or ledger: do not calculate balances, resolve item identity
across records, or infer ownership that the transcript does not establish.
For every occurrence, use the supplied canonical item `name`. Record a stated
quantity as an integer and leave it null when the transcript does not state
one. Preserve the stated currency denomination through the selected canonical
registry name.
Use `discovered` when the party learns of or encounters an item without
establishing possession. Use `acquired` when the party or a party member gains
possession. Use `lost` when party possession ends through a gift, sale, payment,
theft, abandonment, or destruction not caused by intended use. Use `consumed`
when intended use depletes an expendable item. Monetary spending, purchases, and
payments are always `lost`, not `consumed`. Classify currency as `consumed` only
when the transcript explicitly describes it being physically destroyed or
expended as a non-payment component. Use `transferred` only when possession
moves between two distinct named party members.
Return both `from` and `to` for every occurrence. Use JSON `null`, not an empty
string, whenever a holder does not apply. Follow this holder matrix exactly:
| `kind` | required `from` | required `to` |
| --- | --- | --- |
| `discovered` | `null` | `null` |
| `acquired` | `null` | `party` or the named party member gaining possession |
| `lost` | `party` or the named party member losing possession | `null` |
| `consumed` | `party` or the named party member consuming the item | `null` |
| `transferred` | one named party member | a different named party member |
Use `party` only for collective or unresolved party possession, never for
either side of a transfer. Do not emit a transfer for a gift, sale, or payment
outside the party.
Ordinary non-depleting use is not an occurrence. Do not infer acquisition from a
discovery, or discovery from an acquisition: emit both only when each is
independently established.

View File

@@ -1,6 +0,0 @@
Use the supplied item registry only to ground each occurrence. Every record
must use one registry item's canonical `name`; do not invent, rename, merge,
or infer registry items. The registry is not transcript evidence: cite only the
current transcript chunk in `source_refs`.
{{ input "item_registry" }}

View File

@@ -1,45 +0,0 @@
id: dnd.item_occurrences
version: "v1"
default_profile: dnd-extraction
inputs:
- name: transcript
required: true
content_type: application/json
- name: players
required: false
content_type: text/plain
- name: party
required: false
content_type: text/plain
- name: glossary
required: false
content_type: text/plain
- name: item_registry
required: true
content_type: application/json
messages:
- role: system
content_file: ./sharedassets/common-dnd-system.md
- role: user
content_file: ./sharedassets/common-dnd-identity.md
- role: user
content_file: ./sharedassets/common-dnd-references.md
cache_control:
type: ephemeral
- role: user
content_file: ./sharedassets/common-dnd-transcript-chunk.md
cache_control:
type: ephemeral
- role: user
content_file: ./sharedassets/common-dnd-extraction-evidence.md
- role: user
content_file: ./item-registry.md
- role: user
content_file: ./instructions.md
cache_control:
type: ephemeral
output:
format: json
validation_mode: json_schema
schema_path: dnd_item_occurrences_llm.v1.json
repair_attempts: 1

View File

@@ -1,39 +0,0 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "notarius.dnd.item_occurrences.llm",
"type": "object",
"additionalProperties": false,
"required": ["occurrences"],
"properties": {
"occurrences": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": false,
"required": ["name", "kind", "quantity", "from", "to", "source_refs"],
"properties": {
"name": {"type": "string"},
"kind": {
"type": "string",
"enum": ["discovered", "acquired", "lost", "consumed", "transferred"]
},
"quantity": {"type": ["integer", "null"]},
"from": {"type": ["string", "null"]},
"to": {"type": ["string", "null"]},
"source_refs": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": false,
"required": ["start_unit_id", "end_unit_id"],
"properties": {
"start_unit_id": {"type": "integer"},
"end_unit_id": {"type": "integer"}
}
}
}
}
}
}
}
}

View File

@@ -1,12 +0,0 @@
Extract only items established by the provided Dungeons & Dragons transcript.
Include named unique items, concrete reusable item types, and stable unique
designations. Record each currency denomination separately when it is
established, such as copper pieces, silver pieces, gold pieces, or platinum
pieces. Do not use capitalization as an eligibility test. Keep distinct names
and designations as separate candidates; do not merge aliases or invent
qualifiers.
Do not record vague categories such as "loot", "treasure", or "some gear";
generic weapons; inferred properties; quantities; or inferred uniqueness. Omit
uncertain or unsupported items.

View File

@@ -1,40 +0,0 @@
id: dnd.item_registry
version: "v1"
default_profile: dnd-extraction
inputs:
- name: transcript
required: true
content_type: application/json
- name: players
required: false
content_type: text/plain
- name: party
required: false
content_type: text/plain
- name: glossary
required: false
content_type: text/plain
messages:
- role: system
content_file: ./sharedassets/common-dnd-system.md
- role: user
content_file: ./sharedassets/common-dnd-identity.md
- role: user
content_file: ./sharedassets/common-dnd-references.md
cache_control:
type: ephemeral
- role: user
content_file: ./sharedassets/common-dnd-transcript-chunk.md
cache_control:
type: ephemeral
- role: user
content_file: ./sharedassets/common-dnd-extraction-evidence.md
- role: user
content_file: ./instructions.md
cache_control:
type: ephemeral
output:
format: json
validation_mode: json_schema
schema_path: dnd_item_registry_llm.v1.json
repair_attempts: 1

View File

@@ -1,32 +0,0 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "notarius.dnd.item_registry.llm",
"type": "object",
"additionalProperties": false,
"required": ["items"],
"properties": {
"items": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": false,
"required": ["name", "source_refs"],
"properties": {
"name": {"type": "string"},
"source_refs": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": false,
"required": ["start_unit_id", "end_unit_id"],
"properties": {
"start_unit_id": {"type": "integer"},
"end_unit_id": {"type": "integer"}
}
}
}
}
}
}
}
}

View File

@@ -1,9 +0,0 @@
Determine whether candidates identify the same item type or unique designation
using their contextual labels and cited transcript windows. Do not treat nearby
evidence, similar objects, or a shared owner as sufficient.
Keep currency denominations and materially different item types separate. Keep
uncertain aliases separate. Do not infer an item property or uniqueness.
Set `canonical_candidate_number` to the supplied candidate number whose label
is the clearest established designation.

View File

@@ -1,30 +0,0 @@
id: dnd.item_registry.normalize
version: "v1"
default_profile: dnd-extraction
inputs:
- name: candidates
required: true
content_type: application/json
- name: transcript
required: true
content_type: application/json
messages:
- role: system
content_file: ./sharedassets/common-dnd-system.md
- role: user
content_file: ./sharedassets/protocol.md
- role: user
content_file: ./instructions.md
cache_control:
type: ephemeral
- role: user
content_file: ./sharedassets/candidates.md
- role: user
content_file: ./sharedassets/transcript-windows.md
cache_control:
type: ephemeral
output:
format: json
validation_mode: json_schema
schema_path: semantic_reconciliation_llm.v1.json
repair_attempts: 1

View File

@@ -1,34 +0,0 @@
Extract Dungeons & Dragons location occurrences from the supplied transcript.
Include an occurrence only when the transcript establishes one supplied
location, one occurrence kind, and a coherent passage supporting both.
Use exactly one kind per occurrence:
- visited: party members are physically present, arrive, remain, or depart;
- planned: the party explicitly proposes, intends, or agrees to future travel;
- recalled: the transcript explicitly recounts an earlier party visit; or
- mentioned: the location is explicitly referenced without stronger support,
including non-actionable speculation or a mere hypothetical reference.
A mere hypothetical or speculative reference is not planned unless the
transcript also establishes an actual proposal, intention, or agreement to
travel. When the hypothetical explicitly names a supplied location, it may be
mentioned.
A generic phrase in the current chunk may refer to a supplied named registry
location only when the chunk's context supports that coreference. It must not
create a registry location, and registry content or provenance must never
replace current-chunk evidence.
For every occurrence, return the exact selector from the location registry:
the canonical `name`, plus an empty `registry_refs` array for a unique name or
the complete ordered `registry_refs` array for a repeated name. Registry ranges
and context identify the location only; they are not occurrence evidence.
For overlapping support, visited outranks planned, recalled, and mentioned;
planned outranks recalled and mentioned; recalled outranks mentioned. A passage
may produce multiple records when it independently establishes separate facts,
such as recalling an earlier visit while planning a return. Omit inferred,
unstated, uncertain, or unsupported places and occurrences. Do not infer a
location or occurrence from surrounding events when the transcript does not
state it. Do not summarize location descriptions.

View File

@@ -1,11 +0,0 @@
A contextual location registry is provided below for identity grounding. It may
be empty. Every record supplies a canonical display name. A name that appears
once is selected with that name and an empty `registry_refs` array. A repeated
name is selected only by copying both its name and its complete, ordered
`registry_refs` array exactly as supplied.
Registry content is context, not occurrence evidence. Do not derive an
occurrence or `source_refs` range from the registry. Do not invent a location
or selector that is absent from it.
{{ input "location_registry" }}

View File

@@ -1,45 +0,0 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "notarius.dnd.location_occurrences.llm",
"type": "object",
"additionalProperties": false,
"required": ["occurrences"],
"properties": {
"occurrences": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": false,
"required": ["name", "registry_refs", "kind", "source_refs"],
"properties": {
"name": {"type": "string"},
"registry_refs": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": false,
"required": ["start_unit_id", "end_unit_id"],
"properties": {
"start_unit_id": {"type": "integer", "minimum": 1},
"end_unit_id": {"type": "integer", "minimum": 1}
}
}
},
"kind": {"enum": ["visited", "planned", "recalled", "mentioned"]},
"source_refs": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": false,
"required": ["start_unit_id", "end_unit_id"],
"properties": {
"start_unit_id": {"type": "integer"},
"end_unit_id": {"type": "integer"}
}
}
}
}
}
}
}
}

View File

@@ -1,13 +0,0 @@
Extract only physical places established by the provided Dungeons & Dragons
transcript that have a stable proper name or unique in-world designation. This
includes named planes, regions, settlements, districts, buildings, rooms,
landmarks, routes, and geographic features.
Do not create a registry location for generic, temporary, relative, or merely
descriptive phrases, including "the room", "the bar", "the hallway",
"outside", and "upstairs". Do not use capitalization as an eligibility test.
Keep aliases and nested places when the transcript identifies them; do not merge
or invent qualifiers for similarly named places.
Exclude people, creatures, objects, organizations, abstract concepts, and
places merely inferred from an event. Omit uncertain or unsupported places.

View File

@@ -1,32 +0,0 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "notarius.dnd.location_registry.llm",
"type": "object",
"additionalProperties": false,
"required": ["locations"],
"properties": {
"locations": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": false,
"required": ["name", "source_refs"],
"properties": {
"name": {"type": "string"},
"source_refs": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": false,
"required": ["start_unit_id", "end_unit_id"],
"properties": {
"start_unit_id": {"type": "integer"},
"end_unit_id": {"type": "integer"}
}
}
}
}
}
}
}
}

View File

@@ -1,9 +0,0 @@
Determine whether candidates identify the same physical place using their
contextual labels and cited transcript windows. Do not treat matching names,
nearby evidence, nested places, or generic labels as sufficient.
Keep parent and child places separate, as well as similarly named places and
uncertain aliases.
Set `canonical_candidate_number` to the supplied candidate number whose label
is the clearest established name.

View File

@@ -1,30 +0,0 @@
id: dnd.location_registry.normalize
version: "v1"
default_profile: dnd-extraction
inputs:
- name: candidates
required: true
content_type: application/json
- name: transcript
required: true
content_type: application/json
messages:
- role: system
content_file: ./sharedassets/common-dnd-system.md
- role: user
content_file: ./sharedassets/protocol.md
- role: user
content_file: ./instructions.md
cache_control:
type: ephemeral
- role: user
content_file: ./sharedassets/candidates.md
- role: user
content_file: ./sharedassets/transcript-windows.md
cache_control:
type: ephemeral
output:
format: json
validation_mode: json_schema
schema_path: semantic_reconciliation_llm.v1.json
repair_attempts: 1

View File

@@ -1,45 +0,0 @@
id: dnd.npc_occurrences
version: "v1"
default_profile: dnd-extraction
inputs:
- name: transcript
required: true
content_type: application/json
- name: players
required: false
content_type: text/plain
- name: party
required: false
content_type: text/plain
- name: glossary
required: false
content_type: text/plain
- name: npc_registry
required: true
content_type: application/json
messages:
- role: system
content_file: ./sharedassets/common-dnd-system.md
- role: user
content_file: ./sharedassets/common-dnd-identity.md
- role: user
content_file: ./sharedassets/common-dnd-references.md
cache_control:
type: ephemeral
- role: user
content_file: ./sharedassets/common-dnd-transcript-chunk.md
cache_control:
type: ephemeral
- role: user
content_file: ./sharedassets/common-dnd-extraction-evidence.md
- role: user
content_file: ./sharedassets/common-dnd-npc-registry.md
- role: user
content_file: ./instructions.md
cache_control:
type: ephemeral
output:
format: json
validation_mode: json_schema
schema_path: dnd_npc_occurrences_llm.v1.json
repair_attempts: 1

View File

@@ -1,19 +0,0 @@
Extract the individually identifiable Dungeons & Dragons non-player characters
established by the provided transcript.
Include an in-world non-PC only when the transcript factually establishes a
proper name or a stable, individually distinguishing title or alias. A factual
third-party mention establishes that identity even when the NPC is not
physically present, does not speak, and takes no direct action in this chunk.
Record only the NPC identity and the transcript evidence that establishes it;
do not infer or classify a separate occurrence.
Exclude human players, transcript speakers, and the GM as out-of-world people;
player characters identified by the player or party references; names used only
in hypothetical, speculative, or imagined examples; corrected transcription
mistakes; anonymous or generic roles; indistinguishable crowds or groups;
invented descriptive labels; and temporary summoned creatures or spell effects
without a persistent individual identity.
Preserve observed display spelling. Do not invent a label for an anonymous
creature, crowd, or generic role.

View File

@@ -1,12 +0,0 @@
Determine whether candidates refer to the same individual using their
contextual labels and cited transcript windows. Preserve distinct individuals
even when their names are similar or their contextual descriptions are
identical.
Set `canonical_candidate_number` to the supplied candidate number whose label
is the preferred canonical display name. Prefer a complete, stable proper name
over an abbreviation. Prefer an unadorned proper name over that name plus a
contextual class, role, title, or relationship descriptor unless the transcript
establishes the descriptor as part of the person's name. A longer display name
is not inherently more canonical; for example, do not prefer `Captain Aria`
over `Aria` solely because it includes the contextual title `Captain`.

View File

@@ -1,5 +0,0 @@
id: dnd-extraction
backend: openrouter
model: openai/gpt-5.6-luna
timeout_seconds: 240
service_tier: flex

View File

@@ -1,8 +0,0 @@
Classify only whether substantive active combat occurs in the supplied
transcript chunk under the shared combat policy. Do not judge the scene title,
summary, non-combat subtype, scene boundary, or any other aspect of a scene
description.
Return `combat` when the chunk contains substantive active combat and
`non_combat` otherwise. Give a concise, transcript-grounded explanation for
the classification.

View File

@@ -1,23 +0,0 @@
id: dnd.scene_descriptions.validate_combat
version: "v1"
default_profile: dnd-extraction
inputs:
- name: transcript
required: true
content_type: application/json
messages:
- role: system
content_file: ./sharedassets/common-dnd-system.md
- role: user
content_file: ./sharedassets/common-dnd-scene-combat-policy.md
- role: user
content_file: ./instructions.md
- role: user
content_file: ./sharedassets/common-dnd-transcript-chunk.md
cache_control:
type: ephemeral
output:
format: json
validation_mode: json_schema
schema_path: dnd_scene_combat_semantics_llm.v1.json
repair_attempts: 1

View File

@@ -1,18 +0,0 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "notarius.dnd.scene_descriptions.combat_semantics.llm",
"type": "object",
"additionalProperties": false,
"required": ["classification", "explanation"],
"properties": {
"classification": {
"type": "string",
"enum": ["combat", "non_combat"]
},
"explanation": {
"type": "string",
"minLength": 1,
"maxLength": 512
}
}
}

View File

@@ -1,8 +0,0 @@
Transcript units are the only evidence for extracted events and factual claims.
Every reported factual claim must be supported by cited transcript units. Use
integer `start_unit_id` and `end_unit_id` values from the transcript.
Within each range, `start_unit_id` must identify the earlier transcript unit and
`end_unit_id` the same or a later unit according to transcript order.
When supporting evidence is non-contiguous, use multiple narrow ranges rather
than a broad range that bridges unrelated conversation.

View File

@@ -1,13 +0,0 @@
Use `combat` only when substantive active combat materially organizes the
scene. Active combat includes initiative or turn exchanges, attacks, combat
spells, damage, saves, movement, or similarly sustained hostile action.
Do not use `combat` for planning or preparing for a possible fight; threats,
hostile dialogue, or a tense confrontation; immediate aftermath, looting,
healing, or discussion of a completed fight; a recap or in-world recollection
of earlier combat; or out-of-character rules discussion without active
encounter play.
When a chunk contains substantive active combat alongside brief setup, rules
clarification, interruption, phase transition, or immediate aftermath, classify
it as `combat`.

View File

@@ -1,5 +0,0 @@
You process Dungeons & Dragons gameplay transcripts.
As input, you will receive one or more portions of a transcript. The transcript may contain transcription errors, repeated lines, incomplete sentences, and misheard proper nouns.
Return exactly one JSON object that conforms to the configured response schema, with no explanatory prose.

View File

@@ -1,3 +0,0 @@
One extraction chunk from a Dungeons & Dragons gameplay transcript is provided below. Report and infer only what is within this chunk. Its unit IDs retain their source-wide meaning.
{{ input "transcript" }}

View File

@@ -1,3 +0,0 @@
The complete ordered transcript of this Dungeons & Dragons gameplay session is provided below.
{{ input "transcript" }}

View File

@@ -1,14 +0,0 @@
Extract Dungeons & Dragons spell-cast artifacts from the provided transcript.
Include an actual casting event or an unambiguous declared casting attempt.
Exclude spell mentions, hypothetical plans, rules discussion, and catalog
matches that do not establish a casting event in the transcript.
For every extracted cast, the transcript evidence must collectively support the
in-world caster, the spell, and the fact that the cast or declared attempt
occurred.
Attribute every cast to its in-world caster. Map first-person player speech to
the associated player character, and attribute a spell narrated by the GM to
the in-world creature that casts it. If the caster cannot be resolved, use only
the most specific in-world identity supported by the transcript; do not invent
a name.

View File

@@ -1,6 +0,0 @@
The spell catalog for this extraction is provided below as JSON. Each entry
lists a `canonical_name` and its recognized `aliases`. If the transcript uses
an alias, select that entry's `canonical_name`. Return spell names using the
canonical spelling exactly; never return an alias as a spell name.
{{ input "spell_catalog" }}

View File

@@ -1,3 +0,0 @@
Candidate material, including the exact valid candidate-number range:
{{ input "candidates" }}

View File

@@ -1,5 +0,0 @@
Identify only high-confidence duplicate entities among the supplied candidates.
Preserve distinct entities even when their names are similar. Treat contextual descriptions and transcript evidence as supporting material, not as permission to merge ambiguous records.
When several records are duplicates, choose as canonical the candidate with the clearest stable identity. Prefer a complete proper name over an abbreviation, and prefer an unadorned proper name over one with incidental descriptors unless the evidence establishes those descriptors as part of the name. A longer name is not inherently more canonical.

View File

@@ -1,27 +0,0 @@
id: generic.semantic_reconciliation
version: "v1"
inputs:
- name: candidates
required: true
content_type: application/json
- name: transcript
required: true
content_type: application/json
messages:
- role: system
content_file: ./system.md
- role: user
content_file: ./protocol.md
- role: user
content_file: ./instructions.md
cache_control:
type: ephemeral
- role: user
content_file: ./candidates.md
- role: user
content_file: ./transcript-windows.md
output:
format: json
validation_mode: json_schema
schema_path: semantic_reconciliation_llm.v1.json
repair_attempts: 1

View File

@@ -1,9 +0,0 @@
Use only the positive integer `candidate_number` values supplied in the candidate material. Valid candidate numbers are exactly the inclusive `first` through `last` values declared in `candidate_number_range`; use the explicit number attached to each candidate.
Transcript unit `id` values and evidence `start_unit_id` and `end_unit_id` values identify source positions. They are not candidate numbers and must never be used in `candidate_numbers` or `canonical_candidate_number`.
Return a duplicate group only when the evidence supports that every selected candidate describes the same underlying entity. Each group must contain at least two distinct candidate numbers, and its `canonical_candidate_number` must be one of those numbers. A candidate may appear in at most one group.
Omit uncertain matches and candidates that should remain distinct. Do not invent candidates or infer a number that is not explicitly supplied. An empty `duplicate_groups` array is valid.
The response must conform exactly to the selected JSON schema. Return candidate numbers only: do not copy candidate names, evidence, transcript text, source identifiers, or source ranges into the response.

View File

@@ -1,2 +0,0 @@
You reconcile structured records that may describe the same underlying entity.
Follow the supplied protocol and return only the requested structured result.

View File

@@ -1,3 +0,0 @@
Transcript evidence windows:
{{ input "transcript" }}

View File

@@ -1,32 +0,0 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "notarius.generic.semantic_reconciliation.llm",
"title": "notarius_semantic_reconciliation_llm_v1",
"type": "object",
"additionalProperties": false,
"required": ["duplicate_groups"],
"properties": {
"duplicate_groups": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": false,
"required": ["candidate_numbers", "canonical_candidate_number"],
"properties": {
"candidate_numbers": {
"type": "array",
"minItems": 2,
"items": {
"type": "integer",
"minimum": 1
}
},
"canonical_candidate_number": {
"type": "integer",
"minimum": 1
}
}
}
}
}
}

View File

@@ -1,15 +0,0 @@
// Package assets exposes embedded LLM-facing content.
package assets
import (
"embed"
"io/fs"
)
//go:embed dnd generic
var embedded embed.FS
// FS returns the embedded read-only asset filesystem.
func FS() fs.FS {
return embedded
}

View File

@@ -1,6 +1,6 @@
# ADR-0004: Package modules by domain, not by stage # ADR-0004: Package modules by domain, not by stage
**Status:** Accepted — its asset-co-location rule is superseded by [ADR-0011](0011-centralize-llm-assets.md); its domain-first module packaging decision remains accepted. **Status:** Accepted
**Date:** 2026-07-13 **Date:** 2026-07-13
## Context ## Context

View File

@@ -1,49 +0,0 @@
# ADR-0010: Use workload-oriented LLM profile defaults
**Status:** Accepted
**Date:** 2026-08-03
## Context
LLM-backed D&D operations share an execution-policy choice, but repeating a
provider or model-named profile on every module binding ties pipeline structure
to a deployment decision. Different environments may require different model,
backend, timeout, or reasoning settings while retaining the same workload.
Notarius also needs a usable default for maintained D&D prompts without making
an operator profile mandatory. That default must remain owned by the D&D
family, while generic LLM infrastructure stays unaware of domain-specific
policy.
## Decision
Pipelines may name one workload-oriented default profile, inherited only by
selected LLM-backed bindings and validators. Binding-level profile IDs remain
intentional exceptions, and the run-wide CLI profile override has highest
precedence.
The D&D family owns an embedded fallback profile named `dnd-extraction`.
Operators may provide a complete profile with the same ID through a PromptKit
filesystem source. PromptKit selects the higher-precedence matching definition;
Notarius does not merge profile documents. Production, development, and local
deployments can therefore use different execution policy behind one unchanged
pipeline ID.
## Alternatives considered
- Repeat a model-named profile on every binding. This makes routine deployment
policy changes noisy and obscures the shared workload intent.
- Require every deployment to install a profile file. This adds configuration
friction and leaves maintained D&D prompts without an application-owned
fallback.
- Put D&D profile policy in generic LLM infrastructure. This breaks domain
ownership and makes generic code depend on one workload.
## Consequences
Pipeline configuration expresses workload intent rather than a specific
provider or model. Operators can replace the complete execution policy without
editing bindings, while binding-level and run-wide exceptions remain available.
Profile changes affect resolved pipeline and checkpoint identity, so they may
intentionally cause work to be recomputed. The D&D fallback becomes a
maintained application execution-policy asset.

View File

@@ -1,69 +0,0 @@
# ADR-0011: Centralize LLM-facing assets in a content-only package
**Status:** Accepted
**Date:** 2026-08-05
## Context
LLM prompts, private response schemas, generic schemas, and fallback profiles
are authored and reviewed as content, but package-local embedding scattered that
content across implementation trees. Finding all of the assets that contribute
to a prompt family required navigating code ownership boundaries rather than a
single discoverable content boundary.
The repository must retain module ownership of prompt semantics, schema
identities, registration, and prompt-cache behavior. Durable artifact schemas
and non-LLM domain data have different compatibility and ownership rules, so
they must not move merely because they are embedded files.
## Decision
LLM-facing content is embedded by the root `assets` package. It is a data-only
dependency leaf: its single `FS() fs.FS` API returns the read-only embedded
filesystem, and the package contains no business logic or internal or PromptKit
dependencies. The accepted import path is
`gitea.maximumdirect.net/eric/notarius/assets`; it makes repository-owned
content available to its consumers, not a public extension contract.
Consumers scope that filesystem to the subtree they own before reading or
registering content. Modules continue to own their manifests, prompt ordering,
private response-schema identity, and registration. Centralizing physical files
does not centralize domain semantics or transfer those responsibilities to the
root package.
The root package contains prompt content, private LLM response schemas, generic
LLM schemas, shared fragments, and fallback profiles. Durable artifact schemas
and non-LLM domain data remain with their current owners. A module fingerprint
is derived from its manifest-selected module and shared files, rather than from
an entire asset tree. The relocation is accepted to cause a one-time checkpoint
invalidation.
This decision supersedes only the physical asset-co-location portion of
ADR-0004's decision that places domain-specific prompt fragments and schemas
within the domain tree. ADR-0004's domain-first packaging and registrar
ownership decisions remain accepted.
## Alternatives Considered
- Keep package-local assets. This preserves physical co-location with code but
makes prompt-author discovery and cross-family review unnecessarily costly.
- Use `internal/llmassets`. This would hide content from legitimate owners
outside the `internal` subtree and would make the root asset boundary depend
on implementation-layer placement.
- Build a behavioral central registry. This would mix content discovery with
prompt selection and registration behavior, moving module semantics into a
shared registry.
- Use runtime filesystem overlays. This would add runtime configuration and
failure modes where compile-time embedded content is sufficient.
## Consequences
Prompt authors can find in-scope LLM content in one top-level tree while module
packages continue to define its meaning and registration. Consumers have an
explicit, narrow dependency on only the content they need. The root package is
intentionally importable but must remain a stable, content-only leaf rather
than becoming a general extension API.
The initial relocation invalidates existing checkpoints once. Later checkpoint
identity changes remain limited to the manifest-selected prompt and shared
content, so unrelated files do not trigger recomputation.

View File

@@ -1,64 +0,0 @@
# ADR-0012: Resolve opaque entity identifiers deterministically
**Status:** Accepted
**Date:** 2026-08-08
## Context
Entity IDs in durable Notarius artifacts are application-owned, deterministic
identifiers. They are useful to artifact consumers, but their hash-based form
does not help a model distinguish entities and would make the model reproduce
an opaque implementation detail. A plain name is likewise insufficient where
multiple supplied records share that name.
The LLM boundary must preserve the typed artifact and durable-schema ownership
of [ADR-0003](0003-typed-interfaces-with-two-zone-data-model.md) and the distinction
between disambiguating references and source evidence in
[ADR-0009](0009-minimal-evidence-grounded-extraction-artifacts.md).
## Decision
Callers present a model with semantic selections: a canonical name when it is
unique in the request, or a contextual descriptor containing the name and
source coordinates when that context is needed to distinguish supplied
records. The model returns only those supplied selections. The caller resolves
each accepted selection against the request-local supplied records and attaches
the opaque application ID deterministically.
Source coordinates are permitted in a selection solely as identity context.
They neither establish an occurrence fact nor replace that occurrence's
current-transcript evidence. A selector must resolve exactly; unknown,
ambiguous, partial, reordered, or otherwise unsafe selections are not mapped.
Where an operation requires a complete grounded artifact, that failure rejects
the complete artifact rather than accepting a partially mapped result.
An explicitly scoped request-local short label is permitted only when a
contextual descriptor would be impractical and the caller can deterministically
map the label within that one request. Such a label is not a durable ID, must
not escape the request boundary, and requires a concrete justification in its
own module contract.
## Alternatives considered
- Ask the model to return durable IDs. This exposes opaque implementation
state, does not improve semantic disambiguation, and makes model output
depend on hash formatting.
- Select by name alone. This cannot safely distinguish same-name records.
- Make request-local labels durable identifiers. This would turn prompt
presentation into a public identity contract and create avoidable migration
pressure.
- Let the model invent identifiers or resolve ambiguity. This makes identity
assignment non-deterministic and weakens validation.
## Consequences
Durable integration contracts retain their exact ID/name pairs while models
operate on readable contextual selections. Calling modules must own selector
construction, exact resolution, ambiguity handling, and conversion into their
durable artifact type; PromptKit and its adapter remain transport-only.
Some ambiguous or invalid proposals are deliberately omitted, retried, or
rejected according to the caller's existing failure policy. Internal candidate
keys may support deterministic request-local mapping, but they are not
model-visible selectors or durable data. This adds local validation work while
keeping identity assignment auditable and stable.

View File

@@ -1,89 +0,0 @@
# ADR-0013: Use request-local candidate handles for semantic reconciliation
**Status:** Accepted
**Date:** 2026-08-09
## Context
Several typed normalize stage modules need semantic reconciliation after
deterministic preprocessing: a model can judge whether source-backed candidates
refer to the same underlying entity, while application code remains responsible
for constructing the normalized artifact. Requiring the model to reproduce a
candidate's full contextual selector makes the response larger and introduces
avoidable formatting, ordering, and transcription failure modes.
Reconciliation must preserve the exact typed artifact boundary established by
[ADR-0003](0003-typed-interfaces-with-two-zone-data-model.md), the domain-neutral
framework and concrete-domain dependency direction established by
[ADR-0004](0004-package-modules-by-domain.md), and the distinction in
[ADR-0009](0009-minimal-evidence-grounded-extraction-artifacts.md) between source
evidence and auxiliary identity context. It also needs a concrete, narrowly
scoped application of the request-local-label exception allowed by
[ADR-0012](0012-resolve-opaque-entity-identifiers-deterministically.md).
## Decision
Semantic reconciliation will be a domain-neutral framework mechanism used by
typed normalize stage modules. A consuming artifact family will retain
ownership of its typed records, identity rules, consolidation policy, durable
IDs, and domain warnings; the framework mechanism will not infer those rules
from arbitrary data.
For each reconciliation request, deterministic code will assign every eligible
model-visible candidate a contiguous, one-based integer handle. The model may
receive the candidate's contextual label, source references, and bounded source
context needed to judge identity, but its structured response will identify
candidates only by those supplied handles. A handle is local to one request,
does not represent entity identity, and must never enter a durable artifact or
be used to derive a durable ID.
The model will propose duplicate groups and select one supplied member of each
group as canonical. Deterministic code will resolve the handles through the
retained request mapping, validate the complete proposal, discard unsafe
groups, and apply only validated groups through typed domain-owned policy. The
model will not synthesize replacement records or directly mutate an artifact.
Every reconciliation prompt will combine a mandatory framework-owned protocol
and safety policy with an explicitly selected semantic policy. The semantic
policy may be the conservative generic policy or a domain-owned policy, but it
cannot replace the shared response protocol or deterministic safety boundary.
## Alternatives considered
- Return durable application IDs. Opaque IDs do not help semantic judgment,
expose application identity mechanics, and make model output reproduce data
that deterministic code already owns.
- Return names alone or copied contextual selectors. Names can be ambiguous,
while reproducing labels and source ranges adds response complexity and
creates mismatches without adding semantic information. Request-local
handles preserve exact selection without either failure mode.
- Ask the model to return synthesized canonical replacement records. This
would transfer typed artifact construction, provenance consolidation, and
durable identity policy to a probabilistic boundary.
- Reconcile reflection-discovered fields or arbitrary JSON. This would weaken
the typed artifact contract and move domain semantics into generic code.
- Hide reconciliation inside extraction or another stage. This would obscure
stage ownership and create cross-stage behavior outside the fixed pipeline;
reconciliation remains explicit normalize-stage behavior.
- Let each domain replace the complete prompt protocol. This would duplicate
safety mechanics and allow domain policy to bypass the common response and
validation contract.
## Consequences
Model responses become smaller and easier to validate, while deterministic
application code retains authority over identity, provenance, ordering, and
typed artifact construction. The framework requires a request-local mapping,
bounded context preparation, a private integer response contract, proposal
assessment, and shared prompt assets. Each consuming artifact family still
requires a typed adapter for its irreducibly domain-specific rules.
Request-local handles are deliberately unsuitable for persistence, logging as
entity identity, checkpoint contracts, or cross-request correlation. Changes
to shared protocol and policy assets must participate in the normal prompt,
schema, and checkpoint fingerprint mechanisms.
The shared mechanism and its initial D&D registry consumers are now
implemented. Current behavior is documented in
[Module Internals](../internal/modules.md#semantic-reconciliation) and
[D&D Module Internals](../internal/dnd.md#semantic-registry-reconciliation).

View File

@@ -1,59 +0,0 @@
# ADR-0014: Use feedback-aware validation retries
**Status:** Accepted
**Date:** 2026-08-26
## Context
Validation can identify a candidate defect after a producer has returned an
otherwise well-formed result. Retrying without the validator's deterministic,
bounded feedback wastes the useful diagnosis, while treating validator
execution failures as defects would ask a producer to repair conditions it
cannot control. The mechanism must preserve typed producer ownership,
checkpoint safety, and the repository's sensitive-data boundaries.
## Decision
The implementation will keep three independent budgets: the producer binding's
outer `retries` budget, PromptKit's structured-output repair budget, and each
validator's execution-retry budget. Validators will run sequentially in their
configured order and aggregate both rejections and execution failures before a
candidate disposition is selected.
A correction-capable producer will provide the exact single LLM response that
controlled its candidate using the `single_response_v1` protocol. A correction
attempt will reconstruct the ordinary request and append exactly two fresh
messages: that latest response as `assistant`, followed by one deterministic
aggregate correction request as `user`. Earlier turns will not accumulate.
Validator failures will not recurse into correction. Pipeline policy owns
terminal disposition, with field-by-field producer overrides over pipeline
defaults: structural failure and semantic rejection default to `fail_run`, and
validator execution failure defaults to `warn_continue`. Validators can report
facts and bounded corrective guidance, but never decide disposition.
Rejected and structurally invalid candidates will not advance. A candidate
allowed through after a validator execution failure will retain explicit
incomplete-validation provenance and will not be checkpointed. Exact response
and correction text remain attempt-local: they are excluded from ordinary
errors, warnings, manifests, receipts, caches, checkpoints, and default debug
summaries.
## Alternatives considered
- Retry every producer after any validation outcome. This conflates producer
defects with validator operational failures and wastes retry budget.
- Let validators decide whether to continue. This would distribute pipeline
disposition policy across validators and undermine consistent defaults.
- Reuse the full prior conversation. Accumulated turns introduce unbounded
prompt growth and make correction behavior depend on incidental history.
- Persist raw responses to simplify diagnosis. Raw model output and correction
guidance may be sensitive and do not belong in durable pipeline records.
## Consequences
The framework gains transport-neutral correction and candidate contracts,
producer capability checks, policy resolution, aggregated validation outcomes,
and conservative checkpoint handling. Prompt construction remains inside the
LLM adapter, while modules remain responsible for accurately exposing the
single response that directly controlled a candidate.

View File

@@ -1,67 +0,0 @@
# ADR-0015: Separate process warnings from quality diagnostics
**Status:** Accepted
**Date:** 2026-08-27
## Context
Notarius currently represents process degradation, incomplete validation,
extraction-quality doubt, and routine normalization with one flat warning
record. That makes ordinary successful runs noisy, loses the framework context
needed to explain a finding, and gives `warning_count` no stable operational
meaning. It also permits output encoders to add a warning after the durable
warning file has already been written.
The application needs one bounded diagnostic model that preserves exact
occurrence counts while retaining only safe, representative samples. Fresh and
resumed logical runs must present the same groups. The model must not alter
validation decisions, retry budgets, rejected-output behavior, or process exit
policy.
## Decision
Warnings are reserved for a completed run that advanced under an allowed
process-level degradation or incomplete-work policy. Extraction-quality signals
are advisories, and routine accepted transformations are observations. A
non-degraded successful run therefore has zero actionable warnings.
Modules and validators own a diagnostic's disposition, category, reason code,
scope, and safe message. The framework adds pipeline origin, including stage,
step, lane, module, validator, and chunk context where applicable. It then
aggregates deterministically by disposition, category, reason code, and full
origin. Chunk context remains on representative samples so equivalent findings
across chunks aggregate together.
Diagnostics carry exact occurrence counts, at most three distinct samples, and
numeric omitted-sample metadata. Producers and validators are bounded to 64
local groups. Final actionable warning groups are bounded without truncation;
the non-warning collection may truncate represented groups while preserving an
exact total occurrence count and explicit truncation metadata.
The public contracts will be versioned: grouped actionable warnings use
`notarius.warnings.v2`, grouped advisories and observations use
`notarius.diagnostics.v1`, and the run receipt uses
`notarius.run-result.v2`. Successful output encoders return logical files or
an error; they do not add post-encoding warnings.
## Alternatives considered
- Keep one warning list and filter only CLI output. This would leave durable
consumers with the same semantically mixed, unbounded contract.
- Map reason codes to severity in a central framework registry. This would
split module-owned meaning between synchronized policy tables and make new
diagnostic meaning implicit.
- Preserve local omission warning records. They inflate visible group counts
and lose exact occurrence semantics.
- Keep output-encoder warnings. A one-pass encoder cannot include those
records consistently in files it has already serialized; a two-phase encoder
protocol is deferred until a demonstrated need exists.
## Consequences
The framework gains validated diagnostic primitives, local collection,
origin-aware aggregation, and versioned durable presentation. Existing warning
transport remains temporarily while producers migrate. Current architecture,
operator, integration, and internal documentation will describe the behavior
only as each implementation step lands; this accepted decision does not claim
that the migration is complete.

View File

@@ -1,76 +0,0 @@
# ADR-0016: Use feedback-aware module-requested retries
**Status:** Accepted
**Date:** 2026-08-28
## Context
An LLM-backed module can discover that a structurally valid model response is
unsafe while translating it into a typed candidate, before the ordinary
validator chain runs. Semantic registry reconciliation is the first such case:
the shared core can identify invalid duplicate-group proposals, and typed
application can reject a domain-incompatible group while preserving a safe
partial result. Repeating the original request without the rejected response or
corrective explanation gives the model no information with which to improve.
The existing feedback-aware validation mechanism already establishes the safe
correction protocol, but module-owned retry directives also carry internal
reason codes, operator messages, and fallback diagnostics. Those values are not
model instructions. Some module retry conditions, including exhausted
structured-output repair, also have no valid exact response to append.
## Decision
An LLM-backed normalizer may attach optional, bounded model-facing correction
guidance to a module-requested retry. Guidance is a separate contract field; the
framework never derives it from a reason code, operator message, diagnostic, or
error. A feedback-capable directive must include the exact model candidate that
controlled the safe fallback and must use `single_response_v1`.
The artifact-neutral producer-attempt state machine constructs the correction
from that exact latest response and the supplied guidance. The producer rebuilds
its complete ordinary request and appends the response as one assistant message
and the guidance as one user message. Earlier attempts do not accumulate, and
the attempt consumes the same configured stage retry budget as operational,
structural, validation, and feedback-free module retries.
A module retry without guidance remains valid and starts a fresh attempt. This
is the required behavior when no valid model candidate exists, including after
structured-output repair is exhausted. If feedback is supplied without a valid
supported candidate, the framework fails the module contract rather than
retrying blindly or inventing guidance.
After retry exhaustion, the normalizer's safe fallback continues through its
validator chain. Safe semantic groups may remain applied, unsafe groups remain
separate, and bounded fallback diagnostics may explain the process degradation.
Exact responses and correction text remain attempt-local and are excluded from
ordinary errors, warnings, manifests, receipts, caches, and checkpoints.
This decision extends, rather than supersedes,
[ADR-0014](0014-feedback-aware-validation-retries.md): both correction sources
use the same transport-neutral payload, replacement-request semantics, outer
retry budget, and sensitive-data boundary.
## Alternatives considered
- Continue blind module retries. This preserves a smaller contract but wastes
the module's deterministic diagnosis and commonly repeats the same defect.
- Convert module safety checks into validators. Typed reconciliation must apply
only safe proposal groups and retain a fallback before validation; moving
artifact-owned translation and application policy into validators would blur
stage ownership.
- Copy the retry reason or operator message into the model request. Those values
are written for provenance and humans, can contain opaque internal labels,
and do not reliably describe a correct replacement.
- Require feedback for every module retry. Structural failures may have no
valid exact candidate, so this would either prevent useful fresh retries or
fabricate prior-response material.
## Consequences
The normalize retry contract and generic producer-attempt directive gain an
optional correction-guidance field and candidate-pairing validation. Modules
that use it must provide semantically meaningful bounded prose and an exact
candidate. Registry reconciliation maintains separate operator and model
renderers, and policy fingerprints change so checkpoints created under blind
retry behavior are not reused.

View File

@@ -1,59 +0,0 @@
# ADR-0017: Defer cache access policy to the deployment
**Status:** Accepted
**Date:** 2026-09-11
## Context
Canonical chunk plans and invocation checkpoints are reconstructible cache
state. They may include source-derived content, so access and retention matter,
but Notarius cannot determine whether a deployment needs a private cache or a
cache shared by a trusted group. The current application-enforced `0700` and
`0600` modes prevent ordinary Unix provisioning from making that choice.
[ADR-0005](0005-cache-canonical-chunk-plans-by-source.md) governs canonical
chunk-plan reuse, while [ADR-0006](0006-separate-output-cache-and-debug-state.md)
established the output, cache, and debug separation refined by
[ADR-0007](0007-separate-checkpoint-recording-from-reuse.md). None of those
decisions requires Notarius to own exact cache access modes.
## Decision
Deployments own cache access policy through provisioned directories, process
umask, setgid inheritance, and default ACLs. Notarius requests conventional
maximum creation modes for cache entries and leaves the kernel to select their
effective permissions.
Notarius continues to own confined paths, symlink rejection, validation,
atomic publication, cleanup, and recovery. Those correctness boundaries are
independent from deciding which accounts may access cache state.
Source-derived cache contents can be sensitive. Operators own their access and
retention policy, while API credentials remain outside cache payloads under the
existing secret boundary. Existing cache inode modes are not migrated
automatically.
This decision does not supersede ADR-0005, ADR-0006, or ADR-0007.
## Alternatives considered
- Continue enforcing private `0700` directory and `0600` file modes. This
makes one deployment choice for every operator and prevents trusted-group
provisioning.
- Add configurable Notarius permission modes or a private/group cache setting.
This preserves application ownership of a deployment concern and expands the
public configuration surface without a durable policy reason.
- Recursively converge existing cache ownership, modes, or ACLs. This would
make upgrades mutate operator-managed state and cannot safely infer an
intended policy.
## Consequences
Operators can use a restrictive umask for private cache state or provision a
setgid/default-ACL cache root for trusted-group access. They must deliberately
recreate or administratively migrate pre-existing cache trees when their
existing modes do not match the desired deployment policy.
Notarius does not add user or group management, ACL mutation, permission
configuration, or recursive cache migration. Output and debug permission
contracts remain separate from this decision.

View File

@@ -1,53 +0,0 @@
# ADR-0018: Adopt ConfigKit for configuration loading
**Status:** Accepted
**Date:** 2026-09-13
## Context
Notarius's former configuration boundary owned strict YAML parsing, input
presence bookkeeping, defaults, normalization, environment overlays, and
configuration inspection. Those general-purpose responsibilities had grown
alongside application-specific pipeline and module semantics.
ConfigKit v0.3.0 provides strict typed YAML loading, imports, configuration
profiles, explicit CLI overrides, recorded defaults and normalization, safe
inspection rendering, and released source-origin APIs. The application needs
those source records to resolve filesystem paths according to the file that
supplied a surviving declaration.
## Decision
Notarius will adopt the root public `gitea.maximumdirect.net/eric/ConfigKit`
package at v0.3.0 for its version 5 configuration contract. ConfigKit owns
source composition, strict decoding, input presence, metadata, and safe
effective/explanation rendering. Notarius retains its typed document schema,
field semantics, runtime conversion, catalog-dependent pipeline resolution,
PromptKit profile handling, and physical run preparation.
Version 5 is a deliberate breaking file-contract change. It uses canonical
object module bindings and reference sources, and enables ConfigKit imports
and configuration profiles. The selected configuration file's canonical
directory is the base for its declared application paths; ConfigKit's released
`Resolution.InputInfo` and `Resolution.Source` APIs provide their origins.
`NOTARIUS_CONFIG` remains file discovery, while field-level operational
environment overlays are retired. Explicit configuration-valued CLI flags are
represented as ConfigKit overrides; invocation actions remain outside that
boundary.
## Alternatives considered
- Continue extending the custom YAML loader. This preserves duplicated
general-purpose parsing, presence, composition, and inspection machinery.
- Retain version 4 alongside version 5. This would leave a supported dual
configuration boundary and delay removal of the machinery being replaced.
- Wrap ConfigKit in a compatibility pre-parser. This weakens ConfigKit's
strict contract and recreates the application-owned parser complexity.
## Consequences
The released application has one ConfigKit-backed loader and one version 5
contract. Operator documentation and maintained examples move with the cutover.
ConfigKit metadata is retained only at the configuration publication boundary;
its types do not become module or runtime pipeline interfaces.

View File

@@ -10,26 +10,14 @@ defined in [Operations](operations.md).
~~~ ~~~
notarius help notarius help
notarius --version
notarius run <pipeline-id> --input path/to/source.json [--json] [flags] notarius run <pipeline-id> --input path/to/source.json [--json] [flags]
notarius config validate [--config path/to/config.yml] [--config-overlay name] [--pipeline pipeline-id] [--only lane-a,lane-b] notarius config validate [--config path/to/config.yml] [--pipeline pipeline-id] [--only lane-a,lane-b]
notarius pipelines list [--config path/to/config.yml] [--config-overlay name] [--json] notarius pipelines list [--config path/to/config.yml] [--json]
~~~ ~~~
Running Notarius without arguments, or with **help**, **--help**, or **-h**, Running Notarius without arguments, or with **help**, **--help**, or **-h**,
writes the command summary to standard output and exits with status 0. writes the command summary to standard output and exits with status 0.
`notarius --version` is valid only as the sole root argument. It writes exactly
`notarius <version>` followed by a newline to standard output and exits with
status 0. A tagged `go install` build can report its main-module stable tag,
and controlled builds can inject a stable tag at link time through
`gitea.maximumdirect.net/eric/notarius/internal/buildinfo.Override`; an ordinary
unversioned checkout reports `development`. Invalid injected version content is
a runtime error with exit status 1, while extra `--version` arguments are a
syntax error with exit status 2. This diagnostic does not replace the
[run-result](integrations/run-result.md) or artifact contracts for downstream
compatibility decisions.
## run ## run
~~~ ~~~
@@ -42,7 +30,6 @@ pipeline ID and **--input** are required.
| Flag | Meaning | | Flag | Meaning |
| --- | --- | | --- | --- |
| **--config path** | Use this configuration file. When omitted, configuration discovery applies; see [Configuration](config.md). | | **--config path** | Use this configuration file. When omitted, configuration discovery applies; see [Configuration](config.md). |
| **--config-overlay name** | Select a ConfigKit configuration overlay. Empty and **default** select the composed base configuration. |
| **--input path** | Source input file to process. Required. | | **--input path** | Source input file to process. Required. |
| **--output-dir path** | Override the configured output root for this run. | | **--output-dir path** | Override the configured output root for this run. |
| **--json** | Write the successful run-result receipt as JSON to standard output. | | **--json** | Write the successful run-result receipt as JSON to standard output. |
@@ -52,92 +39,46 @@ pipeline ID and **--input** are required.
| **--debug** | Retain a debug bundle for this run. | | **--debug** | Retain a debug bundle for this run. |
| **--debug-dir path** | Override the debug-bundle root. Requires **--debug**. | | **--debug-dir path** | Override the debug-bundle root. Requires **--debug**. |
| **--only lane-a,lane-b** | Run only the selected comma-separated artifact lanes when that selection is valid for the configured pipeline. | | **--only lane-a,lane-b** | Run only the selected comma-separated artifact lanes when that selection is valid for the configured pipeline. |
| **--llm-profile id** | Highest-precedence configured profile for selected LLM-backed bindings and validators; it replaces binding and [pipeline](config.md#pipelines) defaults. | | **--llm-profile id** | Override effective LLM-capable module bindings with one configured profile. |
| **--session-id id** | Override the generated prompt session identifier with a non-empty value for LLM-backed module calls. | | **--session-id id** | Supply a non-empty prompt session identifier to LLM-backed module calls. |
| **--reasoning-effort value** | Replace the selected PromptKit profile's reasoning effort for every LLM-backed call in this run. The value must be non-empty and the flag may be specified only once. | | **--reference selector=path** | Add or replace a file reference binding. Repeatable. |
| **--clear-reasoning-effort** | Clear reasoning effort inherited from the selected PromptKit profile for every LLM-backed call in this run. | | **--without-reference selector** | Remove a configured optional reference binding. Repeatable. |
| **--reference selector=path** | Add or replace external file reference bindings at pipeline, lane, chunk, or binding scope. Repeatable. |
| **--without-reference selector** | Remove matching configured external reference bindings. Repeatable. |
**--chunk_cache** accepts only **auto**, **bypass**, or **refresh**. **--chunk_cache** accepts only **auto**, **bypass**, or **refresh**.
**--debug-dir**, **--output-dir**, **--session-id**, and **--debug-dir**, **--output-dir**, **--session-id**, and
**--reasoning-effort**, and **--recompute-step** reject explicit empty values. **--recompute-step** reject explicit empty values. **--recompute-step**
**--reasoning-effort** and **--clear-reasoning-effort** are mutually exclusive. requires **--resume**; checkpoint requirements and reuse behavior are
When neither is present, reasoning effort comes from the selected PromptKit documented in [Operations](operations.md).
profile. These controls apply to the shared run client, including retries and
LLM-backed validators, and do not modify configuration or profile files.
Persistent reasoning settings remain a PromptKit profile concern.
**--recompute-step** requires **--resume**; checkpoint requirements and reuse
behavior are documented in [Operations](operations.md).
Every run uses one effective prompt session. Without **--session-id**, Notarius
generates a stable `notarius:v1:` identifier from the trimmed resolved input
module key and the input file's exact raw bytes. The same module and bytes
therefore produce the same identifier, regardless of pipeline, references,
profile, retries, or run settings. An explicit non-empty value replaces that
default. Session identifiers are visible to providers; they are non-secret
correlation identifiers, not credential storage. See
[Operations](operations.md#operational-limits) for privacy and workflow
guidance.
### Reference selectors ### Reference selectors
Use **--reference** only for reference slots declared by the selected Use **--reference** only for a reference slot declared by the selected
configured targets. Qualification narrows the scope of an override: configured target. The accepted selector forms are:
| Form | Target | | Form | Target |
| --- | --- | | --- | --- |
| slot=path | Every selected target that declares slot. | | slot=path | The unique selected target that declares slot. |
| chunk.slot=path | The chunker. | | chunk.slot=path | The chunker. |
| lane.slot=path | Every extractor, merger, or normalizer in lane that declares slot. | | merge.slot=path | The unique selected merger that declares slot. |
| lane.slot=path | The unique extractor, merger, or normalizer in lane that declares slot. |
| lane.extract.slot=path | The extractor in lane. | | lane.extract.slot=path | The extractor in lane. |
| lane.merge.slot=path | The merger in lane. | | lane.merge.slot=path | The merger in lane. |
| lane.normalize.slot=path | The normalizer in lane. | | lane.normalize.slot=path | The normalizer in lane. |
Pipeline- and lane-scoped selectors are expected to match multiple targets and **--without-reference** uses the same selector forms without =path. Slot
fail if they match none. A stage-specific selector fails when its lane is not names, requiredness, and configured bindings are part of the
selected or its target does not declare the slot. There is no stage-wide
`merge.slot` shorthand; name the lane when targeting a merger.
CLI bindings override configured external paths. For overlapping CLI
selectors, a binding-specific or chunk selector overrides a lane selector, and
a lane selector overrides a pipeline selector. The last occurrence wins at
equal scope. Binding and unbinding the same concrete target at equal scope is
an error; a narrower bind or unbind may create an intentional exception to a
broader action.
**--without-reference** uses the same selector forms without `=path` and
removes external bindings only. Neither flag replaces or removes a generated
artifact handoff; an external/generated collision is a resolution error.
Required slots are checked after all effective changes. CLI reference paths
are resolved relative to the process working directory, so subprocess and
service callers should use absolute paths. Slot names, accepted media types,
size limits, requiredness, and configured generated bindings are part of the
[configuration contract](config.md). [configuration contract](config.md).
For example, one shared campaign reference can reach every compatible target,
with an optional lane-specific exception:
~~~
notarius run dnd-session \
--input /data/transcript.json \
--reference party=/data/references/party.txt \
--reference npc-registry.party=/data/references/npc-party-context.txt
~~~
### Run output ### Run output
Without **--json**, standard output contains the completed pipeline ID, counts Without **--json**, standard output contains the completed pipeline ID, counts
of normalized and rejected outputs, and the output directory. A debug-enabled of normalized and rejected outputs, and the output directory. A debug-enabled
run also prints its debug-bundle path to standard output. A successful run with run also prints its debug-bundle path to standard output. A successful run with
actionable process warnings reports their group and occurrence counts to warnings reports the warning count to standard error. The published JSON bundle
standard error. When the selected output module publishes `warnings.json`, the
summary also reports that durable file's path. Advisory and observation findings
do not produce a warning line. The published JSON bundle
is defined by the [JSON output contract](integrations/json-output.md). is defined by the [JSON output contract](integrations/json-output.md).
With **--json**, successful standard output is exactly one With **--json**, successful standard output is exactly one
`notarius.run-result.v2` JSON document followed by a newline, with no `notarius.run-result.v1` JSON document followed by a newline, with no
human-oriented status or debug-path line. Its fields and compatibility policy human-oriented status or debug-path line. Its fields and compatibility policy
are defined by the [run-result contract](integrations/run-result.md). A caller are defined by the [run-result contract](integrations/run-result.md). A caller
must check for exit status 0 before decoding this output; a failed write can must check for exit status 0 before decoding this output; a failed write can
@@ -155,10 +96,10 @@ OPENROUTER_API_KEY=your-api-key \
## config validate ## config validate
~~~ ~~~
notarius config validate [--config path/to/config.yml] [--config-overlay name] [--pipeline pipeline-id] [--only lane-a,lane-b] notarius config validate [--config path/to/config.yml] [--pipeline pipeline-id] [--only lane-a,lane-b]
~~~ ~~~
This command loads and validates the selected composed configuration. With **--pipeline**, it also This command loads and validates a configuration. With **--pipeline**, it also
resolves that pipeline against the production module catalog. **--only** selects resolves that pipeline against the production module catalog. **--only** selects
lanes during that resolution and requires **--pipeline**. lanes during that resolution and requires **--pipeline**.
@@ -181,32 +122,10 @@ OPENROUTER_API_KEY=validation-placeholder \
The placeholder in the second command is sufficient only for offline The placeholder in the second command is sufficient only for offline
validation; it cannot run a provider-backed pipeline. validation; it cannot run a provider-backed pipeline.
## config effective
~~~
notarius config effective [--config path/to/config.yml] [--config-overlay name]
~~~
This command writes the selected composed configuration as deterministic,
redacted YAML. Standard output contains only that YAML: it does not validate a
pipeline, prepare modules, materialize references, or contact a provider.
## config explain
~~~
notarius config explain [--config path/to/config.yml] [--config-overlay name] <json-pointer>
~~~
This command writes ConfigKit's redacted, human-readable explanation for one
canonical JSON Pointer. Supply `""` explicitly to explain the root. A malformed
pointer is a syntax error; a well-formed unknown path is a command failure.
The explanation can describe a value displaced by a selected overlay. Standard
output contains only the explanation.
## pipelines list ## pipelines list
~~~ ~~~
notarius pipelines list [--config path/to/config.yml] [--config-overlay name] [--json] notarius pipelines list [--config path/to/config.yml] [--json]
~~~ ~~~
This command lists configured pipeline IDs in sorted order. By default, it This command lists configured pipeline IDs in sorted order. By default, it
@@ -223,11 +142,7 @@ go run ./cmd/notarius pipelines list \
Successful commands write their primary result to standard output. Warnings and Successful commands write their primary result to standard output. Warnings and
errors are written to standard error. errors are written to standard error.
Configuration load errors include ConfigKit's field diagnostics and source For **run --json**, warnings remain on standard error and standard output is a
coordinates, followed by a source-ID legend identifying the acquired files or
source labels. This includes imported files and the selected configuration overlay.
For **run --json**, actionable process warnings remain on standard error and standard output is a
machine-readable success result only. Syntax and runtime diagnostics remain on machine-readable success result only. Syntax and runtime diagnostics remain on
standard error. Parse the result only after the process exits with status 0. standard error. Parse the result only after the process exits with status 0.
@@ -238,6 +153,6 @@ standard error. Parse the result only after the process exits with status 0.
| 2 | The command or flag syntax was invalid, including unknown commands, missing required arguments, invalid flag values, or invalid flag combinations. | | 2 | The command or flag syntax was invalid, including unknown commands, missing required arguments, invalid flag values, or invalid flag combinations. |
The root help spellings are the supported help path. Invoking **--help** on The root help spellings are the supported help path. Invoking **--help** on
**run**, a **config** subcommand, or **pipelines list** is handled by the flag **run**, **config validate**, or **pipelines list** is handled by the flag
parser as a usage error: it writes an error to standard error and exits with parser as a usage error: it writes an error to standard error and exits with
status 2. status 2.

View File

@@ -1,7 +1,7 @@
# Configuration # Configuration
This is the canonical reference for Notarius configuration. Configuration files This is the canonical reference for Notarius configuration. Configuration files
are YAML and must declare version 5. They select pipelines and their modules; are YAML and must declare version 3. They select pipelines and their modules;
the [CLI reference](cli.md) owns invocation syntax, and the [CLI reference](cli.md) owns invocation syntax, and
[Operations](operations.md) owns run-state procedures. [Operations](operations.md) owns run-state procedures.
@@ -19,16 +19,12 @@ The command fails if none of these paths provides a configuration file.
For configuration values, precedence is: For configuration values, precedence is:
1. built-in defaults; 1. built-in defaults;
2. composed YAML sources, including imports; 2. the selected YAML file;
3. the selected ConfigKit overlay; and 3. supported operational environment variables; and
4. explicitly supplied configuration-valued run flags. 4. the CLI run overrides that apply to a command.
`--config-overlay` selects a ConfigKit overlay from `overlays/<name>.yml` Environment variables do not provide a second configuration schema. They only
beneath the selected configuration root. Empty and `default` select no overlay. override the fields listed below.
`profiles/<name>.yml` is not a fallback location, and `--config-profile` is no
longer accepted. Configuration overlays are distinct from PromptKit LLM profiles selected by
`llm_profile` or `--llm-profile`. The retired operational environment variables
have no effect; `NOTARIUS_CONFIG` remains a source-discovery input.
## Maintained Examples ## Maintained Examples
@@ -36,9 +32,7 @@ have no effect; `NOTARIUS_CONFIG` remains a source-discovery input.
single-lane Seriatim-to-spell pipeline. single-lane Seriatim-to-spell pipeline.
- [Complete D&D configuration](../examples/dnd-complete.config.yml) uses - [Complete D&D configuration](../examples/dnd-complete.config.yml) uses
ordered steps, all implemented D&D lanes, generated references, state ordered steps, all implemented D&D lanes, generated references, state
settings, bounded LLM concurrency, and the maintained settings, and bounded LLM concurrency.
[LLM profile](../examples/llm-profiles/dnd-extraction.yml) and imports its
pipeline from [a fragment](../examples/pipelines/dnd-session.yml).
Use these complete files as starting points rather than combining the Use these complete files as starting points rather than combining the
illustrative fragments in this reference. illustrative fragments in this reference.
@@ -51,8 +45,8 @@ other than **version** is optional.
| Field | Type | Default | Rules | | Field | Type | Default | Rules |
| --- | --- | --- | --- | | --- | --- | --- | --- |
| **version** | integer or quoted string | none | Required; must be 5. Maintained files use `version: 5`. | | **version** | integer | none | Required; must be 3. |
| **promptkit** | object | none | Profile source and optional local-backend configuration. | | **scriptorium** | object | none | Profile source configuration. |
| **pipelines** | map | empty | Maps pipeline IDs to pipeline definitions. | | **pipelines** | map | empty | Maps pipeline IDs to pipeline definitions. |
| **concurrency** | object | see below | Global LLM and extraction limits. | | **concurrency** | object | see below | Global LLM and extraction limits. |
| **output** | object | see below | Published output settings. | | **output** | object | see below | Published output settings. |
@@ -63,7 +57,7 @@ Built-in defaults are:
| Field | Default | | Field | Default |
| --- | --- | | --- | --- |
| **concurrency.total_llm** | 16 | | **concurrency.total_llm** | 1 |
| **concurrency.stage_workers.extract** | Effective **total_llm** | | **concurrency.stage_workers.extract** | Effective **total_llm** |
| **output.directory** | **./notarius-output** | | **output.directory** | **./notarius-output** |
| **cache.chunk_plans.mode** | **auto** | | **cache.chunk_plans.mode** | **auto** |
@@ -74,112 +68,39 @@ Built-in defaults are:
An empty cache directory in YAML deliberately selects the corresponding An empty cache directory in YAML deliberately selects the corresponding
per-user root. An explicit empty output or debug directory is invalid. per-user root. An explicit empty output or debug directory is invalid.
Selecting a chunk-plan or checkpoint directory does not configure filesystem
access policy. Cache permissions come from deployment provisioning, process
umask, setgid inheritance, and default ACLs; see [Operations](operations.md#cache-provisioning).
Every nonempty relative configured destination or reference path resolves from ## Scriptorium Profiles
the canonical directory of the file that supplies its surviving declaration.
This includes imported fragments and selected configuration overlays. Built-in
relative destinations use the selected main configuration file's directory;
explicit `--output-dir` and `--debug-dir` values use the invocation working
directory. Absolute values remain unchanged.
The root file can use ConfigKit imports to compose disjoint definitions. Imports The optional **scriptorium** object selects one source of profile definitions:
and selected overlays remain under the configuration root; they do not merge or
append a pipeline's `steps` sequence. An overlay replaces sequences as defined
by ConfigKit, so changing an individual ordered binding requires supplying the
whole replacement sequence.
## PromptKit Profiles
The optional **promptkit** object selects one source of profile definitions and
may register one conventional local OpenAI-compatible backend:
~~~yaml
version: 5
promptkit:
profile_dir: ./profiles
# profile_file: ./profiles.yml
transient_retries: 2
local_backend:
endpoint: http://localhost:8000/v1
concurrency_limit: 2
~~~
| Field | Type | Rules | | Field | Type | Rules |
| --- | --- | --- | | --- | --- | --- |
| **profile_dir** | string | Non-empty directory containing profile files. | | **profile_dir** | string | Non-empty directory containing profile files. |
| **profile_file** | string | Non-empty profile file. | | **profile_file** | string | Non-empty profile file. |
| **transient_retries** | integer | Non-negative additional built-in HTTP attempts for eligible transient provider failures; defaults to 2. Set 0 to disable these retries. |
| **local_backend** | object | Optional registration for the conventional PromptKit backend ID **local**. |
| **local_backend.endpoint** | string | Required when **local_backend** is present; absolute HTTP or HTTPS URL with a host. |
| **local_backend.concurrency_limit** | integer | Optional non-negative limit; defaults to 0. |
Set at most one of **profile_dir** and **profile_file**. Every nonempty relative Set at most one of these fields. Profile IDs used by a binding must be available
configured filesystem path is resolved from the canonical directory of the from the selected Scriptorium profile source when the pipeline is resolved.
file that supplies it. The complete example's Keep credentials out of this file: configure a profile to read its credential
`./llm-profiles/dnd-extraction.yml` value therefore resolves beside that example. from an environment variable, then set that environment variable only in the
Use absolute paths for services and containers when their deployment layout is run environment.
not configuration-relative.
An operator source is optional. For a requested ID, PromptKit checks the ## Operational Environment Variables
configured operator source first, then Notarius's embedded fallback profiles,
then its own built-in catalog. A matching profile is complete: it replaces a
lower-precedence definition rather than merging with it. The maintained
[`dnd-extraction` LLM profile](../examples/llm-profiles/dnd-extraction.yml)
is a secret-free deployment artifact; production, development, and local
deployments can each provide a complete definition with that same workload ID.
Use workload-oriented IDs for new profiles instead of model names.
[Operations](operations.md#promptkit-profile-deployment) owns the deployment
workflow and credential-handling guidance.
When **local_backend** is present, its endpoint is trimmed and must use HTTP or These variables are applied after YAML values:
HTTPS case-insensitively, be absolute, and have a non-empty host. URL paths are
allowed. User information, queries, and fragments are rejected. A zero
**concurrency_limit** leaves the local backend unrestricted inside PromptKit;
a positive value limits simultaneous local generations. The application-wide
**concurrency.total_llm** limit still applies in both cases. Neither local
backend field has an environment override. Omitting **local_backend** registers
nothing and preserves existing built-in and endpoint-only profile behavior.
A file-backed PromptKit profile selects the registration by its case-sensitive | Variable | Overrides | Rules |
backend ID: | --- | --- | --- |
| **NOTARIUS_TOTAL_LLM_CONCURRENCY** | **concurrency.total_llm** | Integer. |
| **NOTARIUS_STAGE_WORKERS_EXTRACT** | **concurrency.stage_workers.extract** | Integer. |
| **NOTARIUS_OUTPUT_DIR** | **output.directory** | Non-empty path. |
| **NOTARIUS_CACHE_CHUNK_PLANS_MODE** | **cache.chunk_plans.mode** | **auto**, **bypass**, or **refresh**. |
| **NOTARIUS_CACHE_CHUNK_PLANS_DIR** | **cache.chunk_plans.directory** | Non-empty path. |
| **NOTARIUS_CACHE_CHECKPOINTS_DIR** | **cache.checkpoints.directory** | Non-empty path. |
| **NOTARIUS_DEBUG_DIR** | **debug.directory** | Non-empty path. |
~~~yaml Integer values are trimmed then parsed as base-10 integers. Directory and
id: local-summary output values reject NUL characters. **NOTARIUS_CONFIG** participates only in
backend: local configuration discovery.
model: example-model
~~~
Keep credentials out of the local-backend object. A PromptKit profile may name
its credential environment variable through `api_key_env`; set that variable
only in the run environment. PromptKit owns the
[pinned profile-file format](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/formats.md),
including `base_profile` inheritance. Notarius passes profiles through without
merging them. Filesystem profiles cannot express PromptKit's in-memory
`APIKeyRequired` setting; an unset `api_key_env` is optional and may reach the
provider without authorization.
The [PromptKit upstream boundary](integrations/pkg-promptkit.md) identifies the
supported package API, and [Operations](operations.md#operational-limits)
describes the effective concurrency layers.
`notarius config validate --pipeline <id>` resolves the selected pipeline and
inspects every explicit effective profile without contacting a provider or
requiring credential values. It rejects absent, malformed, or incompatible
profiles before a run prepares modules. Credential availability is checked only
when a generation is prepared.
## Version 5 Migration
Version 3 and version 4 files are rejected and are not rewritten at load time.
Use `version: 5`, rename the former `scriptorium` section to `promptkit`, and
convert every module binding to an object containing `module`. Convert every
external reference to `{path: ...}` and move any lane-level references to the
corresponding extraction binding. Put durable operational values in YAML or
use the documented explicit run flags; the former field-level environment
variables are not read.
## Concurrency, Output, Cache, And Debug ## Concurrency, Output, Cache, And Debug
@@ -204,14 +125,11 @@ debug:
**concurrency.total_llm** must be greater than zero. The only supported **concurrency.total_llm** must be greater than zero. The only supported
**concurrency.stage_workers** key is **extract**; its value must be from 1 **concurrency.stage_workers** key is **extract**; its value must be from 1
through **total_llm**. When omitted, it is recalculated from the effective through **total_llm**. When omitted, it is recalculated from the effective
**total_llm** after imports, overlay selection, and explicit ConfigKit **total_llm** after YAML and environment precedence.
overrides.
**cache.chunk_plans.mode** accepts **auto**, **bypass**, or **refresh**. **cache.chunk_plans.mode** accepts **auto**, **bypass**, or **refresh**.
**cache.checkpoints.enabled** is a boolean. The CLI can override the output **cache.checkpoints.enabled** is a boolean. The CLI can override the output
directory and chunk-plan mode for one run; see [CLI reference](cli.md#run). directory and chunk-plan mode for one run; see [CLI reference](cli.md#run).
Cache directory fields select only locations; Notarius has no cache permission
field or operational environment override.
## Pipelines ## Pipelines
@@ -220,28 +138,18 @@ Each **pipelines** entry has a unique, non-empty ID and the following shape:
~~~yaml ~~~yaml
pipelines: pipelines:
dnd-session: dnd-session:
llm_profile: dnd-extraction input: seriatim
input: chunk: generic
module: seriatim output: json
chunk:
module: generic
output:
module: json
artifacts: artifacts:
spells: spells:
extract: extract: dnd/spells
module: dnd/spells merge: appendorder
merge: normalize: dnd/spells
module: appendorder
normalize:
module: dnd/spells
~~~ ~~~
| Field | Type | Default | Rules | | Field | Type | Default | Rules |
| --- | --- | --- | --- | | --- | --- | --- | --- |
| **llm_profile** | string | none | Optional non-empty default PromptKit profile ID for selected LLM-backed bindings and validators. An explicitly present blank value is invalid. |
| **structured_output_repair_attempts** | integer | prompt-owned (1 in maintained production prompts) | Optional structural-repair limit from 0 through 3 for selected LLM-backed bindings and validators. Omission leaves the prompt's declared policy in control; explicit 0 disables structural repair at that scope. |
| **validation_policy** | object | see below | Optional terminal policy defaults for producer validation. Its fields inherit independently into chunk, extract, merge, and normalize bindings. |
| **input** | module binding | none | Required. | | **input** | module binding | none | Required. |
| **chunk** | module binding | **generic** | Optional. | | **chunk** | module binding | **generic** | Optional. |
| **output** | module binding | **json** | Optional. | | **output** | module binding | **json** | Optional. |
@@ -255,48 +163,6 @@ needs a unique non-empty **id**, an **artifacts** map, and may have
**references**. A lane ID must not appear more than once in a pipeline, **references**. A lane ID must not appear more than once in a pipeline,
including across explicit steps. including across explicit steps.
For each selected LLM-backed binding or validator, profile selection occurs
after module, validator, and `--only` lane selection. It uses the
run-level **--llm-profile** value first, then the binding's **llm_profile**,
then the pipeline's **llm_profile**, and finally the PromptKit default.
Deterministic bindings do not receive these defaults or run overrides.
Structural output repair is resolved after module, validator, and `--only` lane
selection. An object's **structured_output_repair_attempts** value takes
precedence over the pipeline value; otherwise, an LLM-backed binding or
validator inherits the pipeline value. If both are omitted, PromptKit uses the
prompt's declared repair policy. The value must be an integer from 0 through 3;
explicit `null` and non-integer values are invalid. An explicit value on a
deterministic binding or validator is invalid, while a pipeline value simply
does not apply to deterministic selections.
`validation_policy` controls terminal disposition for one complete producer
attempt and validator chain. It may appear on a pipeline or a **chunk**,
**extract**, **merge**, or **normalize** module binding; input, output, and
validator bindings reject it. Every field is optional and resolves in binding,
pipeline, then application-default order:
| Field | Values | Default |
| --- | --- | --- |
| **producer_structural_failure** | **fail_run**, **reject_output** | **fail_run** |
| **semantic_rejection** | **fail_run**, **reject_output** | **fail_run** |
| **validator_failure** | **warn_continue**, **fail_run** | **warn_continue** |
The policy object and its fields must be non-null, and unknown fields are
rejected. A deterministic producer may not explicitly set
**producer_structural_failure** on its binding, although a pipeline-level
default remains valid for pipelines that include LLM-backed producers.
After the producer binding's retry budget is exhausted, an invalid structured
response uses **producer_structural_failure**. One or more semantic validator
rejections use **semantic_rejection**; rejection takes precedence over an
exhausted validator failure or skip. With no rejection, an exhausted validator
failure or skip uses **validator_failure**. `reject_output` records the
terminal rejection without advancing that candidate. `warn_continue` is valid
only for validator execution failure: it advances a structurally valid,
otherwise unrejected result with incomplete-validation provenance and without
making it reusable checkpoint state.
A lane has these fields: A lane has these fields:
| Field | Type | Default | Rules | | Field | Type | Default | Rules |
@@ -304,20 +170,19 @@ A lane has these fields:
| **extract** | module binding | none | Required. | | **extract** | module binding | none | Required. |
| **merge** | module binding | **appendorder** | Optional. | | **merge** | module binding | **appendorder** | Optional. |
| **normalize** | module binding | **noop** | Optional. | | **normalize** | module binding | **noop** | Optional. |
| **references** | map | none | Not supported at lane scope; place references on the eligible binding. | | **references** | map | none | Supported compatibility alias for **extract.references**. |
| **validators** | list | none | Non-empty lane-level lists are rejected. Set validator overrides on a binding instead. | | **validators** | list | none | Non-empty lane-level lists are rejected. Set validator overrides on a binding instead. |
Lane-level **references** are not supported. Put a lane extraction reference The lane-level **references** alias remains accepted. When the alias and
under **extract.references**. **extract.references** bind the same slot, **extract.references** wins. Use
the binding-local form in new configurations.
## Module Bindings And Validators ## Module Bindings And Validators
Every module and validator binding uses an object, even when it has no fields Use a module key directly when no other binding fields are needed:
beyond its module key:
~~~yaml ~~~yaml
input: input: seriatim
module: seriatim
~~~ ~~~
Use an object for fields: Use an object for fields:
@@ -325,43 +190,26 @@ Use an object for fields:
~~~yaml ~~~yaml
extract: extract:
module: dnd/spells module: dnd/spells
llm_profile: dnd-extraction llm_profile: gemini-2-flash
retries: 2 retries: 2
references: references:
spell_catalog: spell_catalog: ./dnd-spell-catalog.json
path: ./dnd-spell-catalog.json
~~~ ~~~
| Binding field | Type | Default | Rules | | Binding field | Type | Default | Rules |
| --- | --- | --- | --- | | --- | --- | --- | --- |
| **module** | string | none | Required for an object binding. Must be a registered compatible key. | | **module** | string | none | Required for an object binding. Must be a registered compatible key. |
| **llm_profile** | string | none | Optional non-empty PromptKit profile ID for an LLM-backed binding. It overrides the pipeline default unless the run supplies **--llm-profile**. | | **llm_profile** | string | none | Optional non-empty Scriptorium profile ID. |
| **structured_output_repair_attempts** | integer | pipeline or prompt-owned (1 in maintained production prompts) | Optional structural-repair limit from 0 through 3 for an LLM-backed binding. It overrides the pipeline value; explicit 0 disables structural repair. | | **retries** | integer | 0 | Non-negative additional attempts for chunk, extract, merge, and normalize bindings. |
| **validation_policy** | object | pipeline or application defaults | Optional field-by-field terminal-policy override for a chunk, extract, merge, or normalize binding. |
| **retries** | integer | 0 | Non-negative additional complete producer attempts for chunk, extract, merge, and normalize bindings. This single budget covers operational errors, invalid structured output, module-requested normalization retry, and semantic correction. |
| **options** | object | none | Must satisfy the selected module. | | **options** | object | none | Must satisfy the selected module. |
| **references** | map | none | Valid only on chunk, extract, merge, and normalize bindings. | | **references** | map | none | Valid only on chunk, extract, merge, and normalize bindings. |
| **validators** | list | production chain | Valid only on chunk, extract, merge, and normalize bindings. | | **validators** | list | production chain | Valid only on chunk, extract, merge, and normalize bindings. |
Omitting **validators** uses the registered chain. **validators: []** selects Omitting **validators** uses the registered chain. **validators: []** selects
an empty chain; a non-empty list replaces the chain in the listed order. an empty chain; a non-empty list replaces the chain in the listed order.
Validator bindings accept only **module**, **llm_profile**, Validator bindings accept only **module**, **llm_profile**, and **options**.
**structured_output_repair_attempts**, **retries**, and **options**. Their They reject **references**, **retries**, and nested **validators**. Deterministic
**retries** value is a non-negative additional validator-execution budget and validators reject an explicit **llm_profile**.
is valid only when the selected validator is LLM-backed. A validator retry
rechecks the same immutable candidate; it never regenerates the producer.
They reject
**validation_policy**, **references**, and nested **validators**. Deterministic
validators reject explicit **llm_profile** and
**structured_output_repair_attempts**.
Deterministic module bindings also reject those explicit fields.
An LLM-backed chunk, extract, merge, or normalize producer with both a
non-empty validator chain and positive **retries** must declare the supported
single-response correction capability. Preparation rejects a configuration
that could require semantic correction from a producer that cannot provide an
exact prior response. A deterministic producer, or an LLM attempt that did
not make a model call, cannot consume a semantic retry after rejection.
The **json** output module accepts optional **include_chunk_map** and The **json** output module accepts optional **include_chunk_map** and
**evidence_context** settings: **evidence_context** settings:
@@ -375,7 +223,7 @@ output:
enabled: true enabled: true
window_units: 3 window_units: 3
lanes: lanes:
- npc-registry - npcs
- spells - spells
~~~ ~~~
@@ -396,37 +244,32 @@ Unknown outer or nested option fields are rejected, as are incompatible YAML
types. The allowlist remains valid when a run uses lane filtering: a configured types. The allowlist remains valid when a run uses lane filtering: a configured
lane that is not active for that invocation simply contributes no evidence. lane that is not active for that invocation simply contributes no evidence.
Evidence publication is opt-in because it can persist source text and metadata. Evidence publication is opt-in because it can persist source text and metadata.
When enabled, it publishes the selected source-unit excerpt defined by the Its payload contract is [Published Evidence Context](integrations/evidence-context.md).
[Published Evidence Context contract](integrations/evidence-context.md).
## References And Ordered Handoffs ## References And Ordered Handoffs
Reference maps bind named slots that the selected target declares. An external Reference maps bind named slots that the selected target declares. A scalar is
reference is an object with `path`. Pipeline-level maps accept only external an external path. Pipeline-level maps accept only external paths; step-local
paths; step-local and binding-local maps may also select a normalized artifact and binding-local maps may also select a normalized artifact from an earlier
from an earlier step: step:
~~~yaml ~~~yaml
steps: steps:
- id: describe-session - id: describe-session
artifacts: artifacts:
npc-registry: npcs:
extract: extract: dnd/npcs
module: dnd/npc-registry normalize: dnd/npcs
normalize:
module: dnd/npc-registry
- id: extract-events - id: extract-events
references: references:
npc_registry: npcs:
artifact: artifact:
step: describe-session step: describe-session
lane: npc-registry lane: npcs
artifacts: artifacts:
spells: spells:
extract: extract: dnd/spells
module: dnd/spells normalize: dnd/spells
normalize:
module: dnd/spells
~~~ ~~~
An artifact selector contains only **step** and **lane**. The producer must be An artifact selector contains only **step** and **lane**. The producer must be
@@ -435,16 +278,10 @@ slot. A generated binding supplies one accepted normalized artifact; it does
not name a file. A configured generated dependency remains required even when not name a file. A configured generated dependency remains required even when
that consumer slot is otherwise optional. that consumer slot is otherwise optional.
Pipeline references are configuration defaults. A matching step-local or Pipeline references are defaults. A matching step-local or binding-local
binding-local external path overrides a pipeline default. CLI reference external path overrides a pipeline default. Required slots must be bound after
bindings are then operational overrides of configured external paths; their these configuration values and any CLI reference overrides are applied.
pipeline, lane, and binding scopes and precedence are defined by the Reference paths in YAML are resolved relative to the configuration file.
[CLI reference](cli.md#reference-selectors). A CLI file reference cannot
replace a configured generated artifact handoff. Required slots must be bound
after configuration and CLI reference actions are applied. A YAML reference
path resolves from the file that supplies its surviving declaration, including
an imported fragment or selected overlay. CLI reference paths resolve relative
to the process working directory.
### D&D Reference Slots ### D&D Reference Slots
@@ -458,38 +295,14 @@ selected target declares them:
| **players** | Optional text player context. | | **players** | Optional text player context. |
| **glossary** | Optional text campaign glossary. | | **glossary** | Optional text campaign glossary. |
| **spell_catalog** | Optional JSON spell-catalog overlay for spell extraction and normalization. See [spell-catalog overlays](integrations/dnd-spell-catalog-overlays.md). | | **spell_catalog** | Optional JSON spell-catalog overlay for spell extraction and normalization. See [spell-catalog overlays](integrations/dnd-spell-catalog-overlays.md). |
| **location_registry** | Required normalized location registry for location-occurrence extraction and normalization. | | **npcs** | Normalized NPC registry. Optional for spells and combat turns; required for NPC interactions. |
| **item_registry** | Required normalized item registry for item-occurrence extraction and normalization. | | **scene_descriptions** | Required normalized scene-description artifact for combat-turn extraction. |
| **npc_registry** | Normalized NPC registry. Optional for spells and combat turns; required for NPC occurrences and enemy-event extraction and normalization. |
| **scene_descriptions** | Required normalized scene-description artifact for combat-turn and enemy-event extraction. |
| **combat_turns** | Required normalized combat-turn artifact for enemy-event extraction. |
| **npc_occurrences** | Required normalized NPC-occurrence artifact for enemy-event extraction. |
Registry-backed occurrence and enemy-event artifact slots have the following
exact binding contracts. Durable semantics and wire shapes remain in their
[NPC occurrence](integrations/dnd-npc-occurrence-artifacts.md),
[location occurrence](integrations/dnd-location-occurrence-artifacts.md),
[item occurrence](integrations/dnd-item-occurrence-artifacts.md), and
[enemy-event](integrations/dnd-enemy-event-artifacts.md) contracts.
| Slot | Accepted artifact kind | Media type | Maximum size | Required stage |
| --- | --- | --- | --- | --- |
| `npc_registry` | `dnd/npc-registry` | `application/json` | 1,048,576 bytes | extract and normalize |
| `scene_descriptions` | `dnd/scene-description-list` | `application/json` | 1,048,576 bytes | extract only |
| `combat_turns` | `dnd/combat-turn-list` | `application/json` | 1,048,576 bytes | extract only |
| `npc_occurrences` | `dnd/npc-occurrence-list` | `application/json` | 1,048,576 bytes | extract only |
| `location_registry` | `dnd/location-registry` | `application/json` | 1,048,576 bytes | location-occurrence extract and normalize |
| `item_registry` | `dnd/item-registry` | `application/json` | 1,048,576 bytes | item-occurrence extract and normalize |
Scene descriptions accept **party**, **players**, and **glossary**, but not Scene descriptions accept **party**, **players**, and **glossary**, but not
**roster**. NPC occurrences require **npc_registry** for both extraction and **roster**. NPC interactions require **npcs** for both extraction and
normalization. Combat turns require **scene_descriptions** for extraction; the normalization. Combat turns require **scene_descriptions** for extraction; the
normalized combat-turn module may use optional **npc_registry**. Location occurrences normalized combat-turn module may use optional **npcs**. The complete example
require **location_registry** for extraction and normalization. Item occurrences require shows generated **npcs** and **scene_descriptions** bindings.
**item_registry** for extraction and normalization. Enemy-event extraction requires all
four of its JSON artifact slots; its normalizer requires **npc_registry**.
The [complete example](../examples/dnd-complete.config.yml) shows the ordered
generated bindings.
## Production Module Keys ## Production Module Keys
@@ -497,29 +310,18 @@ generated bindings.
| --- | --- | | --- | --- |
| Input | **seriatim** | | Input | **seriatim** |
| Chunk | **generic**, **dnd/scenes** | | Chunk | **generic**, **dnd/scenes** |
| Extract | **dnd/spells**, **dnd/npc-registry**, **dnd/combat-turns**, **dnd/item-occurrences**, **dnd/item-registry**, **dnd/npc-occurrences**, **dnd/scene-descriptions**, **dnd/enemy-events**, **dnd/location-registry**, **dnd/location-occurrences** | | Extract | **dnd/spells**, **dnd/npcs**, **dnd/combat-turns**, **dnd/item-events**, **dnd/npc-interactions**, **dnd/scene-descriptions** |
| Merge | **appendorder** | | Merge | **appendorder** |
| Normalize | **noop**, **dnd/spells**, **dnd/npc-registry**, **dnd/combat-turns**, **dnd/item-occurrences**, **dnd/item-registry**, **dnd/npc-occurrences**, **dnd/scene-descriptions**, **dnd/enemy-events**, **dnd/location-registry**, **dnd/location-occurrences** | | Normalize | **noop**, **dnd/spells**, **dnd/npcs**, **dnd/combat-turns**, **dnd/item-events**, **dnd/npc-interactions**, **dnd/scene-descriptions** |
| Output | **json** | | Output | **json** |
`dnd/scenes` and every D&D extractor are `llm_backed`. The
`dnd/npc-registry`, `dnd/location-registry`, and `dnd/item-registry`
normalizers are also `llm_backed` for bounded duplicate proposals; every other
D&D normalizer is `deterministic`. LLM-backed bindings use the effective
[PromptKit profile](#promptkit-profiles). The complete example binds each
registry in an earlier step before its occurrence consumer.
The D&D artifact contracts define each emitted schema: The D&D artifact contracts define each emitted schema:
[spells](integrations/dnd-spell-artifacts.md), [spells](integrations/dnd-spell-artifacts.md),
[NPC registry](integrations/dnd-npc-registry-artifacts.md), [NPCs](integrations/dnd-npc-artifacts.md),
[NPC occurrences](integrations/dnd-npc-occurrence-artifacts.md), [NPC interactions](integrations/dnd-npc-interaction-artifacts.md),
[combat turns](integrations/dnd-combat-turn-artifacts.md), [combat turns](integrations/dnd-combat-turn-artifacts.md),
[item registry](integrations/dnd-item-registry-artifacts.md), [item events](integrations/dnd-item-event-artifacts.md), and
[item occurrences](integrations/dnd-item-occurrence-artifacts.md), [scene descriptions](integrations/dnd-scene-description-artifacts.md).
[scene descriptions](integrations/dnd-scene-description-artifacts.md),
[enemy events](integrations/dnd-enemy-event-artifacts.md),
[location registry](integrations/dnd-location-registry-artifacts.md), and
[location occurrences](integrations/dnd-location-occurrence-artifacts.md).
## Production Validator Keys And Default Chains ## Production Validator Keys And Default Chains
@@ -529,37 +331,11 @@ Available validator keys are:
| --- | --- | | --- | --- |
| Generic | **generic/always_accept**, **generic/always_reject**, **generic/valid_json**, **generic/valid_json_schema** | | Generic | **generic/always_accept**, **generic/always_reject**, **generic/valid_json**, **generic/valid_json_schema** |
| Spells | **extract/dnd/spells/shape**, **extract/dnd/spells/catalog**, **extract/dnd/spells/source_refs**, **extract/dnd/spells/source_relatedness** | | Spells | **extract/dnd/spells/shape**, **extract/dnd/spells/catalog**, **extract/dnd/spells/source_refs**, **extract/dnd/spells/source_relatedness** |
| NPC registry | **extract/dnd/npc-registry/shape**, **extract/dnd/npc-registry/source_refs**, **extract/dnd/npc-registry/source_relatedness**, **normalize/dnd/npc-registry/identity** | | NPCs | **extract/dnd/npcs/shape**, **extract/dnd/npcs/source_refs**, **extract/dnd/npcs/source_relatedness**, **normalize/dnd/npcs/identity** |
| Combat turns | **extract/dnd/combat-turns/shape**, **extract/dnd/combat-turns/source_refs**, **extract/dnd/combat-turns/source_relatedness**, **normalize/dnd/combat-turns/invariants** | | Combat turns | **extract/dnd/combat-turns/shape**, **extract/dnd/combat-turns/source_refs**, **extract/dnd/combat-turns/source_relatedness**, **normalize/dnd/combat-turns/invariants** |
| Item occurrences | **extract/dnd/item-occurrences/shape**, **extract/dnd/item-occurrences/registry**, **extract/dnd/item-occurrences/source_refs**, **extract/dnd/item-occurrences/source_relatedness**, **normalize/dnd/item-occurrences/invariants** | | Item events | **extract/dnd/item-events/shape**, **extract/dnd/item-events/source_refs**, **extract/dnd/item-events/source_relatedness**, **normalize/dnd/item-events/invariants** |
| Item registry | **extract/dnd/item-registry/shape**, **extract/dnd/item-registry/source_refs**, **extract/dnd/item-registry/source_relatedness**, **normalize/dnd/item-registry/identity** | | NPC interactions | **extract/dnd/npc-interactions/shape**, **extract/dnd/npc-interactions/registry**, **extract/dnd/npc-interactions/source_refs**, **extract/dnd/npc-interactions/source_relatedness**, **normalize/dnd/npc-interactions/invariants** |
| NPC occurrences | **extract/dnd/npc-occurrences/shape**, **extract/dnd/npc-occurrences/registry**, **extract/dnd/npc-occurrences/source_refs**, **extract/dnd/npc-occurrences/source_relatedness**, **normalize/dnd/npc-occurrences/invariants** | | Scene descriptions | **extract/dnd/scene-descriptions/shape**, **extract/dnd/scene-descriptions/source_refs**, **extract/dnd/scene-descriptions/source_relatedness**, **normalize/dnd/scene-descriptions/invariants** |
| Scene descriptions | **extract/dnd/scene-descriptions/shape**, **extract/dnd/scene-descriptions/source_refs**, **extract/dnd/scene-descriptions/source_relatedness**, **extract/dnd/scene-descriptions/combat_semantics** (LLM-backed, opt-in), **normalize/dnd/scene-descriptions/invariants** |
| Enemy events | **extract/dnd/enemy-events/shape**, **extract/dnd/enemy-events/engagements**, **extract/dnd/enemy-events/source_refs**, **extract/dnd/enemy-events/source_relatedness**, **normalize/dnd/enemy-events/invariants** |
| Location registry | **extract/dnd/location-registry/shape**, **extract/dnd/location-registry/source_refs**, **extract/dnd/location-registry/source_relatedness**, **normalize/dnd/location-registry/identity** |
| Location occurrences | **extract/dnd/location-occurrences/shape**, **extract/dnd/location-occurrences/registry**, **extract/dnd/location-occurrences/source_refs**, **extract/dnd/location-occurrences/source_relatedness**, **normalize/dnd/location-occurrences/invariants** |
`extract/dnd/scene-descriptions/combat_semantics` is not in a production default chain. To opt in, replace the scene extractor validator chain with the current ordered chain plus the semantic validator last, and set a positive producer retry budget if a rejection should request a corrected scene:
~~~yaml
extract:
module: dnd/scene-descriptions
retries: 1
validators:
- module: generic/valid_json
- module: extract/dnd/scene-descriptions/shape
- module: extract/dnd/scene-descriptions/source_refs
- module: generic/valid_json_schema
- module: extract/dnd/scene-descriptions/source_relatedness
- module: extract/dnd/scene-descriptions/combat_semantics
retries: 1
~~~
An override replaces, rather than extends, the default chain. See [Module Bindings And Validators](#module-bindings-and-validators) for binding, profile, repair, retry, and failure-policy rules.
The validator retry shown above permits one additional execution against the
same scene candidate when the LLM-backed validator itself fails; it is separate
from both the extractor's producer retry and PromptKit structural repair.
When no override is configured, production D&D bindings use the following When no override is configured, production D&D bindings use the following
ordered chains. Each row lists extract then normalize; spell chains are the ordered chains. Each row lists extract then normalize; spell chains are the
@@ -568,15 +344,11 @@ same at both stages.
| Lane | Extract | Normalize | | Lane | Extract | Normalize |
| --- | --- | --- | | --- | --- | --- |
| Spells | generic/valid_json, extract/dnd/spells/shape, extract/dnd/spells/catalog, extract/dnd/spells/source_refs, generic/valid_json_schema, extract/dnd/spells/source_relatedness | Same as extract | | Spells | generic/valid_json, extract/dnd/spells/shape, extract/dnd/spells/catalog, extract/dnd/spells/source_refs, generic/valid_json_schema, extract/dnd/spells/source_relatedness | Same as extract |
| NPC registry | generic/valid_json, extract/dnd/npc-registry/shape, extract/dnd/npc-registry/source_refs, generic/valid_json_schema, extract/dnd/npc-registry/source_relatedness | generic/valid_json, extract/dnd/npc-registry/shape, normalize/dnd/npc-registry/identity, extract/dnd/npc-registry/source_refs, generic/valid_json_schema, extract/dnd/npc-registry/source_relatedness | | NPCs | generic/valid_json, extract/dnd/npcs/shape, extract/dnd/npcs/source_refs, generic/valid_json_schema, extract/dnd/npcs/source_relatedness | generic/valid_json, extract/dnd/npcs/shape, normalize/dnd/npcs/identity, extract/dnd/npcs/source_refs, generic/valid_json_schema, extract/dnd/npcs/source_relatedness |
| Combat turns | generic/valid_json, extract/dnd/combat-turns/shape, extract/dnd/combat-turns/source_refs, generic/valid_json_schema, extract/dnd/combat-turns/source_relatedness | generic/valid_json, extract/dnd/combat-turns/shape, normalize/dnd/combat-turns/invariants, extract/dnd/combat-turns/source_refs, generic/valid_json_schema, extract/dnd/combat-turns/source_relatedness | | Combat turns | generic/valid_json, extract/dnd/combat-turns/shape, extract/dnd/combat-turns/source_refs, generic/valid_json_schema, extract/dnd/combat-turns/source_relatedness | generic/valid_json, extract/dnd/combat-turns/shape, normalize/dnd/combat-turns/invariants, extract/dnd/combat-turns/source_refs, generic/valid_json_schema, extract/dnd/combat-turns/source_relatedness |
| Item occurrences | generic/valid_json, extract/dnd/item-occurrences/shape, extract/dnd/item-occurrences/registry, extract/dnd/item-occurrences/source_refs, generic/valid_json_schema, extract/dnd/item-occurrences/source_relatedness | generic/valid_json, extract/dnd/item-occurrences/shape, extract/dnd/item-occurrences/registry, normalize/dnd/item-occurrences/invariants, extract/dnd/item-occurrences/source_refs, generic/valid_json_schema, extract/dnd/item-occurrences/source_relatedness | | Item events | generic/valid_json, extract/dnd/item-events/shape, extract/dnd/item-events/source_refs, generic/valid_json_schema, extract/dnd/item-events/source_relatedness | generic/valid_json, extract/dnd/item-events/shape, normalize/dnd/item-events/invariants, extract/dnd/item-events/source_refs, generic/valid_json_schema, extract/dnd/item-events/source_relatedness |
| Item registry | generic/valid_json, extract/dnd/item-registry/shape, extract/dnd/item-registry/source_refs, generic/valid_json_schema, extract/dnd/item-registry/source_relatedness | generic/valid_json, extract/dnd/item-registry/shape, normalize/dnd/item-registry/identity, extract/dnd/item-registry/source_refs, generic/valid_json_schema, extract/dnd/item-registry/source_relatedness | | NPC interactions | generic/valid_json, extract/dnd/npc-interactions/shape, extract/dnd/npc-interactions/registry, extract/dnd/npc-interactions/source_refs, generic/valid_json_schema, extract/dnd/npc-interactions/source_relatedness | generic/valid_json, extract/dnd/npc-interactions/shape, extract/dnd/npc-interactions/registry, normalize/dnd/npc-interactions/invariants, extract/dnd/npc-interactions/source_refs, generic/valid_json_schema, extract/dnd/npc-interactions/source_relatedness |
| NPC occurrences | generic/valid_json, extract/dnd/npc-occurrences/shape, extract/dnd/npc-occurrences/registry, extract/dnd/npc-occurrences/source_refs, generic/valid_json_schema, extract/dnd/npc-occurrences/source_relatedness | generic/valid_json, extract/dnd/npc-occurrences/shape, extract/dnd/npc-occurrences/registry, normalize/dnd/npc-occurrences/invariants, extract/dnd/npc-occurrences/source_refs, generic/valid_json_schema, extract/dnd/npc-occurrences/source_relatedness |
| Scene descriptions | generic/valid_json, extract/dnd/scene-descriptions/shape, extract/dnd/scene-descriptions/source_refs, generic/valid_json_schema, extract/dnd/scene-descriptions/source_relatedness | generic/valid_json, extract/dnd/scene-descriptions/shape, normalize/dnd/scene-descriptions/invariants, extract/dnd/scene-descriptions/source_refs, generic/valid_json_schema, extract/dnd/scene-descriptions/source_relatedness | | Scene descriptions | generic/valid_json, extract/dnd/scene-descriptions/shape, extract/dnd/scene-descriptions/source_refs, generic/valid_json_schema, extract/dnd/scene-descriptions/source_relatedness | generic/valid_json, extract/dnd/scene-descriptions/shape, normalize/dnd/scene-descriptions/invariants, extract/dnd/scene-descriptions/source_refs, generic/valid_json_schema, extract/dnd/scene-descriptions/source_relatedness |
| Enemy events | generic/valid_json, extract/dnd/enemy-events/shape, extract/dnd/enemy-events/engagements, extract/dnd/enemy-events/source_refs, generic/valid_json_schema, extract/dnd/enemy-events/source_relatedness | generic/valid_json, extract/dnd/enemy-events/shape, normalize/dnd/enemy-events/invariants, extract/dnd/enemy-events/source_refs, generic/valid_json_schema, extract/dnd/enemy-events/source_relatedness |
| Location registry | generic/valid_json, extract/dnd/location-registry/shape, extract/dnd/location-registry/source_refs, generic/valid_json_schema, extract/dnd/location-registry/source_relatedness | generic/valid_json, extract/dnd/location-registry/shape, normalize/dnd/location-registry/identity, extract/dnd/location-registry/source_refs, generic/valid_json_schema, extract/dnd/location-registry/source_relatedness |
| Location occurrences | generic/valid_json, extract/dnd/location-occurrences/shape, extract/dnd/location-occurrences/registry, extract/dnd/location-occurrences/source_refs, generic/valid_json_schema, extract/dnd/location-occurrences/source_relatedness | generic/valid_json, extract/dnd/location-occurrences/shape, extract/dnd/location-occurrences/registry, normalize/dnd/location-occurrences/invariants, extract/dnd/location-occurrences/source_refs, generic/valid_json_schema, extract/dnd/location-occurrences/source_relatedness |
Chains are only registered for the D&D extract and normalize modules shown Chains are only registered for the D&D extract and normalize modules shown
above; select an explicit override when a different compatible chain is above; select an explicit override when a different compatible chain is
@@ -593,16 +365,7 @@ go run ./cmd/notarius config validate \
~~~ ~~~
Configuration validation rejects invalid YAML, unsupported fields, invalid Configuration validation rejects invalid YAML, unsupported fields, invalid
defaults or explicit CLI overrides, incompatible module keys, unknown options, defaults or environment overrides, incompatible module keys, unknown options,
invalid reference bindings, missing required reference slots, invalid validator invalid reference bindings, missing required reference slots, invalid validator
overrides, and incompatible generated artifact handoffs. Use overrides, and incompatible generated artifact handoffs. Use
[pipelines list](cli.md#pipelines-list) to inspect configured IDs. [pipelines list](cli.md#pipelines-list) to inspect configured IDs.
## Inspection
Use [config effective](cli.md#config-effective) to write the selected composed
configuration as deterministic, redacted YAML. Use
[config explain](cli.md#config-explain) with a canonical JSON Pointer to inspect
the effective or historical provenance for one path, including a value displaced
by a selected overlay. Neither command resolves a pipeline, prepares modules,
materializes references, or requires provider credentials.

View File

@@ -1,221 +0,0 @@
# Consuming The Complete D&D Pipeline
Use this workflow when an orchestrator runs the maintained complete D&D
pipeline and consumes its structured JSON artifacts. The generic
[subprocess consumer guide](subprocess.md) owns process-level responsibilities;
this guide connects that workflow to the complete D&D configuration, its
Seriatim input, and its artifact inventory.
The [CLI reference](../cli.md), [configuration reference](../config.md),
[run-result receipt](../integrations/run-result.md), and
[published JSON output contract](../integrations/json-output.md) remain the
canonical definitions of those public interfaces.
## Prepare And Validate The Deployment
Start from the maintained
[complete D&D configuration](../../examples/dnd-complete.config.yml). It uses
the `dnd-session` pipeline and demonstrates every implemented D&D lane, ordered
artifact handoffs, campaign references, chunk-map publication, and evidence
context.
A deployment must provide its own PromptKit profile and campaign reference
files. Use absolute paths for service and subprocess deployments. In
particular, observe these different resolution rules:
- reference and PromptKit profile paths in YAML are resolved relative to the
file that supplies their surviving declaration;
- reference paths passed with `--reference` are resolved relative to the
Notarius process working directory.
Do not copy the repository example's relative profile path into a deployment
without preserving its configuration-relative layout. The complete path and
profile rules are defined in [Configuration](../config.md).
Preflight the deployed configuration before processing sessions and whenever
it changes:
```sh
notarius config validate \
--config /absolute/path/to/notarius.yml \
--pipeline dnd-session
```
Provide credentials through the environment or the documented configuration
mechanism. Do not put credentials in command arguments, generated
configuration, or logs.
## Supply The Transcript
The complete pipeline consumes a Seriatim JSON document. The
[Seriatim input contract](../integrations/seriatim.md) defines its required
metadata, segments, and validation rules. Preserve segment IDs: D&D artifact
citations use those segment IDs as source-unit ranges.
When the caller maintains several transcript tiers, use the final trimmed JSON
transcript so extraction operates on the same session content presented to
later consumers. For example, Narratio identifies this implemented artifact as
`narratio.transcript.final_trimmed` and normally stores it at
`transcripts/final.trimmed.json`.
Notarius generates a stable prompt session from the resolved input module and
the exact input bytes. An ordinary orchestrator should not pass `--session-id`.
Use that override only when intentionally changing the routing relationship
between invocations; it is not a credential or output identity.
## Run Notarius
Invoke the pipeline with explicit absolute paths and request its
machine-readable receipt:
```sh
notarius run dnd-session \
--config /absolute/path/to/notarius.yml \
--input /absolute/path/to/transcripts/final.trimmed.json \
--output-dir /absolute/path/to/notarius-output \
--reference party=/absolute/path/to/references/party.txt \
--reference players=/absolute/path/to/references/players.txt \
--reference glossary=/absolute/path/to/references/glossary.txt \
--reference spell_catalog=/absolute/path/to/references/spells.json \
--json
```
Each unqualified reference is pipeline-scoped: Notarius supplies it to every
selected D&D target that declares the slot. A deployment may omit an optional
reference it does not maintain, and may use the lane- or binding-qualified
forms from the [CLI reference](../cli.md#reference-selectors) for an exceptional
override. The registry, scene-description, combat-turn, and NPC-occurrence
references declared between ordered steps in the complete configuration are
generated artifacts. Do not pass those handoffs on the CLI.
The caller should:
- capture stdout and stderr separately;
- propagate cancellation and impose an operator-appropriate timeout;
- wait for process completion before interpreting stdout; and
- retain stderr for diagnosis without copying secrets or transcript content
into other logs.
Only exit status 0 permits decoding stdout as a receipt. Ignore stdout after a
nonzero exit because a failed receipt write can leave partial bytes. The
[CLI reference](../cli.md#output-streams-and-exit-statuses) defines the complete
stream and exit-status contract.
## Discover The Published Bundle
Decode the successful stdout document as a supported run-result schema. For
the current contract, `schema_version` is `notarius.run-result.v2`. Tolerate
unknown fields allowed by that version, but reject an unsupported schema
version.
Use the receipt's absolute `output_directory` as the exact run-specific bundle
root. Do not scan the output root for its newest directory, guess a run ID, or
construct a bundle path. Resolve `index_file` beneath `output_directory` and
reject an absolute logical path or any result that escapes the bundle root.
The complete configuration uses the application validation defaults. A caller
that requires fully validated D&D artifacts must also require receipt
`validation_status: approved`; a successful `incomplete` result reflects the
configured validator-failure continuation policy and carries its bounded
validator provenance in `validation_summaries`.
Read `index.json` and locate each requested lane in `output_files` by its exact
`lane_id`. Do not guess a lane filename. Before decoding a payload:
1. resolve its descriptor's relative `file` beneath the bundle root with the
same confinement check;
2. verify the descriptor's media type and schema identity against the linked
artifact contract; and
3. decode the payload according to that contract.
The [published JSON output contract](../integrations/json-output.md) defines
the index and bundle layout. Treat all paths obtained from a decoded external
document as untrusted until confined to their documented root.
## Complete Artifact Inventory
When every configured lane is accepted, the complete example publishes these
lane artifacts:
| Lane ID | Purpose | Canonical contract |
| --- | --- | --- |
| `item-registry` | Canonical registry of encountered items and currency. | [Item registry](../integrations/dnd-item-registry-artifacts.md) |
| `npc-registry` | Canonical registry of named NPCs. | [NPC registry](../integrations/dnd-npc-registry-artifacts.md) |
| `location-registry` | Canonical registry of named locations. | [Location registry](../integrations/dnd-location-registry-artifacts.md) |
| `scene-descriptions` | Classification, title, and summary for each scene. | [Scene descriptions](../integrations/dnd-scene-description-artifacts.md) |
| `item-occurrences` | Source-grounded item discovery, acquisition, use, transfer, and loss events. | [Item occurrences](../integrations/dnd-item-occurrence-artifacts.md) |
| `spells` | Source-grounded spell casts and casters. | [Spell casts](../integrations/dnd-spell-artifacts.md) |
| `combat-turns` | Source-grounded combat turn participation. | [Combat turns](../integrations/dnd-combat-turn-artifacts.md) |
| `npc-occurrences` | Source-grounded NPC interaction occurrences. | [NPC occurrences](../integrations/dnd-npc-occurrence-artifacts.md) |
| `location-occurrences` | Source-grounded location occurrences. | [Location occurrences](../integrations/dnd-location-occurrence-artifacts.md) |
| `enemy-events` | Source-grounded enemy combat events. | [Enemy events](../integrations/dnd-enemy-event-artifacts.md) |
The JSON encoder always publishes these bundle-management files:
| File | Purpose |
| --- | --- |
| `index.json` | Discovery document for lane and pipeline-wide artifacts. |
| `manifest.json` | Run provenance and result summaries. |
| `rejected.json` | Rejected pipeline outputs. |
| `warnings.json` | Actionable process-degradation warnings. |
| `diagnostics.json` | Advisory and observation findings for accepted artifacts. |
The complete configuration also requests two pipeline-wide artifacts:
- [`chunk-map.json`](../integrations/chunk-map.md), the accepted chunk plan and
chunk metadata; and
- [`evidence-context.json`](../integrations/evidence-context.md), a reading
excerpt containing the union of selected cited source units and the
configured surrounding window.
Discover both from their top-level `index.json` descriptors rather than
treating them as lanes. Evidence context is convenient reading material, not
authoritative provenance; citations in the normalized lane payloads remain the
evidence contract.
Every optional or lane file is published only when its corresponding artifact
is available. A successful process does not guarantee that all configured
lanes were accepted.
## Decide What Counts As Consumer Success
Exit status 0 means Notarius completed the pipeline and published its result
bundle. The receipt or bundle may still report warnings, rejected outputs, or
missing lane descriptors. A downstream consumer must define its own required
artifact set explicitly.
A caller that claims to consume the complete D&D workflow should normally
require all ten lane IDs in the table and verify each descriptor's expected
contract. If any required lane is missing, rejected, or incompatible, fail the
caller's extraction step while retaining the Notarius bundle for diagnosis. A
consumer that needs only a subset may define and document a narrower policy.
Keep the successful receipt with the complete published bundle. Retain
`manifest.json`, `rejected.json`, `warnings.json`, and captured process logs as
required by the caller's provenance, diagnosis, and retention policies. Avoid
selectively copying payload files without also preserving enough index and
manifest information to identify their originating run and contracts.
The transcript, lane artifacts, evidence context, manifest, debug data, and
logs can all contain private campaign information. Apply the same access,
publication, and retention controls used for the source transcript.
## Consumer Checklist
- Validate the deployed Notarius configuration and `dnd-session` pipeline.
- Pass the final trimmed Seriatim JSON transcript with stable segment IDs.
- Use absolute configuration, input, output-root, profile, and reference paths
in service deployments.
- Capture stdout and stderr separately and enforce cancellation and timeout.
- Parse stdout only after exit status 0.
- Accept only supported receipt, index, and artifact schema versions while
tolerating permitted unknown fields.
- Use the receipt's `output_directory`; never guess the run directory.
- Confine `index_file` and every descriptor path to the published bundle root.
- Discover lanes by `lane_id` and verify descriptor compatibility before
decoding payloads.
- Enforce an explicit required-lane policy and inspect rejections and warnings.
- Preserve the receipt and sufficient bundle provenance for every retained
artifact.
- Protect all transcript-derived files and diagnostic streams as sensitive
campaign data.

View File

@@ -6,21 +6,14 @@ statuses, while the [run-result receipt](../integrations/run-result.md) and
[Published JSON Output contract](../integrations/json-output.md) own the [Published JSON Output contract](../integrations/json-output.md) own the
durable result formats. durable result formats.
For the maintained complete D&D workflow, including its transcript input,
configured lane inventory, and downstream acceptance checklist, see
[Consuming The Complete D&D Pipeline](dnd-pipeline.md).
## Run And Check The Process ## Run And Check The Process
Optionally preflight a selected configuration and pipeline before work starts: Optionally preflight a selected configuration and pipeline before work starts:
```sh ```sh
notarius config validate --config /path/to/notarius.yml --config-overlay production --pipeline pipeline-id notarius config validate --config /path/to/notarius.yml --pipeline pipeline-id
``` ```
Omit `--config-overlay` when the composed base configuration is the intended
selection; it is independent from any `--llm-profile` run override.
Invoke the run with explicit paths and machine-readable output. Capture Invoke the run with explicit paths and machine-readable output. Capture
standard output and standard error separately; do not combine them before standard output and standard error separately; do not combine them before
processing the result. processing the result.
@@ -28,43 +21,16 @@ processing the result.
```sh ```sh
notarius run pipeline-id \ notarius run pipeline-id \
--config /path/to/notarius.yml \ --config /path/to/notarius.yml \
--config-overlay production \
--input /path/to/source.json \ --input /path/to/source.json \
--output-dir /path/to/output-root \ --output-dir /path/to/output-root \
--json --json
``` ```
Use absolute paths for supplied input, configuration, output-root, and Use absolute paths for supplied input, configuration, output-root, and
reference files. Pass each external reference as its own argument-vector pair; reference files. When a stable prompt session identifier or references are
do not construct and invoke a shell command. An unqualified reference selector needed, pass the supported CLI flags. Supply credentials through Notarius's
supplies that file to every compatible selected target. Lane and stage documented configuration and environment mechanisms, never as command-line
qualification are available for exceptional overrides, while generated arguments or generated secret-bearing configuration.
same-run references remain part of configured pipeline composition. The
[CLI reference](../cli.md#reference-selectors) owns the exact selector and
precedence contract.
The maintained D&D subprocess workflow uses this facility for campaign context:
```sh
notarius run dnd-session \
--config /absolute/path/to/notarius.yml \
--input /absolute/path/to/transcripts/final.trimmed.json \
--output-dir /absolute/path/to/notarius-output \
--reference party=/absolute/path/to/references/party.txt \
--reference players=/absolute/path/to/references/players.txt \
--reference glossary=/absolute/path/to/references/glossary.txt \
--reference spell_catalog=/absolute/path/to/references/spells.json \
--json
```
Only pass the external references available to and desired by the deployment.
Notarius generates a stable prompt session for the resolved input module and
exact input bytes; reference changes do not change it. Pass **--session-id**
only when intentionally grouping different invocations under a different
session. Supply credentials through Notarius's documented configuration and
environment mechanisms, never as command-line arguments or generated
secret-bearing configuration. In particular, a session identifier is
provider-visible and is not a credential mechanism.
Wait for the process before interpreting standard output. Only an exit status Wait for the process before interpreting standard output. Only an exit status
of 0 permits decoding the receipt. On a nonzero exit, retain standard error for of 0 permits decoding the receipt. On a nonzero exit, retain standard error for
@@ -86,30 +52,23 @@ contract. The JSON bundle contract links to the available lane contracts.
If `index.json` has an `evidence_context` descriptor, treat it as a If `index.json` has an `evidence_context` descriptor, treat it as a
pipeline-wide artifact rather than a lane entry. Verify its six descriptor pipeline-wide artifact rather than a lane entry. Verify its six descriptor
fields before decoding the linked file according to the [Published Evidence fields before decoding the linked file according to the [Published Evidence
Context contract](../integrations/evidence-context.md). Decode its top-level Context contract](../integrations/evidence-context.md). Use each
source-unit array as a reading excerpt. Obtain authoritative citations and lane `evidence_refs` entry as the citation to source material. Its surrounding
provenance from the normalized lane artifacts; the excerpt has neither and its context range and included units explain the citation, but do not widen or
nearby units do not widen a lane artifact's cited source reference. replace the cited source reference.
A zero exit status may still report rejected outputs, warnings, or absent A zero exit status may still report rejected outputs, warnings, or absent
lanes. The caller decides which lane IDs are required for its own work and lanes. The caller decides which lane IDs are required for its own work and
which are optional; it should make that decision explicitly rather than infer which are optional; it should make that decision explicitly rather than infer
failure from the receipt counts alone. failure from the receipt counts alone.
When complete validation is required, also require receipt
`validation_status: approved` and inspect `validation_summaries`. A successful
run with `validation_status: incomplete` contains a structurally valid result
that advanced after validator execution could not complete under the configured
`warn_continue` policy. It is not reusable checkpoint state and should not be
silently treated as fully reviewed by the caller.
## Preserve Provenance And Handle Data Carefully ## Preserve Provenance And Handle Data Carefully
Keep the receipt with the published `manifest.json`, and retain Keep the receipt with the published `manifest.json`, and retain
`rejected.json`, `warnings.json`, and `diagnostics.json` when review or later provenance requires `rejected.json` and `warnings.json` when review or later provenance requires
them. Treat the input, output bundle, cache, debug bundle, and captured process them. Treat the input, output bundle, cache, debug bundle, and captured process
logs as potentially sensitive data. Apply the caller's access controls and logs as potentially sensitive data. Apply the caller's access controls and
retention policy, and avoid copying secrets into arguments, logs, or retention policy, and avoid copying secrets into arguments, logs, or
provenance records. An evidence-context artifact contains source-unit text and provenance records. An evidence-context artifact contains source-unit text and
metadata and can cover most of an input; preserve and share it only when that metadata, and selected lanes can cover most of an input; preserve and share it
source content is authorized for the recipient. only when that source content is authorized for the recipient.

View File

@@ -9,9 +9,6 @@ the [README](../README.md) for product context, [Architecture](policy/architectu
for system boundaries, and [Internal Overview](internal/overview.md) for the for system boundaries, and [Internal Overview](internal/overview.md) for the
implemented component map. implemented component map.
Read [Agent Policy](policy/agents.md) before recommending, planning, or
implementing changes. It owns implementation discipline and local commit rules.
## What To Read ## What To Read
| When working on | Read | Why | | When working on | Read | Why |
@@ -21,14 +18,13 @@ implementing changes. It owns implementation discipline and local commit rules.
| Any documentation addition or revision | [Documentation Policy](policy/documentation.md) | It defines canonical homes, audiences, current-behavior rules, and maintenance requirements. | | Any documentation addition or revision | [Documentation Policy](policy/documentation.md) | It defines canonical homes, audiences, current-behavior rules, and maintenance requirements. |
| Adding, changing, reviewing, or deleting tests | [Testing Policy](policy/testing.md) | It defines risk-based sufficiency, durable test boundaries, test-double guidance, and criteria for retaining tests. | | Adding, changing, reviewing, or deleting tests | [Testing Policy](policy/testing.md) | It defines risk-based sufficiency, durable test boundaries, test-double guidance, and criteria for retaining tests. |
| CLI composition or command behavior | [CLI Internals](internal/cli.md) and [CLI Reference](cli.md) | The internal guide owns composition and command flow; the reference owns public syntax. | | CLI composition or command behavior | [CLI Internals](internal/cli.md) and [CLI Reference](cli.md) | The internal guide owns composition and command flow; the reference owns public syntax. |
| Building a subprocess caller or changing its result protocol | [Subprocess Consumer Guide](consumers/subprocess.md), [Complete D&D Consumer Guide](consumers/dnd-pipeline.md), [Run Result Receipt](integrations/run-result.md), and [CLI Internals](internal/cli.md) | These separate generic caller workflow, the complete D&D workflow, the durable receipt contract, and CLI implementation behavior. | | Building a subprocess caller or changing its result protocol | [Subprocess Consumer Guide](consumers/subprocess.md), [Run Result Receipt](integrations/run-result.md), and [CLI Internals](internal/cli.md) | These separate caller workflow, durable receipt contract, and CLI implementation behavior. |
| Configuration loading, resolution, or user-visible configuration behavior | [Configuration Internals](internal/configuration.md) and [Configuration](config.md) | The internal guide owns loading and resolution mechanics; the reference owns the configuration contract. | | Configuration loading, resolution, or user-visible configuration behavior | [Configuration Internals](internal/configuration.md) and [Configuration](config.md) | The internal guide owns loading and resolution mechanics; the reference owns the configuration contract. |
| Pipeline resolution or execution | [Pipeline Internals](internal/pipeline.md) | It documents profiles, references, validation, retries, checkpoints, and runner behavior. | | Pipeline resolution or execution | [Pipeline Internals](internal/pipeline.md) | It documents profiles, references, validation, retries, checkpoints, and runner behavior. |
| Production modules or validators | [Module Internals](internal/modules.md), [D&D Module Internals](internal/dnd.md), and [D&D integration contracts](integrations/) | The generic guide owns extension mechanics, the D&D guide owns shared family conventions, and the contracts own durable output shapes. | | Production modules or validators | [Module Internals](internal/modules.md), [D&D Module Internals](internal/dnd.md), and [D&D integration contracts](integrations/) | The generic guide owns extension mechanics, the D&D guide owns shared family conventions, and the contracts own durable output shapes. |
| LLM clients, prompts, schemas, profiles, or scheduling | [LLM Runtime](internal/llm.md) | It documents the transport boundary and PromptKit integration. | | LLM clients, prompts, schemas, profiles, or scheduling | [LLM Runtime](internal/llm.md) | It documents the transport boundary and Scriptorium integration. |
| Output, cache, resume, or debug artifacts | [Run State Internals](internal/state.md), [Operations](operations.md), and [Configuration](config.md) | These separate implementation details, operator behavior, and configuration contracts. | | Output, cache, resume, or debug artifacts | [Run State Internals](internal/state.md), [Operations](operations.md), and [Configuration](config.md) | These separate implementation details, operator behavior, and configuration contracts. |
| External input formats, artifact schemas, or durable output files | [Integration Contracts](integrations/) | Integration documents define external and durable data contracts. | | External input formats, artifact schemas, or durable output files | [Integration Contracts](integrations/) | Integration documents define external and durable data contracts. |
| Release preparation, tagging, publication, or optional diagnostics | [Release Policy](policy/release.md) and [Source Releases](release.md) | Policy owns release requirements and completion; the runbook supplies the publication command and recovery procedure. |
| Proposed or unimplemented behavior | [Roadmap](roadmap/) | Future work belongs only in roadmap documentation until implemented. | | Proposed or unimplemented behavior | [Roadmap](roadmap/) | Future work belongs only in roadmap documentation until implemented. |
For an existing subsystem, also inspect its focused tests and the package-local For an existing subsystem, also inspect its focused tests and the package-local

View File

@@ -55,7 +55,7 @@ record controls eligibility only: its title, summary, and reference do not
become turn evidence. No exact matching scene also produces an empty list and become turn evidence. No exact matching scene also produces an empty list and
the `scene_classification_unavailable` warning. the `scene_classification_unavailable` warning.
An optional normalized [NPC registry artifact](dnd-npc-registry-artifacts.md) can ground an An optional normalized [NPC artifact](dnd-npc-artifacts.md) can ground an
actor name. Its registry references are provenance, never combat evidence. actor name. Its registry references are provenance, never combat evidence.
Normalization trims and, where possible, canonicalizes actor names; orders and Normalization trims and, where possible, canonicalizes actor names; orders and
deduplicates exact source references; orders valid-evidence turns by source deduplicates exact source references; orders valid-evidence turns by source
@@ -63,9 +63,7 @@ chronology; and collapses only duplicates with the same actor identity, turn
kind, and complete valid evidence. It does not infer turns, initiative, or kind, and complete valid evidence. It does not infer turns, initiative, or
actions from registry or scene data. actions from registry or scene data.
The [NPC-occurrence artifact](dnd-npc-occurrence-artifacts.md) records The [NPC-interaction artifact](dnd-npc-interaction-artifacts.md) records
broader NPC occurrences. The [enemy-event artifact](dnd-enemy-event-artifacts.md) broader NPC occurrences. The [JSON output contract](json-output.md) defines
uses combat turns as grounding only; turns do not establish an enemy event or publication, and [D&D module internals](../internal/dnd.md) describes routing
its outcome. The [JSON output contract](json-output.md) defines publication, and validation mechanics.
and [D&D module internals](../internal/dnd.md) describes routing and validation
mechanics.

View File

@@ -1,114 +0,0 @@
# D&D Enemy-Event Artifact
This contract defines the durable, source-grounded enemy-event occurrence list.
It records enemies directly established as opposing the party and explicitly
observed combat outcomes. It is an ordered observation artifact from which a
consumer may derive a ledger; it is not a ledger, encounter roster, or terminal
state model.
## Identity and compatibility
| Property | Value |
| --- | --- |
| Artifact kind | `dnd/enemy-event-list` |
| Schema ID | `notarius.dnd.enemy_events` |
| Schema name | `notarius_dnd_enemy_events_v1` |
| Schema version | `v1` |
| Media type | `application/json` |
`v1` is a strict JSON object with required `events`; the array may be empty.
Event and source-reference objects reject unknown fields. An incompatible shape
change requires a new schema version.
## Wire shape
Every event has these required fields:
| Field | Contract |
| --- | --- |
| `name` | Non-empty display name or directly grounded collective subject label. |
| `kind` | `engaged`, `killed`, `fled`, `captured`, or `incapacitated`. |
| `source_refs` | One or more current-transcript evidence ranges. |
Each source reference has exactly `source_id`, `start_unit_id`, and
`end_unit_id`. It identifies an inclusive current-transcript range; unit IDs
are positive and the start may not follow the end.
```json
{
"events": [
{
"name": "Ashfang",
"kind": "engaged",
"source_refs": [
{"source_id": "session-7", "start_unit_id": 41, "end_unit_id": 42}
]
},
{
"name": "Ashfang",
"kind": "fled",
"source_refs": [
{"source_id": "session-7", "start_unit_id": 57, "end_unit_id": 58}
]
}
]
}
```
## Event semantics and evidence
| Kind | Required evidence |
| --- | --- |
| `engaged` | The subject is directly established as actively opposing the party in combat. At most one engagement is emitted for one subject in one combat scene. |
| `killed` | The transcript explicitly establishes that the subject died or was killed. Damage, defeat, disappearance, or combat ending is insufficient. |
| `fled` | The subject explicitly escapes, retreats, or otherwise leaves combat to avoid continued engagement. Movement or absence from later turns is insufficient. |
| `captured` | The subject is explicitly taken prisoner or secured under the party's control. A grapple or temporary restraint alone is insufficient. |
| `incapacitated` | The subject is explicitly rendered unable to continue acting without being established as killed or captured. A missed turn is insufficient. |
The current transcript is the only event evidence. Campaign context and
normalized NPC, scene-description, combat-turn, and NPC-occurrence artifacts
can ground names or control combat eligibility, but none may supply event
evidence. An outcome may share evidence with an engagement, in which case both
events are retained.
Extraction is limited to chunks with an exact combat-scene classification. An
exact non-combat classification produces an accepted empty list. Missing or
mismatched classification also produces an accepted empty list and a
`scene_classification_unavailable` warning.
## Subjects, normalization, and order
A subject matching the normalized NPC registry uses that registry's canonical
display name. Unmatched hostile creatures, summoned entities, and directly
grounded groups remain valid subjects. An unnamed homogeneous group uses the
narrowest transcript-grounded label, such as `Orcs`, `One orc`, or `Remaining
orcs`; the artifact never invents synthetic member identities or quantities.
Party members, allies, neutral observers, mentioned-but-absent enemies, hazards,
traps, and environmental effects are excluded.
Normalization collapses surrounding and repeated internal whitespace in subject
display values, canonicalizes recognized registry names, canonicalizes and
deduplicates exact source ranges, then orders events by valid evidence
chronology, normalized subject identity, display name, kind, and reference
sequence. The deterministic kind tie order is `engaged`,
`incapacitated`, `captured`, `fled`, then `killed`. Only entries with the same
normalized name, kind, and complete canonical evidence sequence are collapsed.
Different kinds, evidence, repeated engagement in separate scenes, and later
outcomes remain separate. A later engagement for the same named subject is
preserved after an earlier outcome because the artifact does not assert an
irreversible state transition.
## Non-goals
The artifact has no NPC or scene ID, quantity, confidence, description,
rationale, summary, current state, or inferred terminal outcome. It does not
emit `active` or `unresolved`; consumers may derive an unresolved ledger view
only when an engagement has no later explicit outcome. It never infers an
outcome from turn absence, scene termination, initiative order, hit-point
guesses, or other artifacts.
The [JSON output contract](json-output.md) defines publication. Configuration
keys, required generated-reference slots, and validator-chain selection are
defined in the [configuration reference](../config.md). Implementation and
prompt-grounding mechanics are described in the
[D&D module internals](../internal/dnd.md).

View File

@@ -0,0 +1,78 @@
# D&D Item-Event Artifact
This contract defines the durable item and currency occurrence list produced by
`dnd/item-events`. It records source-grounded discoveries and possession
changes; it does not maintain an inventory, balance, or ledger.
## Identity and compatibility
| Property | Value |
| --- | --- |
| Artifact kind | `dnd/item-event-list` |
| Schema ID | `notarius.dnd.item_events` |
| Schema name | `notarius_dnd_item_events_v1` |
| Schema version | `v1` |
| Media type | `application/json` |
`v1` is a strict JSON object with required `events`; the array may be empty.
Event and source-reference objects reject unknown fields. An incompatible
shape change requires a new schema version.
## Wire shape
Every event has required `name`, `kind`, and `source_refs`. `quantity`, `from`,
and `to` are optional where the event kind permits them.
| Field | Contract |
| --- | --- |
| `name` | Non-empty item or currency display name. |
| `kind` | `discovered`, `acquired`, `lost`, `consumed`, or `transferred`. |
| `quantity` | Optional positive integer; omit it when no count is established. |
| `from` | Optional non-empty losing holder, when allowed by `kind`. |
| `to` | Optional non-empty gaining holder, when allowed by `kind`. |
| `source_refs` | One or more transcript evidence ranges. |
Each source reference has exactly `source_id`, `start_unit_id`, and
`end_unit_id`. It identifies an inclusive current-transcript range; unit IDs
are positive and the start may not follow the end.
```json
{
"events": [
{
"name": "Silver Pieces",
"kind": "acquired",
"quantity": 20,
"to": "party",
"source_refs": [
{"source_id": "session-7", "start_unit_id": 2, "end_unit_id": 2}
]
}
]
}
```
## Holder rules and minimal extraction
`discovered` has neither holder; `acquired` requires `to` and forbids `from`;
`lost` and `consumed` require `from` and forbid `to`; `transferred` requires
both holders. `party` denotes collective possession. A transfer cannot use
`party` for either holder and its two normalized holders must differ.
Only an evidenced discovery or possession change belongs in this artifact.
It does not infer quantities or holders, convert currency denominations,
calculate balances, or merge nearby events. Campaign references may
disambiguate names but are never event evidence. Currency uses the ordinary
`name` field and an explicit `quantity` only when the transcript establishes
one; each denomination remains a separate event.
Normalization trims display whitespace, orders and removes exact duplicate
source references, then orders events by valid source chronology, name identity
and display value, kind, holders, quantity, and reference sequence. It
collapses only entries with the same normalized durable fields and complete
valid evidence.
The [JSON output contract](json-output.md) defines publication. See
[D&D module internals](../internal/dnd.md) for implementation details and the
[NPC-interaction artifact](dnd-npc-interaction-artifacts.md) for a distinct
kind of occurrence.

View File

@@ -1,72 +0,0 @@
# D&D Item-Occurrence Artifact
`dnd/item-occurrences` currently produces this source-grounded item and currency
occurrence list. It records discoveries and possession changes, not an
inventory, balance, or ledger.
## Identity and compatibility
| Property | Value |
| --- | --- |
| Artifact kind | `dnd/item-occurrence-list` |
| Schema ID | `notarius.dnd.item_occurrences` |
| Schema name | `notarius_dnd_item_occurrences_v1` |
| Schema version | `v1` |
| Media type | `application/json` |
`v1` accepts one strict JSON object with required `occurrences`; the array may
be empty. Each occurrence has required `item_id`, `name`, `kind`, and
`source_refs`, and occurrence and source-reference objects reject unknown
fields. `quantity`, `from`, and `to` appear only when their kind permits them.
An incompatible shape change requires a new schema version.
## Registry grounding
Both extraction and normalization require an `item_registry` reference bound to
an earlier normalized `dnd/item-registry` artifact. The registry is immutable
for an operation and contributes names-only grounding after the shared evidence
message. Notarius resolves the model's selected name into the unchanged exact
durable ID/name pair. It is never occurrence evidence.
Each occurrence must use one exact registry ID/name pair. An extraction response
with an unknown or ambiguous selected name is rejected as invalid model output;
the configured pipeline may retry it and never accepts a partial artifact.
Normalization and validation remain defense in depth for artifacts entering
through other boundaries: normalization canonicalizes a recognized name by ID,
preserves unknown values for the registry validator, and the registry validator
rejects unknown or mismatched pairs.
## Wire shape
Each source reference has exactly `source_id`, `start_unit_id`, and
`end_unit_id`. It identifies an inclusive range in the current transcript;
unit IDs are positive and the start may not follow the end.
```json
{
"occurrences": [
{
"item_id": "item:sha256:…",
"name": "Silver Pieces",
"kind": "acquired",
"quantity": 20,
"to": "party",
"source_refs": [
{"source_id": "session-7", "start_unit_id": 2, "end_unit_id": 2}
]
}
]
}
```
The five kinds remain `discovered`, `acquired`, `lost`, `consumed`, and
`transferred`. Holder, quantity, currency, ordering, and exact-duplicate rules
are unchanged: discovered has no holder; acquired requires `to`; lost and
consumed require `from`; transferred requires distinct non-`party` holders.
The only current downstream compatibility requirement is its registry handoff;
the normalized occurrence list is otherwise published for callers. See
[Configuration](../config.md#d-d-reference-slots) for the binding and
[JSON output](json-output.md) for publication.
See [item registry](dnd-item-registry-artifacts.md) for the grounding artifact
and [D&D module internals](../internal/dnd.md) for implementation details.

View File

@@ -1,95 +0,0 @@
# D&D Item Registry Artifact
This contract defines the durable, source-grounded item registry produced by
`dnd/item-registry`. It records transcript-established item types and unique
designations for one source document; it is not an inventory, holder record,
quantity ledger, or item-occurrence artifact.
## Identity and compatibility
| Property | Value |
| --- | --- |
| Artifact kind | `dnd/item-registry` |
| Schema ID | `notarius.dnd.item_registry` |
| Schema name | `notarius_dnd_item_registry_v1` |
| Schema version | `v1` |
| Media type | `application/json` |
| Identity policy | `dnd.item_registry.identity.v1` |
`v1` accepts one strict JSON object with required `items`; the array may be
empty. Item and source-reference objects reject unknown fields. An incompatible
artifact shape or identity-policy change uses a new version or policy.
## Wire shape and identity
Each item has these required fields:
| Field | Contract |
| --- | --- |
| `id` | `item:sha256:` followed by 64 lowercase hexadecimal characters. |
| `name` | Non-empty transcript-established item type or unique designation. |
| `source_refs` | One or more transcript evidence ranges that establish the item. |
A source reference has exactly `source_id`, `start_unit_id`, and `end_unit_id`.
The source ID identifies the transcript, unit IDs are positive inclusive unit
identifiers, and the start may not follow the end.
```json
{
"items": [
{
"id": "item:sha256:31e73b6280ef98e4d8070e07fd4de9b2c3e842cc03af1a09ca631cb95b73e3b3",
"name": "Star Compass",
"source_refs": [
{"source_id": "session-7", "start_unit_id": 4, "end_unit_id": 5}
]
}
]
}
```
The ID is deterministic for an item name or type, rather than for one physical
instance. Notarius normalizes the display name for comparison with Unicode
NFKC, supported apostrophe normalization, collapsed whitespace, and case
folding. It hashes compact JSON for this array:
```text
["dnd.item_registry.identity.v1", comparison_name]
```
The canonical ID is the lowercase SHA-256 digest of those bytes with the
`item:sha256:` prefix. Equal comparison names represent one item identity;
normalization unions their transcript evidence when it safely consolidates a
candidate group.
## Scope, reconciliation, and evidence
The registry includes named unique items, concrete reusable item types, stable
unique designations, and separately established currency denominations. It
excludes vague loot or treasure, generic weapons, quantities, inferred
properties, and inferred uniqueness. Capitalization alone does not establish
eligibility.
Normalization first applies deterministic display, evidence, and ID rules. It
then may use a bounded LLM-assisted proposal to reconcile semantically duplicate
records. The proposal may choose only a supplied candidate display name;
invalid, uncertain, overlapping, or unsafe proposals retain the deterministic
result with retry or fallback diagnostics. A proposal that mixes a recognized
currency denomination with a non-currency item, or combines recognized
denominations, is unsafe and retains every deterministic record. Currency
denominations, materially different item types, and merely nearby objects
remain distinct. Source references establish registry provenance, not evidence
for later artifacts.
## Consumers and publication
`dnd/item-occurrences` requires one approved item registry through its
`item_registry` reference slot for both extraction and normalization. Its
consumer receives names-only grounding; Notarius resolves the selected name
into the unchanged exact durable ID/name pair. The registry’s source references
are never occurrence evidence. Unknown or ambiguous selections are rejected by
the occurrence contract. See the
[item-occurrence artifact](dnd-item-occurrence-artifacts.md) for that strict
wire contract, [Configuration](../config.md#d-d-reference-slots) for binding
rules and validator selection, and the [JSON output contract](json-output.md)
for publication.

View File

@@ -1,86 +0,0 @@
# D&D Location-Occurrence Artifact
This contract defines the durable occurrence list produced by
`dnd/location-occurrences`. It records source-grounded ways the party relates
to locations in a required normalized location registry; it does not extend
that registry or infer a place absent from it.
## Identity and compatibility
| Property | Value |
| --- | --- |
| Artifact kind | `dnd/location-occurrence-list` |
| Schema ID | `notarius.dnd.location_occurrences` |
| Schema name | `notarius_dnd_location_occurrences_v1` |
| Schema version | `v1` |
| Media type | `application/json` |
`v1` accepts one strict JSON object with required `occurrences`; the array may
be empty. Occurrence and source-reference objects reject unknown fields. An
incompatible shape change requires a new schema version.
## Wire shape
Each occurrence has these required fields:
| Field | Contract |
| --- | --- |
| `location_id` | Exact ID from the required normalized [location registry](dnd-location-registry-artifacts.md). |
| `name` | Exact canonical display name for `location_id` in that registry. |
| `kind` | One of `visited`, `planned`, `recalled`, or `mentioned`. |
| `source_refs` | One or more current-transcript evidence ranges for this occurrence. |
A source reference has exactly `source_id`, `start_unit_id`, and `end_unit_id`.
It identifies an inclusive range in the current transcript; unit IDs are
positive and the start may not follow the end.
```json
{
"occurrences": [
{
"location_id": "location:sha256:fb05475da0fc7debf994b517e1906ffe7209887a6a1ec306356d84de820b1a24",
"name": "Moon Gate",
"kind": "visited",
"source_refs": [
{"source_id": "session-7", "start_unit_id": 12, "end_unit_id": 13}
]
}
]
}
```
## Occurrence categories
| Kind | Meaning |
| --- | --- |
| `visited` | The transcript establishes physical party presence, including arrival, continuing presence, or departure. |
| `planned` | The party explicitly proposes, intends, or agrees to future travel; speculation alone is not enough. |
| `recalled` | The transcript explicitly recounts prior party presence before the current live events. |
| `mentioned` | The location is explicit but no stronger category applies, including lore, directions, third-party activity, non-actionable speculation, a mere hypothetical reference, or out-of-character discussion. |
For overlapping evidence, precedence is `visited`, then `planned`, then
`recalled`, then `mentioned`. For example, “What if we went to Moon Gate?” is
eligible as `mentioned` when its narrow evidence explicitly references that
registry location, but it is not `planned` without an actual proposal,
intention, or agreement to travel. Inferred, unstated, uncertain, and
unsupported places or occurrences are omitted. Normalization
canonicalizes the registry name, orders and deduplicates source references, and
orders occurrences by source chronology, location ID, name, kind, and reference
sequence. It collapses only exact duplicates with the same ID, kind, and
complete canonical evidence sequence.
## Required grounding and evidence
Both extraction and normalization require exactly one `location_registry` reference of
kind `dnd/location-registry`, media type `application/json`, and at most 1 MiB. The
registry provides identity grounding only. The model selects a supplied
contextual name-and-registry-reference descriptor, and Notarius resolves it
into the exact durable ID/name pair. Unknown, partial, or ambiguous selections
are rejected rather than guessed or reassigned. The current transcript is the
only evidence source for an occurrence; registry evidence and provenance never
become occurrence evidence.
See [Configuration](../config.md#d-d-reference-slots) for the selectable slot
and generated-handoff compatibility, [D&D module internals](../internal/dnd.md)
for implementation behavior, and the [JSON output contract](json-output.md)
for publication.

View File

@@ -1,93 +0,0 @@
# D&D Location Registry Artifact
This contract defines the durable, source-grounded location registry produced
by `dnd/location-registry`. It records transcript-established physical places for one
source document; it is not a map, location hierarchy, campaign-wide world
registry, or location description.
## Identity and compatibility
| Property | Value |
| --- | --- |
| Artifact kind | `dnd/location-registry` |
| Schema ID | `notarius.dnd.location_registry` |
| Schema name | `notarius_dnd_location_registry_v1` |
| Schema version | `v1` |
| Media type | `application/json` |
| Identity policy | `dnd.location_registry.identity.v1` |
`v1` accepts one strict JSON object with required `locations`; the array may be
empty. Location and source-reference objects reject unknown fields. An
incompatible artifact shape or identity-policy change uses a new version or
policy.
## Wire shape and identity
Each location has these required fields:
| Field | Contract |
| --- | --- |
| `id` | `location:sha256:` followed by 64 lowercase hexadecimal characters. |
| `name` | Non-empty transcript-established display name. |
| `source_refs` | One or more transcript evidence ranges that identify the place. |
A source reference has exactly `source_id`, `start_unit_id`, and `end_unit_id`.
The source ID identifies the transcript, unit IDs are positive inclusive unit
identifiers, and the start may not follow the end.
```json
{
"locations": [
{
"id": "location:sha256:fb05475da0fc7debf994b517e1906ffe7209887a6a1ec306356d84de820b1a24",
"name": "Moon Gate",
"source_refs": [
{"source_id": "session-7", "start_unit_id": 4, "end_unit_id": 5}
]
}
]
}
```
The ID is deterministic and scoped to the source document. Notarius normalizes
the display name for comparison with Unicode NFKC, supported apostrophe
normalization, collapsed whitespace, and case folding. It hashes compact JSON
for this array, using the earliest canonical source reference as the anchor:
```text
["dnd.location_registry.identity.v1", comparison_name, source_id, start_unit_id, end_unit_id]
```
The canonical ID is the lowercase SHA-256 digest of those bytes with the
`location:sha256:` prefix. Equal display names are allowed when their evidence
anchors differ, so a generic name does not force distinct places to collapse.
## Scope, reconciliation, and evidence
Locations are physical or spatial places established by the transcript with a
stable proper name or unique in-world designation, such as named planes,
regions, settlements, districts, buildings, rooms, landmarks, routes, and
geographic features. Generic, temporary, relative, and descriptive phrases
such as “the room,” “the bar,” “the hallway,” “outside,” and “upstairs” are not
registry locations. Capitalization alone does not establish eligibility.
Notarius does not infer an unstated place or add hierarchy, coordinates,
descriptions, participants, or ownership.
Normalization first applies deterministic display, evidence, and ID rules. It
then may use a bounded LLM-assisted proposal to reconcile semantically duplicate
records. The proposal is validated and applied conservatively; invalid or
unusable proposals retain the deterministic result with retry or fallback
diagnostics. The registry's source references establish registry provenance,
not evidence for later artifacts.
## Consumers and publication
`dnd/location-occurrences` requires one approved location registry through its
`location_registry` reference slot. Its prompt receives contextual selectors
containing a canonical name and registry references; Notarius resolves a
selection into the unchanged exact durable ID/name pair. Registry references
must not be treated as occurrence evidence. See the
[location-occurrence artifact](dnd-location-occurrence-artifacts.md)
for that contract, [Configuration](../config.md#references-and-ordered-handoffs)
for binding rules, and the [JSON output contract](json-output.md) for
publication.

View File

@@ -0,0 +1,69 @@
# D&D NPC Artifact
This contract defines the durable NPC registry produced by `dnd/npcs`. It is a
minimal, source-grounded identity registry for other D&D artifacts, not a
character sheet or a relationship summary.
## Identity and compatibility
| Property | Value |
| --- | --- |
| Artifact kind | `dnd/npc-list` |
| Schema ID | `notarius.dnd.npcs` |
| Schema name | `notarius_dnd_npcs_v1` |
| Schema version | `v1` |
| Media type | `application/json` |
| Identity policy | `dnd.npcs.identity.v1` |
`v1` accepts one strict JSON object with required `npcs`; the array may be
empty. NPC and source-reference objects reject unknown fields. An incompatible
artifact shape or identity-policy change uses a new version or policy.
## Wire shape and identity
Each NPC has these required fields:
| Field | Contract |
| --- | --- |
| `id` | `npc:sha256:` followed by 64 lowercase hexadecimal characters. |
| `name` | Non-empty canonical display name. |
| `source_refs` | One or more transcript evidence ranges for the identity. |
A source reference has exactly `source_id`, `start_unit_id`, and `end_unit_id`.
The source ID identifies the transcript, unit IDs are positive inclusive unit
identifiers, and the start may not follow the end.
```json
{
"npcs": [
{
"id": "npc:sha256:99a16589618a04f535a7d21fdcc71a0b1c05d22f752cd492065b1086d97bc3d7",
"name": "Mira Thorn",
"source_refs": [
{"source_id": "session-7", "start_unit_id": 4, "end_unit_id": 5}
]
}
]
}
```
The ID is deterministic: normalize the name to Unicode NFKC, normalize the
supported apostrophe forms, collapse whitespace, case-fold it, SHA-256 the
result, then prefix the lowercase hexadecimal digest with `npc:sha256:`. Each
canonical identity and ID appears at most once. Normalization collapses records
with the same canonical identity, retains their earliest position, and merges
their canonicalized evidence; it does not add aliases, roles, descriptions, or
relationship fields.
## Scope and consumers
Only individually identifiable NPC names with transcript evidence belong in
this artifact. Groups, generic roles, invented labels, and descriptive
enrichment are excluded. Its source references prove registry provenance; they
do not become evidence for a spell, interaction, or combat occurrence.
This registry can ground actor or caster names in the [spell](dnd-spell-artifacts.md)
and [combat-turn](dnd-combat-turn-artifacts.md) artifacts. It is required to
resolve the canonical `name` in an [NPC interaction](dnd-npc-interaction-artifacts.md).
The [JSON output contract](json-output.md) defines publication, and
[D&D module internals](../internal/dnd.md) owns pipeline mechanics.

View File

@@ -1,7 +1,7 @@
# D&D NPC Occurrence Artifact # D&D NPC Interaction Artifact
This contract defines the durable occurrence list produced by This contract defines the durable occurrence list produced by
`dnd/npc-occurrences`. It records discrete, source-grounded occurrences with `dnd/npc-interactions`. It records discrete, source-grounded interactions with
NPCs already present in a normalized registry; it does not extend that registry NPCs already present in a normalized registry; it does not extend that registry
or summarize the session. or summarize the session.
@@ -9,37 +9,35 @@ or summarize the session.
| Property | Value | | Property | Value |
| --- | --- | | --- | --- |
| Artifact kind | `dnd/npc-occurrence-list` | | Artifact kind | `dnd/npc-interaction-list` |
| Schema ID | `notarius.dnd.npc_occurrences` | | Schema ID | `notarius.dnd.npc_interactions` |
| Schema name | `notarius_dnd_npc_occurrences_v1` | | Schema name | `notarius_dnd_npc_interactions_v1` |
| Schema version | `v1` | | Schema version | `v1` |
| Media type | `application/json` | | Media type | `application/json` |
`v1` is a strict JSON object with required `occurrences`; the array may be `v1` is a strict JSON object with required `interactions`; the array may be
empty. Occurrence and source-reference objects reject unknown fields. An empty. Interaction and source-reference objects reject unknown fields. An
incompatible shape change requires a new schema version. incompatible shape change requires a new schema version.
## Wire shape ## Wire shape
Each occurrence has these required fields: Each interaction has these required fields:
| Field | Contract | | Field | Contract |
| --- | --- | | --- | --- |
| `npc_id` | Exact durable ID from the required NPC registry. |
| `name` | Non-empty canonical display name from the required NPC registry. | | `name` | Non-empty canonical display name from the required NPC registry. |
| `kind` | One of the occurrence categories below. | | `kind` | One of the interaction categories below. |
| `source_refs` | One or more transcript evidence ranges. | | `source_refs` | One or more transcript evidence ranges. |
Each source reference has exactly `source_id`, `start_unit_id`, and Each source reference has exactly `source_id`, `start_unit_id`, and
`end_unit_id`. It identifies an inclusive range in the current transcript; `end_unit_id`. It identifies an inclusive range in the current transcript;
unit IDs are positive and the start may not follow the end. Extraction evidence unit IDs are positive and the start may not follow the end. Extraction evidence
for an occurrence is confined to its accepted chunk. for an interaction is confined to its accepted chunk.
```json ```json
{ {
"occurrences": [ "interactions": [
{ {
"npc_id": "npc:sha256:example",
"name": "Mira Thorn", "name": "Mira Thorn",
"kind": "dialogue", "kind": "dialogue",
"source_refs": [ "source_refs": [
@@ -50,7 +48,7 @@ for an occurrence is confined to its accepted chunk.
} }
``` ```
## Occurrence categories ## Interaction categories
| Kind | Meaning | | Kind | Meaning |
| --- | --- | | --- | --- |
@@ -67,24 +65,14 @@ for uncertain classification.
## Identity, evidence, and order ## Identity, evidence, and order
The required normalized [NPC registry artifact](dnd-npc-registry-artifacts.md) The required normalized [NPC artifact](dnd-npc-artifacts.md) resolves `name`.
supplies names-only contextual grounding to the model. Notarius resolves the Registry references are provenance only and never replace an interaction's own
selected name and writes the exact `{npc_id, name}` pair. An unknown or evidence. Normalization canonicalizes recognized registry names, orders and
ambiguous selection rejects the complete model result; normalization does not deduplicates exact source references, then orders interactions by valid source
repair names by similarity. Registry references are provenance only and never
replace an occurrence's own evidence.
The registry may include an identity established by a factual third-party
mention; that provenance alone does not create a `mentioned` occurrence. Each
occurrence remains a separately cited fact in the current transcript.
Normalization validates the exact pair, orders and
deduplicates exact source references, then orders occurrences by valid source
chronology, NPC comparison identity, display name, kind, and reference sequence. chronology, NPC comparison identity, display name, kind, and reference sequence.
Only entries with the same NPC ID, canonical name, kind, and complete valid evidence Only entries with the same canonical name, kind, and complete valid evidence
sequence are collapsed; distinct categories or evidence remain separate. sequence are collapsed; distinct categories or evidence remain separate.
See the [combat-turn artifact](dnd-combat-turn-artifacts.md) for combat-action See the [combat-turn artifact](dnd-combat-turn-artifacts.md) for combat-action
occurrences. The [enemy-event artifact](dnd-enemy-event-artifacts.md) consumes occurrences and the [JSON output contract](json-output.md) for publication.
only `combat_opponent` occurrences as grounding; they never establish an enemy Pipeline mechanics are described in [D&D module internals](../internal/dnd.md).
event or outcome. The [JSON output contract](json-output.md) defines
publication. Pipeline mechanics are described in
[D&D module internals](../internal/dnd.md).

View File

@@ -1,92 +0,0 @@
# D&D NPC Registry Artifact
This contract defines the durable NPC registry produced by `dnd/npc-registry`. It is a
minimal, source-grounded identity registry for other D&D artifacts, not a
character sheet or a relationship summary.
## Identity and compatibility
| Property | Value |
| --- | --- |
| Artifact kind | `dnd/npc-registry` |
| Schema ID | `notarius.dnd.npc_registry` |
| Schema name | `notarius_dnd_npc_registry_v1` |
| Schema version | `v1` |
| Media type | `application/json` |
| Identity policy | `dnd.npc_registry.identity.v1` |
`v1` accepts one strict JSON object with required `npcs`; the array may be
empty. NPC and source-reference objects reject unknown fields. An incompatible
artifact shape or identity-policy change uses a new version or policy.
## Wire shape and identity
Each NPC has these required fields:
| Field | Contract |
| --- | --- |
| `id` | `npc:sha256:` followed by 64 lowercase hexadecimal characters. |
| `name` | Non-empty canonical display name. |
| `source_refs` | One or more transcript evidence ranges for the identity. |
A source reference has exactly `source_id`, `start_unit_id`, and `end_unit_id`.
The source ID identifies the transcript, unit IDs are positive inclusive unit
identifiers, and the start may not follow the end.
```json
{
"npcs": [
{
"id": "npc:sha256:35ba5f679aee69e07ae3bd65c44278f29539d5dc9bb5225db1c0060555b23221",
"name": "Mira Thorn",
"source_refs": [
{"source_id": "session-7", "start_unit_id": 4, "end_unit_id": 5}
]
}
]
}
```
The ID is deterministic: normalize the name to Unicode NFKC, normalize the
supported apostrophe forms, collapse whitespace, case-fold it, then serialize
`["dnd.npc_registry.identity.v1", comparison_name]` as compact JSON. SHA-256
those UTF-8 bytes and prefix the lowercase hexadecimal digest with
`npc:sha256:`. Each canonical identity and ID appears at most once.
Normalization collapses records with the same canonical identity, retains their
earliest position, and merges
their canonicalized evidence; it does not add aliases, roles, descriptions, or
relationship fields.
When evidence supports a semantically duplicate group, the canonical display
name is one of that group's supplied candidates. A complete, stable proper name
is preferred over an abbreviation. An unadorned proper name is preferred over
the same name plus a contextual class, role, title, or relationship descriptor
unless the transcript establishes that descriptor as part of the person's
name. A longer candidate is not preferred solely because it includes such a
descriptor.
## Scope and consumers
Only individually identifiable NPC names with transcript evidence belong in
this artifact. A factual third-party mention can establish an identity even if
the NPC is not present, speaking, or acting in the cited passage. Names used
only in hypothetical, speculative, or imagined examples are excluded, as are
groups, generic roles, invented labels, and descriptive enrichment. Its source
references prove registry provenance; they do not become evidence for a spell,
occurrence, combat, or enemy-event occurrence.
Registry evidence establishes an identity, not an [NPC occurrence](dnd-npc-occurrence-artifacts.md).
That later artifact independently records any current-transcript occurrence
with its own cited evidence and category.
This registry can ground actor or caster names in the [spell](dnd-spell-artifacts.md)
and [combat-turn](dnd-combat-turn-artifacts.md) artifacts. It is required to
resolve the canonical `name` in an [NPC occurrence](dnd-npc-occurrence-artifacts.md).
Occurrence consumers receive names-only grounding; Notarius resolves the
selected canonical name and writes the unchanged exact durable ID/name pair.
Spells, combat turns, and the [enemy-event artifact](dnd-enemy-event-artifacts.md)
also receive names-only grounding for actor or subject display. None of these
projections supply later-artifact evidence. [Configuration](../config.md#d-d-reference-slots)
owns the `npc_registry` binding rules.
The [JSON output contract](json-output.md) defines publication, and
[D&D module internals](../internal/dnd.md) owns pipeline mechanics.

View File

@@ -62,9 +62,8 @@ durable fields, or the same source range with different kind, title, or
summary, is invalid. It does not merge adjacent ranges, alter prose, or infer summary, is invalid. It does not merge adjacent ranges, alter prose, or infer
missing scenes. missing scenes.
The [combat-turn artifact](dnd-combat-turn-artifacts.md) and The [combat-turn artifact](dnd-combat-turn-artifacts.md) uses an exact matching
[enemy-event artifact](dnd-enemy-event-artifacts.md) use an exact matching
`combat` scene only as eligibility control; scene title, summary, and source `combat` scene only as eligibility control; scene title, summary, and source
reference never become their evidence. Publication is defined by the reference never become combat evidence. Publication is defined by the
[JSON output contract](json-output.md); implementation details live in [JSON output contract](json-output.md); implementation details live in
[D&D module internals](../internal/dnd.md). [D&D module internals](../internal/dnd.md).

View File

@@ -61,7 +61,7 @@ only when it has the same canonical spell, the same case- and
whitespace-insensitive caster identity, and the same complete valid reference whitespace-insensitive caster identity, and the same complete valid reference
sequence. Remaining entries retain their merged order. sequence. Remaining entries retain their merged order.
The optional normalized [NPC registry artifact](dnd-npc-registry-artifacts.md) can ground a The optional normalized [NPC artifact](dnd-npc-artifacts.md) can ground a
caster name. Its own references remain registry provenance and are never copied caster name. Its own references remain registry provenance and are never copied
into `source_refs`. into `source_refs`.

View File

@@ -67,12 +67,6 @@ including a collision with the embedded catalog. Matching uses the catalog’s
case, whitespace, and apostrophe normalization, so authors should avoid names case, whitespace, and apostrophe normalization, so authors should avoid names
or aliases that normalize to another spell. or aliases that normalize to another spell.
Spell extraction receives the effective catalog as deterministic canonical-name
and alias pairs. An alias in the transcript selects its associated canonical
name; the extractor is instructed to return that canonical spelling. The
projection contains no catalog source metadata or provenance, and aliases
remain recognition context rather than transcript evidence.
The overlay is a recognition aid only. The durable spell-artifact schema and The overlay is a recognition aid only. The durable spell-artifact schema and
source-evidence rules are defined by the source-evidence rules are defined by the
[D&D spell artifact contract](dnd-spell-artifacts.md). [D&D spell artifact contract](dnd-spell-artifacts.md).

View File

@@ -1,11 +1,9 @@
# Published Evidence Context # Published Evidence Context
This contract defines the optional `source/evidence-context` artifact emitted This contract defines the optional `source/evidence-context` artifact emitted
by the production JSON output. It is a selected source-unit excerpt for by the production JSON output. Its configuration is owned by
convenient reading alongside normalized lane artifacts; it is not a second [Configuration](../config.md#module-bindings-and-validators); its logical-file
citation or provenance model. Its configuration is owned by discovery is owned by [Published JSON Output](json-output.md).
[Configuration](../config.md#module-bindings-and-validators), and its
logical-file discovery is owned by [Published JSON Output](json-output.md).
## Identity And Discovery ## Identity And Discovery
@@ -28,80 +26,91 @@ its absence means evidence publication was not enabled for that bundle.
## Payload ## Payload
The v1 payload is a top-level JSON array of generic source units. There is no The v1 payload is a JSON object with required `source_id`, `source_digest`,
wrapper, source-level metadata, context grouping, lane identifier, or evidence `window_units`, `selected_lanes`, and `contexts` fields. `selected_lanes` and
reference in the payload. An enabled configuration with no contributing `contexts` are always arrays; an enabled configuration with no accepted direct
accepted evidence publishes `[]`. evidence publishes `contexts: []`.
```json ```json
[ {
{ "source_id": "session-alpha",
"id": 10, "source_digest": "sha256:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef",
"kind": "transcript_segment", "window_units": 1,
"text": "Aria casts Cure Wounds.", "selected_lanes": ["npcs", "spells"],
"ref": { "contexts": [
"source_id": "session-alpha", {
"start_unit_id": 10, "context_ref": {
"end_unit_id": 10 "source_id": "session-alpha",
"start_unit_id": 10,
"end_unit_id": 20
},
"evidence_refs": [
{
"lane_id": "spells",
"source_ref": {
"source_id": "session-alpha",
"start_unit_id": 10,
"end_unit_id": 10
}
}
],
"units": [
{
"id": 10,
"kind": "transcript_segment",
"text": "Aria casts Cure Wounds.",
"ref": {
"source_id": "session-alpha",
"start_unit_id": 10,
"end_unit_id": 10
}
},
{
"id": 20,
"kind": "transcript_segment",
"text": "The party regroups.",
"ref": {
"source_id": "session-alpha",
"start_unit_id": 20,
"end_unit_id": 20
}
}
]
} }
}, ]
{ }
"id": 20,
"kind": "transcript_segment",
"text": "The party regroups.",
"ref": {
"source_id": "session-alpha",
"start_unit_id": 20,
"end_unit_id": 20
}
}
]
``` ```
Each source unit has required `id`, `kind`, `text`, and self `ref` fields. Each context requires `context_ref`, `evidence_refs`, and `units` arrays.
`ref` contains `source_id`, `start_unit_id`, and `end_unit_id`, and both unit `context_ref` identifies the first and last included unit. Each evidence entry
endpoints identify that unit's `id`. A unit may also contain source-owned contains a selected `lane_id` and an original `source_ref`. A unit uses the
`metadata`, an open-ended JSON object. Fixed unit and reference fields are existing source-unit shape: required `id`, `kind`, `text`, and self `ref`, plus
strict: consumers must reject unknown fixed fields, malformed units, invalid optional JSON-object `metadata`. Fixed payload objects reject unknown fields;
self-references, units whose `source_id` differs from other units in the same unit metadata may contain application-defined JSON values.
excerpt, and a payload that is not the array described here.
The excerpt preserves each selected unit exactly as represented by the ## Citations And Context
validated generic source document. It does not add evidence-context-specific
annotations or reshape source-owned metadata.
## Selection And Citations `evidence_refs` are the authoritative citations. They identify the direct
references emitted by accepted normalized artifacts. `context_ref` and the
units collection include those cited units plus nearby source units selected by
the configured window. They are explanatory context, not widened citations.
The framework obtains direct source references only through typed evidence Only accepted outputs from the configured lane allowlist contribute. Rejected,
projections of accepted normalized artifacts in the configured lane allowlist. failed, absent, and lane-filtered outputs do not contribute. The artifact never
It validates each reference against the current source document, expands its contains raw input bytes, prompts, model responses, auxiliary reference
range by `window_units` source-unit positions on each side, clamps at document content, credentials, or filesystem paths.
boundaries, and takes the union of all expanded ranges. The output contains
each selected source unit once in source-document position order, regardless
of numeric unit IDs. Repeated references, overlapping windows, and citations
from multiple lanes do not duplicate a unit. Rejected, failed, absent,
inactive, and unselected lanes contribute nothing.
Normalized lane artifacts remain authoritative for citations and for which lane ## Ordering And Compatibility
cited a range. The excerpt has no lane attribution and must not be used to
reconstruct it. Its included nearby units provide reading context only; they
do not widen any citation in a lane artifact.
The excerpt contains at most every generic source unit once. It can therefore The selected lane allowlist is lexical. Contexts and units are in source
equal the complete generic source document when coverage is broad or the document position order, not numeric unit-ID order. Direct evidence entries
window is large. No byte-, token-, or compression-size guarantee is made, and are deterministically ordered by lane and source reference. Overlapping or
the framework does not truncate the excerpt to meet an arbitrary size limit. contiguous windows merge, and each source unit appears at most once in the
resulting contexts.
## Consumer Responsibilities And Data Handling
The artifact is additive to the JSON bundle and is not a lane payload, The artifact is additive to the JSON bundle and is not a lane payload,
normalized-output count, checkpoint, or generated reference. Consumers that normalized-output count, checkpoint, or generated reference. Consumers that
do not need it must tolerate an absent descriptor. Consumers that do use it do not need it must tolerate the absent optional descriptor. Consumers that do
should validate the descriptor and payload before use, retain the artifact with use it should preserve the artifact and its schema identity with the run
its schema identity when needed for a run record, and read citations from the provenance, and should treat its source text and metadata as sensitive durable
corresponding normalized lane artifacts. content.
The excerpt contains source-unit text and source-owned metadata and is durable
output. Treat it as sensitive source content, apply appropriate access controls
and retention, and do not assume its selected form is materially smaller or
less sensitive than the original input.

View File

@@ -9,7 +9,7 @@ Output configuration, including chunk-map and evidence-context publication, belo
## Bundle Layout ## Bundle Layout
All paths below are logical, relative, slash-separated bundle paths. The All paths below are logical, relative, slash-separated bundle paths. The
encoder always emits the first five JSON files below and adds lane or encoder always emits the first four JSON files below and adds lane or
pipeline-wide artifact files when their corresponding artifacts are available: pipeline-wide artifact files when their corresponding artifacts are available:
A subprocess caller first obtains the physical bundle root from the A subprocess caller first obtains the physical bundle root from the
@@ -21,11 +21,10 @@ root for the logical discovery described here.
| `index.json` | Entry point that names the other published files and lane payloads. | | `index.json` | Entry point that names the other published files and lane payloads. |
| `manifest.json` | Run provenance and result summaries. | | `manifest.json` | Run provenance and result summaries. |
| `rejected.json` | Rejected pipeline outputs. | | `rejected.json` | Rejected pipeline outputs. |
| `warnings.json` | Actionable process-degradation warnings. | | `warnings.json` | Accepted-output and run warnings. |
| `diagnostics.json` | Accepted-artifact quality advisories and normalization observations. |
| `lanes/<safe-lane-id>.json` | One normalized artifact payload for each lane. | | `lanes/<safe-lane-id>.json` | One normalized artifact payload for each lane. |
| `chunk-map.json` | Optional accepted chunk map, when its export is enabled and available. | | `chunk-map.json` | Optional accepted chunk map, when its export is enabled and available. |
| `evidence-context.json` | Optional selected source-unit excerpt, when evidence publication is enabled. | | `evidence-context.json` | Optional source-context artifact, when evidence publication is enabled. |
JSON files are pretty-printed with a trailing newline. Lane payloads are JSON files are pretty-printed with a trailing newline. Lane payloads are
accepted only when their media type is `application/json`. accepted only when their media type is `application/json`.
@@ -40,8 +39,7 @@ normalized lanes has this valid minimal index:
"manifest_file": "manifest.json", "manifest_file": "manifest.json",
"output_files": [], "output_files": [],
"rejected_file": "rejected.json", "rejected_file": "rejected.json",
"warnings_file": "warnings.json", "warnings_file": "warnings.json"
"diagnostics_file": "diagnostics.json"
} }
``` ```
@@ -51,7 +49,6 @@ normalized lanes has this valid minimal index:
| `output_files` | Yes | Lane descriptors sorted by `lane_id`. | | `output_files` | Yes | Lane descriptors sorted by `lane_id`. |
| `rejected_file` | Yes | Always `rejected.json`. | | `rejected_file` | Yes | Always `rejected.json`. |
| `warnings_file` | Yes | Always `warnings.json`. | | `warnings_file` | Yes | Always `warnings.json`. |
| `diagnostics_file` | Yes | Always `diagnostics.json`. |
| `chunk_map` | No | Descriptor for the pipeline-wide `chunk-map.json`; never a lane descriptor. | | `chunk_map` | No | Descriptor for the pipeline-wide `chunk-map.json`; never a lane descriptor. |
| `evidence_context` | No | Descriptor for the pipeline-wide `evidence-context.json`; never a lane descriptor. | | `evidence_context` | No | Descriptor for the pipeline-wide `evidence-context.json`; never a lane descriptor. |
@@ -74,15 +71,11 @@ output encoding fail.
Each `lanes/<safe-lane-id>.json` file is the codec-owned normalized JSON for Each `lanes/<safe-lane-id>.json` file is the codec-owned normalized JSON for
that lane. Consumers should use the index descriptor’s schema identity rather that lane. Consumers should use the index descriptor’s schema identity rather
than infer a lane schema from its name. The current D&D payload contracts are than infer a lane schema from its name. The current D&D payload contracts are
[spells](dnd-spell-artifacts.md), [NPC registry](dnd-npc-registry-artifacts.md), [spells](dnd-spell-artifacts.md), [NPCs](dnd-npc-artifacts.md),
[NPC occurrences](dnd-npc-occurrence-artifacts.md), [NPC interactions](dnd-npc-interaction-artifacts.md),
[combat turns](dnd-combat-turn-artifacts.md), [combat turns](dnd-combat-turn-artifacts.md),
[item registry](dnd-item-registry-artifacts.md), [item events](dnd-item-event-artifacts.md), and
[item occurrences](dnd-item-occurrence-artifacts.md), [scene descriptions](dnd-scene-description-artifacts.md).
[scene descriptions](dnd-scene-description-artifacts.md),
[enemy events](dnd-enemy-event-artifacts.md),
[location registry](dnd-location-registry-artifacts.md), and
[location occurrences](dnd-location-occurrence-artifacts.md).
## `manifest.json` ## `manifest.json`
@@ -95,7 +88,7 @@ group into the following externally observable summaries:
| Run identity and result | `run_id`, `pipeline_id`, `pipeline_digest`, `schema_version`, `validation_status`, `started_at`, `completed_at` | | Run identity and result | `run_id`, `pipeline_id`, `pipeline_digest`, `schema_version`, `validation_status`, `started_at`, `completed_at` |
| Resolved components | `input_module`, `chunker`, `extractors`, `merger`, `normalizer`, `output_encoder`, `artifact_lanes`, `validator_chains`, `module_metadata` | | Resolved components | `input_module`, `chunker`, `extractors`, `merger`, `normalizer`, `output_encoder`, `artifact_lanes`, `validator_chains`, `module_metadata` |
| Source and references | `source_digests`, `references` | | Source and references | `source_digests`, `references` |
| Published result summaries | `normalized_outputs`, `rejected_outputs`, `validation_summaries` | | Published result summaries | `normalized_outputs`, `rejected_outputs` |
| Execution summaries | `chunk_plan`, `checkpoint_decisions`, `llm_profiles`, `metadata` | | Execution summaries | `chunk_plan`, `checkpoint_decisions`, `llm_profiles`, `metadata` |
`references` records provenance such as the target, slot, origin, digest, `references` records provenance such as the target, slot, origin, digest,
@@ -105,87 +98,16 @@ summarize results without embedding lane payload bytes. A chunk-plan summary is
provenance for the plan used by this run; cache records, debug artifacts, and provenance for the plan used by this run; cache records, debug artifacts, and
other operational state are not published as bundle files. other operational state are not published as bundle files.
Each `validation_summaries` entry is a bounded outcome for one producer result. ## Rejections And Warnings
It has required `status`, `producer_attempt_count`, and `terminal_action`;
the stage and affected step, lane, module, or chunk identity are present when
applicable. `status` is `complete`, `rejected`, or `incomplete`.
`rejecting_validators`, `reason_codes`, and `incomplete_validators` preserve
configured validator order and omit later duplicates. Entries contain no raw
candidate response, correction guidance, validator diagnostic message, or
artifact payload. The same shape may appear as `validation` on an affected
rejection entry.
When present, `metadata.session_id` is the effective non-secret routing
correlation identifier used for the run. It can be visible to providers and is
not a substitute for a cache or checkpoint identity. Its generation and
override behavior are defined by the [CLI reference](../cli.md#run).
Each `llm_profiles` entry identifies effective, non-secret LLM execution
provenance:
| Field | Required | Meaning |
| --- | --- | --- |
| `id` | Yes | Selected PromptKit profile identifier. |
| `provider` | No | Notarius adapter provider identifier. |
| `model` | No | Effective provider model identifier. |
| `backend_id` | No | Effective PromptKit backend registration identifier. Endpoint-only profiles omit it. |
| `reasoning_effort` | No | Effective opaque provider reasoning setting. An empty or explicitly cleared setting is omitted. |
These values describe observed execution; they are not a backend-registration
interface. Entries that differ by backend or effective reasoning remain
distinct even when their profile, provider, and model are otherwise equal.
## Rejections, Warnings, And Diagnostics
`rejected.json` is always an object with a `rejected` array. Each entry has `rejected.json` is always an object with a `rejected` array. Each entry has
required `stage` and `message`; `step_id`, `lane_id`, `module_key`, `chunk_id`, required `stage` and `message`; `step_id`, `lane_id`, `module_key`, `chunk_id`,
`chunk_index`, `validator_name`, `reason_code`, `attempt_count`, and `chunk_index`, `validator_name`, `reason_code`, `attempt_count`, and
`diagnostic_artifact_path` are present only when applicable. An entry may also `diagnostic_artifact_path` are present only when applicable.
contain the bounded `validation` summary described above; the existing singular
validator and reason fields remain the first configured rejection for
compatibility.
`warnings.json` is always the `notarius.warnings.v2` envelope: `warnings.json` is always an object with a `warnings` array. Each warning has
`reason_code` and `message`; `scope` is optional. Both arrays are empty when
```json there is nothing to report.
{
"schema_version": "notarius.warnings.v2",
"group_count": 0,
"occurrence_count": 0,
"groups": []
}
```
It contains only process warnings. `group_count` is exact, and
`occurrence_count` is the exact sum of its group occurrence counts.
`diagnostics.json` is always the `notarius.diagnostics.v1` envelope:
```json
{
"schema_version": "notarius.diagnostics.v1",
"group_count": 0,
"occurrence_count": 0,
"truncated": false,
"unrepresented_occurrence_count": 0,
"groups": []
}
```
It contains only advisory and observation groups. `group_count` counts groups
represented in `groups`; `occurrence_count` includes both represented and
unrepresented occurrences. When `truncated` is true,
`unrepresented_occurrence_count` is the exact number omitted from group
representation.
Each group has `disposition`, `category`, `reason_code`, framework-owned
`origin`, exact `occurrence_count`, bounded `samples`, and
`omitted_sample_count`. Samples carry safe `scope` and `message`, plus a chunk
ID and zero-based chunk index when applicable. A group retains at most three
distinct samples. The framework fails rather than truncating actionable
warnings beyond 128 groups; it represents at most 256 advisory/observation
groups and records further occurrences through the diagnostic truncation
fields above.
## Compatibility ## Compatibility

View File

@@ -1,13 +0,0 @@
# ConfigKit Integration
Notarius uses the public root package
[`gitea.maximumdirect.net/eric/ConfigKit`](https://gitea.maximumdirect.net/eric/ConfigKit)
at v0.4.0 for version 5 configuration acquisition, imports, overlays, explicit
overrides, diagnostics, and safe inspection rendering.
Notarius owns the typed document, domain defaults and validation, runtime
conversion, catalog-dependent pipeline resolution, source-aware configured
paths, and command discovery. It uses only ConfigKit's public path, result,
diagnostic, and resolution-source APIs. Upstream documentation owns general
ConfigKit semantics; [Configuration](../config.md) owns Notarius's file and
operator contract.

View File

@@ -1,178 +0,0 @@
# PromptKit Integration
Notarius pins
[`gitea.maximumdirect.net/eric/promptkit` v0.11.0](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.11.0)
as its in-process prompt engine. The upstream
[Go package consumer guide](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.11.0/docs/consumers/pkg-promptkit.md)
owns the public engine API, and the upstream
[format reference](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.11.0/docs/formats.md)
owns prompt, profile, and schema file contracts.
## Supported Boundary
Notarius relies on the root `promptkit` package to:
- construct an `Engine` with filesystem-backed prompt, schema, and optional
operator and application-fallback profile sources;
- prepare one frozen execution from a `RunRequest` with named inline artifacts,
variables, a direct session ID, prompt identity, profile selection, and
optional appended rendered messages, then
record credential-redacted details and run that exact execution;
- return rendered debug material, validated structured output, selected
profile, backend, effective model metadata, and token usage;
- register the optional conventional `local` backend through `BackendLocal`,
`LocalBackend`, and `WithBackend`;
- distinguish structured-output validation failure from execution failure; and
- identify a missing explicit profile through `ErrProfileNotFound` and backend
admission exhaustion through `ErrCapacityExceeded`.
The pinned
[`BackendLocal`, `LocalBackend`, and `WithBackend` API](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.11.0/backends.go)
owns the registration and backend-capacity contract.
For one completion, the adapter calls `PrepareExecution`, takes a
caller-owned `Details` snapshot, and calls `RunPrepared` for that same opaque
prepared execution. It defers `Discard` for every unexecuted handle. Explicit
profile preflight uses `Engine.InspectProfile`; it does not prepare a synthetic
prompt. PromptKit's prepared handle, inspection result, and capacity-error
types stay inside the Notarius LLM adapter.
When a PromptKit profile and runtime override leave `temperature`, `max_tokens`,
or `top_p` unset, Notarius leaves that control unset as well. Compatible
providers therefore apply their own defaults; an operator that requires a
specific sampling value must select it explicitly in the profile or runtime
override.
Notarius does not use PromptKit's optional `ArtifactReader`. It materializes
source and reference content itself and supplies owned inline artifacts at the
adapter boundary. It also retains responsibility for pipeline retries,
scheduling, debug persistence, redaction, profile provenance, and conversion
from private model responses into durable domain artifacts.
Notarius sends one stable effective session through PromptKit's direct session
field, which is authoritative for provider session behavior. It also retains
the same value as the `session_id` prompt variable for maintained prompt
compatibility. The generated identifier is 76 ASCII characters, within
PromptKit's 256-code-point session limit. Session IDs are non-secret
correlation identifiers and may be exposed to providers and provider
observability. The CLI contract owns generation and override behavior.
Notarius records PromptKit's selected backend ID and effective reasoning
setting as optional run-manifest provenance. Endpoint-only profiles have no
backend ID. Debug prompt material also retains the selected backend ID and
PromptKit's stable lower-case `effective_model_params` JSON, which may include
`backend_id`. Notarius production configuration exposes one optional
conventional `local` registration. It does not expose a general user-defined
PromptKit backend registry. Endpoint-only profiles remain supported unchanged.
Notarius retains its application-wide scheduled client around the PromptKit
adapter. PromptKit may apply a narrower limit for the selected backend;
endpoint-only profiles have no such backend limit. The adapter translates
PromptKit capacity rejection into the provider-neutral Notarius
`ErrLLMCapacityExceeded` contract. It may include the normalized selected
backend ID in safe diagnostic context, without exposing PromptKit's capacity
error type. PromptKit's built-in client retries eligible transient HTTP failures
with its v0.11.0 default of two additional attempts per generation; Notarius
keeps pipeline-stage retries outside that completion boundary.
The adapter projects built-in generation failures as the application-owned
`LLMGenerationError`, retaining a safe failure kind, HTTP status when present,
and retry-exhaustion attempt count. Transport, malformed successful envelopes,
provider statuses, and generic generation failures remain distinct without
exposing PromptKit error values to Notarius callers.
## Profile Sources And Compatibility
Notarius gives PromptKit the configured operator profile source, registered
application fallback profile assets, and optional backend registration through
the same construction path for inspection and execution. PromptKit owns the
resulting source precedence and strict profile parsing: a matching operator
profile is a complete replacement for a fallback or built-in profile, while an
invalid matching document fails instead of falling through. The operator
configuration and deployment workflow are defined in
[Configuration](../config.md#promptkit-profiles) and
[Operations](../operations.md#promptkit-profile-deployment).
PromptKit owns `base_profile` resolution under its
[pinned format rules](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.11.0/docs/formats.md).
Notarius records the selected leaf identity and resolved target without parsing
or merging inheritance. An unset filesystem `api_key_env` is optional and may
reach the provider without authorization, which can result in a 401 or 403.
PromptKit accepts only the `developer`, `system`, `user`, and
`assistant` text-chat roles after normalizing case and surrounding whitespace.
Maintained Notarius prompt definitions use only `system` and `user`.
For application-owned semantic correction, Notarius uses PromptKit's
`RunRequest.AppendedMessages` after the ordinary rendered prompt. It supplies
exactly two messages in order: the latest validated producer response with
role `assistant`, then deterministic validation guidance with role `user`.
It never exposes a general caller-selected role API, accumulates earlier
correction turns, or changes the ordinary prompt prefix. Ordinary requests
leave appended messages unset.
PromptKit preserves supplied content but does not own Notarius's correction
bounds. Notarius rejects invalid UTF-8, blank, or oversized assistant material
(at most 1 MiB), guidance (at most 64 KiB), and combined content (at most
1,114,112 bytes) before preparing the request. The transport-neutral
application contract owns defensive copying and these limits. Default request
and terminal summaries retain only safe counts, digests, identities, and usage;
complete appended messages remain limited to the explicitly requested detailed
debug trace.
PromptKit obtains its maintained OpenRouter and Rakestrawhome backend and
profile catalogs from independently versioned transitive modules. Notarius
does not import or register either catalog; PromptKit retains catalog source,
identity, precedence, credential, and capacity ownership.
External catalog profile names and settings can change between dependency
releases, including removal without aliases. Review directly selected IDs and
inherited `base_profile` values when upgrading. An application-owned alias that
inherits an external profile does not freeze that profile's settings.
PromptKit's opt-in `promptkit-openrouter-standard` and
`promptkit-openrouter-auto` names provide name continuity within its
selected-and-tested dependency scope, not a fixed model or provider availability
guarantee. See the upstream
[catalog compatibility policy](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.11.0/docs/policy/release.md#catalog-profile-compatibility).
Use a complete operator-owned profile with explicit backend or endpoint, model,
and settings when those choices must remain operator-controlled. The embedded
`dnd-extraction` fallback already follows this model and does not inherit an
external catalog profile. An upgrade updates Notarius's compiled-in catalog
checkpoint identity so prior profile-source checkpoints are not reused.
Notarius supports this boundary against PromptKit v0.11.0. Its fallback source,
prepared-execution, inspection, and typed capacity APIs are used as public
upstream contracts; other PromptKit APIs or file-format behavior are not
implicitly supported. A dependency upgrade requires reviewing the adapter,
profile-source construction, and this compatibility statement against the
pinned upstream documentation.
## Notarius Ownership
[LLM Runtime Internals](../internal/llm.md) describes how Notarius mounts
module assets, maps its transport-neutral completion contract, prepares and
executes requests, validates output, records provenance, captures debug
material, redacts errors, and preserves timeout ownership.
PromptKit provider error details do not cross the ordinary completion boundary.
Notarius exposes a provider-neutral generation category and optional status;
redacted provider details are retained only in requested debug material.
[D&D Module Internals](../internal/dnd.md) owns the embedded
`dnd-extraction` fallback profile and the maintained D&D prompt defaults.
[Configuration](../config.md#promptkit-profiles) defines how a Notarius
configuration selects one PromptKit profile source and optionally registers
the conventional local backend.
PromptKit API or format changes outside this boundary are not implicitly
supported. Updating the pinned version requires reviewing the adapter and
profile/configuration contracts against the upstream documentation. Maintained
production prompts use PromptKit's bounded structural-repair contract; their
current declaration is one additional repair attempt. Notarius retains the
transport-neutral boundary and does not expose PromptKit types to modules.
PromptKit v0.11.0 is licensed under
[LGPL-3.0-or-later](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.11.0/LICENSE).
Its accompanying [GPLv3 notice](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.11.0/COPYING)
is incorporated by that license. This dependency change does not alter
Notarius's source license or distribution workflow.

View File

@@ -9,58 +9,36 @@ Command syntax, streams, and exit statuses are defined in the
## Schema ## Schema
The current schema version is `notarius.run-result.v2`. The current schema version is `notarius.run-result.v1`.
| Field | Required | Meaning | | Field | Required | Meaning |
| --- | --- | --- | | --- | --- | --- |
| `schema_version` | Yes | Exactly `notarius.run-result.v2`. | | `schema_version` | Yes | Exactly `notarius.run-result.v1`. |
| `run_id` | Yes | The finalized Notarius run identifier. | | `run_id` | Yes | The finalized Notarius run identifier. |
| `pipeline_id` | Yes | The effective pipeline identifier. | | `pipeline_id` | Yes | The effective pipeline identifier. |
| `output_directory` | Yes | Absolute path to the published, run-specific output bundle. | | `output_directory` | Yes | Absolute path to the published, run-specific output bundle. |
| `index_file` | For the production JSON output | Logical path `index.json`; omitted for other output modules. | | `index_file` | For the production JSON output | Logical path `index.json`; omitted for other output modules. |
| `normalized_output_count` | Yes | Number of final normalized outputs. | | `normalized_output_count` | Yes | Number of final normalized outputs. |
| `rejected_output_count` | Yes | Number of recorded rejected outputs. | | `rejected_output_count` | Yes | Number of recorded rejected outputs. |
| `warning_group_count` | Yes | Exact number of actionable warning groups. | | `warning_count` | Yes | Number of final run warnings. |
| `warning_occurrence_count` | Yes | Exact occurrences represented by actionable warning groups. |
| `diagnostic_group_count` | Yes | Number of represented advisory and observation groups. |
| `diagnostic_occurrence_count` | Yes | Advisory and observation occurrences, including unrepresented occurrences. |
| `diagnostics_truncated` | Yes | Whether advisory/observation group representation was truncated. |
| `validation_status` | Yes | The final run manifest validation status. | | `validation_status` | Yes | The final run manifest validation status. |
| `validation_summaries` | No | Bounded per-producer validation outcomes; present when producer work ran. |
| `debug_directory` | No | Absolute path to the run-specific debug bundle when requested debug capture completed. | | `debug_directory` | No | Absolute path to the run-specific debug bundle when requested debug capture completed. |
For the production `json` output module, `index_file` is present only when the For the production `json` output module, `index_file` is present only when the
completed run returned exactly one logical output file named `index.json`. completed run returned exactly one logical output file named `index.json`.
For another output module, its absence does not indicate a failed run. For another output module, its absence does not indicate a failed run.
`validation_status` is `approved`, `rejected`, or `incomplete`; `incomplete`
means one or more otherwise accepted results advanced under validator-failure
`warn_continue`.
```json ```json
{ {
"schema_version": "notarius.run-result.v2", "schema_version": "notarius.run-result.v1",
"run_id": "run-1770000000000000000-0123456789abcdef0123456789abcdef", "run_id": "run-1770000000000000000-0123456789abcdef0123456789abcdef",
"pipeline_id": "dnd-session", "pipeline_id": "dnd-session",
"output_directory": "/work/results/run-1770000000000000000-0123456789abcdef0123456789abcdef", "output_directory": "/work/results/run-1770000000000000000-0123456789abcdef0123456789abcdef",
"index_file": "index.json", "index_file": "index.json",
"normalized_output_count": 6, "normalized_output_count": 6,
"rejected_output_count": 2, "rejected_output_count": 2,
"warning_group_count": 1, "warning_count": 1,
"warning_occurrence_count": 2, "validation_status": "rejected"
"diagnostic_group_count": 3,
"diagnostic_occurrence_count": 5,
"diagnostics_truncated": false,
"validation_status": "incomplete",
"validation_summaries": [
{
"stage": "extract",
"lane_id": "spells",
"status": "incomplete",
"incomplete_validators": ["dnd/spells/source_refs"],
"producer_attempt_count": 1,
"terminal_action": "warn_continue"
}
]
} }
``` ```
@@ -70,11 +48,8 @@ means one or more otherwise accepted results advanced under validator-failure
paths. They identify the paths used by Notarius and do not resolve symlinks. paths. They identify the paths used by Notarius and do not resolve symlinks.
`output_directory` is the run-specific bundle, not the configured output root. `output_directory` is the run-specific bundle, not the configured output root.
The receipt is a summary and discovery document. Its optional validation The receipt is a summary and discovery document. It does not contain lane
summaries contain only stable status, identity, validator names, reason codes, descriptors, payloads, manifest data, rejections, warnings, or file contents.
attempt counts, and terminal actions. It does not contain lane descriptors,
payloads, manifest payloads, rejection messages, warnings, raw model responses,
correction guidance, or file contents.
For the production JSON output, resolve `index_file` beneath For the production JSON output, resolve `index_file` beneath
`output_directory`, reject path escapes, and use the `output_directory`, reject path escapes, and use the
[Published JSON Output contract](json-output.md) to discover logical files and [Published JSON Output contract](json-output.md) to discover logical files and

View File

@@ -24,34 +24,23 @@ preparation, and runner mechanics after their inputs are supplied.
## Dispatch And Configuration Handoff ## Dispatch And Configuration Handoff
The root dispatcher handles help, version reporting, configuration validation, The root dispatcher handles help, configuration validation, pipeline listing,
effective-configuration inspection, explanation inspection, pipeline listing, and a pipeline run. It normalizes injectable options before dispatch so that a
and a pipeline run. Version reporting resolves build missing production dependency fails as a command error rather than reaching
information through `internal/buildinfo` before production composition, so the execution.
diagnostic remains available without configuration or runtime collaborators.
The public syntax, streams, exit classes, and version semantics are defined by
the [CLI reference](../cli.md). Other root commands normalize injectable
options before dispatch so that a missing production dependency fails as a
command error rather than reaching execution.
Commands that need configuration use one shared ConfigKit-backed loader. The Commands that need configuration use one shared loader. The CLI discovers the
CLI discovers the file, passes the requested overlay and only explicit file, parses it through **internal/core/config**, starts from defaults, applies
configuration-valued run overrides, and receives a **LoadedConfig**. The the file and supported environment overrides, and then validates it for the
configured discovery and precedence contract is in command. The configured discovery and precedence contract is in
[Configuration](../config.md), while the loading and resolution mechanics are [Configuration](../config.md), while the loading and resolution mechanics are
in [Configuration Internals](configuration.md). in [Configuration Internals](configuration.md).
`config effective` and `config explain` only render the loaded ConfigKit view; Configuration validation without a selected pipeline checks structural
they do not resolve pipelines or construct providers. Configuration validation without a selected pipeline checks structural
configuration only. Validation with a selected pipeline also builds the configuration only. Validation with a selected pipeline also builds the
effective catalog, resolves the pipeline, and verifies every explicit effective effective catalog, resolves the pipeline, and verifies explicitly selected
PromptKit profile. Selected LLM-backed input, chunk, lane, output, and validator Scriptorium profiles. Pipeline listing validates configuration before returning
profiles are inspected normalized, sorted identifiers.
against the configured PromptKit source and backend registrations without
loading a prompt or performing generation, so an unknown or invalid profile
fails before pipeline preparation. Credential availability remains an
execution-time concern. Pipeline listing validates configuration before
returning normalized, sorted identifiers.
## Production Composition ## Production Composition
@@ -62,43 +51,29 @@ catalog used for resolution and the concrete constructors used for preparation.
Tests may provide a catalog or registries instead; production code must not Tests may provide a catalog or registries instead; production code must not
silently merge an injected partial catalog with production registrations. silently merge an injected partial catalog with production registrations.
The production LLM factory builds one PromptKit-backed client from the resolved The production LLM factory builds the Scriptorium-backed client from resolved
**promptkit.profile_dir** or **promptkit.profile_file** source, attaches the configuration, creates one scheduler from the effective global LLM limit, and
profile-provenance recorder, creates one scheduler from the effective global wraps the client before it reaches modules. Registration and LLM construction
LLM limit, and wraps the client before it reaches modules. Registration and LLM errors are returned before a pipeline is prepared. Concrete module keys and
construction errors are returned before a pipeline is prepared. Configuration validator chains are public configuration choices and remain documented in
field definitions remain in [Configuration](../config.md#promptkit-profiles); [Configuration](../config.md).
the D&D registrar's fallback profile assets and the adapter mechanics remain in
[LLM Runtime](llm.md).
The factory also accepts `LLMRuntimeOverrides`, whose reasoning pointer
preserves inherit, replace, and clear states across the composition boundary.
Run orchestration constructs this value from the mutually exclusive
`--reasoning-effort` and `--clear-reasoning-effort` controls. Absence preserves
a nil pointer, replacement is trimmed, and clear uses a non-nil empty string.
The same override reaches the one shared production client, checkpoint
identity, and debug invocation metadata. Persistent reasoning configuration
remains owned by PromptKit profiles; Notarius configuration has no reasoning
field.
## Run Orchestration ## Run Orchestration
After parsing and validating a run invocation, the CLI performs this ordered After parsing and validating a run invocation, the CLI performs this ordered
handoff: handoff:
1. load and validate configuration with ConfigKit command overrides; 1. load and validate configuration, then apply command-level operational
overrides;
2. create and validate a safe run identity, then allocate a debug bundle only 2. create and validate a safe run identity, then allocate a debug bundle only
when requested; when requested;
3. build the effective catalog, resolve requested reference changes, resolve 3. build the effective catalog, resolve requested reference changes, resolve
the effective pipeline, and inspect its explicit effective PromptKit the effective pipeline, and verify explicit Scriptorium profiles;
profiles;
4. materialize external or generated references and record redacted invocation 4. materialize external or generated references and record redacted invocation
and resolution provenance when debug capture is enabled; and resolution provenance when debug capture is enabled;
5. construct registries, the scheduled LLM client, and prepared modules; 5. construct registries, the scheduled LLM client, prepared modules, and the
6. read the source input once, resolve its effective session from the explicit requested cache/checkpoint collaborators;
override or resolved input module and raw bytes, then construct requested 6. read the source input and invoke the framework runner; and
checkpoint collaborators and invoke the framework runner with that same
value; and
7. write the runner's logical output files only after a successful run, then 7. write the runner's logical output files only after a successful run, then
complete the command report and user-facing result. complete the command report and user-facing result.
@@ -109,23 +84,6 @@ final command result. Detailed state lifecycle, resume handling, and physical
path confinement are maintained in [Run State Internals](state.md) and path confinement are maintained in [Run State Internals](state.md) and
[Operations](../operations.md). [Operations](../operations.md).
The CLI owns the versioned generated-session policy and resolves the sole
effective value before checkpoint construction. It records that value in the
final debug invocation summary when capture is enabled and passes it unchanged
to checkpoint identity and `pipeline.RunInput`. The public flag and stability
contract are defined by the [CLI reference](../cli.md#run); framework and LLM
packages only transport the supplied value.
The CLI also owns the scope grammar for reference flags. It enumerates the
selected chunk and lane targets from registered module specifications, expands
pipeline- and lane-scoped actions into exact stage-and-lane bindings, and
resolves overlapping bind and unbind actions by specificity before calling
configuration resolution. The generic pipeline therefore receives only exact
`ReferenceBinding` and `ReferenceUnbind` values and has no knowledge of CLI
selector syntax. Configuration resolution retains ownership of configured
external/generated conflicts, required slots, and module compatibility; file
materialization still occurs afterward.
For `run --json`, the CLI constructs and encodes its private run-result receipt For `run --json`, the CLI constructs and encodes its private run-result receipt
after a successful runner result is available, before it publishes logical after a successful runner result is available, before it publishes logical
output files. It writes the prepared receipt to standard output only after output files. It writes the prepared receipt to standard output only after
@@ -177,9 +135,8 @@ is discoverable.
- **internal/cli/production_contract_test.go** covers registrar composition, - **internal/cli/production_contract_test.go** covers registrar composition,
production catalog contents, assets, and representative configuration production catalog contents, assets, and representative configuration
validation. validation.
- **internal/cli/reference_contract_test.go** covers scoped CLI reference - **internal/cli/reference_contract_test.go** covers CLI reference overrides,
expansion, specificity, bind/unbind conflicts, generated-reference origin separation, and materialization boundaries.
protection, origin separation, and materialization boundaries.
- **internal/cli/state_hardening_test.go** covers safe run identity, state - **internal/cli/state_hardening_test.go** covers safe run identity, state
roots, and failure ordering. roots, and failure ordering.

View File

@@ -8,17 +8,17 @@ either interface.
## Boundary ## Boundary
The configuration package turns a selected ConfigKit-composed YAML document The configuration package turns a selected YAML file and supported environment
into a validated, independently owned configuration. It then resolves a values into a validated, independently owned configuration. It then resolves a
requested pipeline against a module catalog before the framework prepares or requested pipeline against a module catalog before the framework prepares or
runs anything. runs anything.
| Boundary | Inputs | Outputs | Does not own | | Boundary | Inputs | Outputs | Does not own |
| --- | --- | --- | --- | | --- | --- | --- | --- |
| Loading | Selected file path, profile, overrides, and invocation directory | **LoadedConfig** with ConfigKit inspection metadata and a populated **Config** | Choosing the file path or reporting a command result. | | Loading | Selected file path and environment lookup | Parsed file model and a populated **Config** | Choosing the file path or reporting a command result. |
| Validation | **Config** | Structural configuration errors with pipeline, lane, or binding context | Module availability, capabilities, or construction. | | Validation | **Config** | Structural configuration errors with pipeline, lane, or binding context | Module availability, capabilities, or construction. |
| Resolution | Valid **Config**, selected pipeline and lanes, runtime reference changes, LLM override, and module catalog | **EffectiveConfig** with a **ResolvedPipeline** | Materializing reference bytes, preparing modules, execution, or filesystem state. | | Resolution | Valid **Config**, selected pipeline and lanes, runtime reference changes, LLM override, and module catalog | **EffectiveConfig** with a **ResolvedPipeline** | Materializing reference bytes, preparing modules, execution, or filesystem state. |
| Inspection | **LoadedConfig** and an optional JSON Pointer | Redacted effective YAML or explanation text | Pipeline selection, module preparation, or provider work. | | Summary | **Config** or **EffectiveConfig** | Detached redacted payload suitable for debug summaries | Redacting arbitrary process state or provider traffic. |
The CLI discovers a configuration file, invokes this package, and supplies the The CLI discovers a configuration file, invokes this package, and supplies the
result to the framework. Configuration never reads an input file, constructs a result to the framework. Configuration never reads an input file, constructs a
@@ -28,65 +28,50 @@ at their respective [CLI](cli.md), [pipeline](pipeline.md), and
## Loading And Validation ## Loading And Validation
ConfigKit acquires the selected root, imports, optional selected overlay, and The CLI loads configuration in this order:
explicit CLI overrides in precedence order. Its document definition supplies
typed defaults, source-aware path normalization, and structural validation 1. parse the selected YAML file strictly into the file model;
before conversion to the independently owned runtime **Config**. Only 2. start from **Default**;
`NOTARIUS_CONFIG` participates in discovery; retired field-level environment 3. apply the file model; and
variables do not participate in loading. Version 5 decoding rejects scalar 4. apply the supported environment overrides.
module bindings and scalar external references. ConfigKit retains redacted,
historical inspection metadata while **LoadedConfig** keeps that implementation This establishes the public precedence order without giving environment input a
detail private behind effective and explanation writers. second file schema. Loading and application reject malformed YAML, unsupported
file versions, unknown fields, invalid values, and identifiers that are empty
or collide after whitespace normalization. The file application also makes the
effective extraction-worker default follow the effective LLM limit.
**Config.Validate** checks configuration-only invariants before resolution. It **Config.Validate** checks configuration-only invariants before resolution. It
rejects incompatible profile sources, invalid state-surface values, unsupported rejects incompatible profile sources, invalid state-surface values, unsupported
concurrency settings, malformed bindings and references, invalid retries, and concurrency settings, malformed bindings and references, invalid retries, and
invalid pipeline, step, or lane structure. PromptKit local-backend validation invalid pipeline, step, or lane structure. Its errors retain the closest known
accepts only an absolute HTTP or HTTPS endpoint with a host and no user pipeline, lane, and binding context. It deliberately does not require modules
information, query, or fragment, and rejects a negative local concurrency to be registered: that requires a catalog and belongs to resolution.
limit. It also rejects a negative `promptkit.transient_retries` value; decoded
omission resolves to two, while explicit zero remains a distinct opt-out. Its
errors retain the closest known pipeline, lane, and binding context.
It deliberately does not require modules to be registered: that requires a
catalog and belongs to resolution.
The exact user-selectable values and validation rules are defined in The exact user-selectable values and validation rules are defined in
[Configuration](../config.md). Keep document schema, source-aware normalization, [Configuration](../config.md). Keep additions to the file model, an
runtime conversion, and that reference aligned in the same change. environment override, its validation, and that reference in the same change.
## Effective Resolution ## Effective Resolution
**Config.Resolve** derives an absent extraction-worker value in an owned copy **Config.Resolve** first recomputes derived concurrency defaults and validates
and validates the configuration. A programmatically supplied worker value is the configuration. It normalizes the requested pipeline ID, copies the selected
always explicit. It normalizes the requested pipeline ID, copies the selected profile, applies a non-empty command-level LLM profile override to the
profile, and passes the non-empty command-level LLM profile override, requested LLM-capable stage bindings, and calls the framework resolver with the requested
lane selection, and reference changes to the framework resolver. lane selection and reference changes.
After module and validator selection, the resolver applies the effective The command-level override does not replace an explicitly selected validator
profile policy to LLM-backed bindings only: command override, binding profile, profile. Validator bindings remain part of the resolved validator chain and
pipeline profile, then the prompt default. Deterministic bindings remain are resolved under their own declared configuration.
profile-free, and no second inheritance decision occurs during execution. The
public field definitions and precedence are owned by
[Configuration](../config.md#pipelines).
The resolver retains configured `validation_policy` overrides and derives one
detached concrete terminal policy for the chunk producer and every lane's
extract, merge, and normalize producers. That field-by-field inheritance is
complete before preparation, and the effective values contribute to pipeline
and checkpoint identity; execution does not interpret configuration defaults.
The framework resolver supplies defaults, selects lanes, resolves validator The framework resolver supplies defaults, selects lanes, resolves validator
chains, checks registered module and artifact compatibility, validates module chains, checks registered module and artifact compatibility, validates module
options, and returns the fixed ordered pipeline shape. Positive validator retry options, and returns the fixed ordered pipeline shape. The resulting
budgets require an LLM-backed selected validator; deterministic validators are **EffectiveConfig** retains the selected ID, requested selection and reference
rejected during resolution. Eligible LLM-backed producer specifications also
contribute their declared correction protocol to the resolved metadata. The
resulting **EffectiveConfig** retains the selected ID, requested selection and reference
changes, a clone of the input configuration, and the resolved pipeline. changes, a clone of the input configuration, and the resolved pipeline.
Callers may therefore retain or modify their input slices and maps without Callers may therefore retain or modify their input slices and maps without
changing the resolved result, and later consumers cannot mutate the original changing the resolved result, and later consumers cannot mutate the original
configuration through the effective value. This ownership includes the nested configuration through the effective value.
PromptKit local-backend value.
Resolution failures stop before module construction and source parsing. They Resolution failures stop before module construction and source parsing. They
include an error path for an unconfigured pipeline, missing module, missing include an error path for an unconfigured pipeline, missing module, missing
@@ -98,46 +83,46 @@ runtime error class described in the [CLI reference](../cli.md#output-streams-an
The framework assigns the resolved pipeline a deterministic SHA-256 digest The framework assigns the resolved pipeline a deterministic SHA-256 digest
after defaults, lane selection, module bindings, reference bindings, validator after defaults, lane selection, module bindings, reference bindings, validator
chains, selected correction protocols, effective LLM profiles, and artifact chains, and artifact schema identity have been resolved. The digest excludes
schema identity have been resolved. The digest excludes
its own stored value. It identifies resolved composition rather than raw YAML its own stored value. It identifies resolved composition rather than raw YAML
bytes, a debug payload, or all runtime state. The CLI records it as invocation bytes, a debug payload, or all runtime state. The CLI records it as invocation
provenance before execution; cache and checkpoint identity have additional provenance before execution; cache and checkpoint identity have additional
owners in [Run State Internals](state.md). owners in [Run State Internals](state.md).
Configuration publication uses ConfigKit's redacted effective YAML. Resolved Configuration summaries must use **Redacted**, **RedactedSummaryPayload**, or
pipeline publication uses **RedactedResolvedPipelinePayload**, which copies **RedactedResolvedPipelinePayload**, never a direct configuration marshal.
bindings and nested option containers, replaces credential-shaped values with Those methods copy every binding and nested option container, replace values
**[REDACTED]**, and omits materialized reference content while retaining safe whose key is credential-shaped with **[REDACTED]**, and omit materialized
provenance. This redaction is deliberately narrow and does not authorize reference content while retaining safe binding and reference provenance. The
recording arbitrary environment values or provider requests. payload must not alias the source configuration or resolved pipeline. This
redaction is deliberately narrow: it protects configuration summaries and does
not authorize recording arbitrary environment values or provider requests.
## Invariants To Preserve ## Invariants To Preserve
- ConfigKit composition applies defaults, imports, an optional overlay, and - Defaults, YAML values, and environment values are applied in one direction;
explicit overrides in one direction with retained origin metadata. later sources may override only their supported operational settings.
- A configuration is structurally valid before it is resolved, and a resolved - A configuration is structurally valid before it is resolved, and a resolved
pipeline is compatible with the supplied catalog before preparation begins. pipeline is compatible with the supplied catalog before preparation begins.
- Whitespace-normalized identifiers are unique wherever they identify a - Whitespace-normalized identifiers are unique wherever they identify a
pipeline, step, lane, worker, or reference slot. pipeline, step, lane, worker, or reference slot.
- Runtime conversion and resolved-pipeline publication return detached data. - Resolution and summary generation return detached data. Redaction must cover
- The resolved PromptKit transient-retry budget is always passed to the shared every configured and resolved binding, including nested validator bindings.
built-in client; document presence preserves an explicit zero.
- The resolved digest changes when resolved composition changes and never - The resolved digest changes when resolved composition changes and never
includes itself. includes itself.
## Focused Tests ## Focused Tests
- **internal/core/config/document_test.go** and - **internal/core/config/file_config_contract_test.go** covers strict file
**operational_document_test.go** cover ConfigKit document loading, parsing, normalization, file application, and structural rejection.
normalization, imports, overlays, and overrides. - **internal/core/config/env_contract_test.go** covers supported operational
overrides and their precedence.
- **internal/core/config/validation_contract_test.go** covers configuration - **internal/core/config/validation_contract_test.go** covers configuration
invariants and contextual failures. invariants and contextual failures.
- **internal/core/config/effective_config_contract_test.go** covers defaults, - **internal/core/config/effective_config_contract_test.go** covers defaults,
selections, overrides, resolution context, digest changes, and ownership. selections, overrides, resolution context, digest changes, and ownership.
- **internal/core/config/loaded_config_test.go** covers safe effective and - **internal/core/config/redaction_test.go** covers recursive credential
explanation rendering; **redaction_test.go** covers resolved-pipeline redaction, reference-content exclusion, and non-aliasing payloads.
redaction and reference-content exclusion.
Run **go test ./internal/core/config** after changing this boundary. Changes to Run **go test ./internal/core/config** after changing this boundary. Changes to
the handoff or resolved-composition semantics also need the focused framework the handoff or resolved-composition semantics also need the focused framework

View File

@@ -7,40 +7,25 @@ selectable keys, bindings, reference syntax, and default validator chains.
## Durable Artifact Contracts ## Durable Artifact Contracts
The ten lanes have separate durable wire contracts. This guide deliberately The six lanes have separate durable wire contracts. This guide deliberately
does not repeat their JSON shapes or schemas. does not repeat their JSON shapes or schemas.
| Lane | Durable contract | | Lane | Durable contract |
| --- | --- | | --- | --- |
| Spells | [spell artifacts](../integrations/dnd-spell-artifacts.md) | | Spells | [spell artifacts](../integrations/dnd-spell-artifacts.md) |
| NPC registry | [NPC registry artifacts](../integrations/dnd-npc-registry-artifacts.md) | | NPCs | [NPC artifacts](../integrations/dnd-npc-artifacts.md) |
| Combat turns | [combat-turn artifacts](../integrations/dnd-combat-turn-artifacts.md) | | Combat turns | [combat-turn artifacts](../integrations/dnd-combat-turn-artifacts.md) |
| Item occurrences | [item-occurrence artifacts](../integrations/dnd-item-occurrence-artifacts.md) | | Item events | [item-event artifacts](../integrations/dnd-item-event-artifacts.md) |
| Item registry | [item-registry artifacts](../integrations/dnd-item-registry-artifacts.md) | | NPC interactions | [NPC-interaction artifacts](../integrations/dnd-npc-interaction-artifacts.md) |
| NPC occurrences | [NPC-occurrence artifacts](../integrations/dnd-npc-occurrence-artifacts.md) |
| Scene descriptions | [scene-description artifacts](../integrations/dnd-scene-description-artifacts.md) | | Scene descriptions | [scene-description artifacts](../integrations/dnd-scene-description-artifacts.md) |
| Enemy events | [enemy-event artifacts](../integrations/dnd-enemy-event-artifacts.md) |
| Location registry | [location-registry artifacts](../integrations/dnd-location-registry-artifacts.md) |
| Location occurrences | [location-occurrence artifacts](../integrations/dnd-location-occurrence-artifacts.md) |
## Family Composition ## Family Composition
The D&D registrar registers the family’s artifact codecs, extractors, typed The D&D registrar registers the family’s artifact codecs, extractors, typed
append-order mergers, normalizers, validators, prompt assets, fallback LLM append-order mergers, normalizers, validators, prompt assets, and default
profile asset, and default validator chains. Each extractor and normalizer has validator chains. Each extractor and normalizer has a stable module spec,
a stable module spec, explicit execution class, strict option decoding, and a strict option decoding, and a typed builder. Configuration remains the
typed builder. Scene chunking, every extractor, and NPC, location, and canonical owner of the exact keys and validator order.
item-registry normalization are registered as `llm_backed`; the remaining
current D&D mergers and normalizers are `deterministic`. The metadata is
available to catalog inspection and resolved-pipeline debug data and determines
which selected bindings inherit the pipeline profile. The registry normalizers
use `single_response_v1`, forwarding corrections to their reconciliation
completion and retaining the accepted raw proposal only as an owned model
candidate. When deterministic proposal safety or typed application rejects a
group, they provide separate model-facing prose for a corrective module retry;
internal issue categories, reason codes, and operator messages remain
diagnostic-only. Configuration remains the canonical owner of the exact keys,
profile precedence, and validator order.
Private structured-LLM response schemas are deliberately minimal. They reject Private structured-LLM response schemas are deliberately minimal. They reject
invalid JSON structure, missing required fields, incompatible types, and invalid JSON structure, missing required fields, incompatible types, and
@@ -48,267 +33,46 @@ unknown fields, while preserving semantic candidates for deterministic
validation. Do not promote a private response envelope into a durable schema; validation. Do not promote a private response envelope into a durable schema;
the contracts above define durable data. the contracts above define durable data.
A D&D producer that declares `single_response_v1` forwards any supplied
semantic correction to its structured completion and returns an owned copy of
that completion's exact validated raw response as its model candidate. It does
not serialize normalized artifacts to create that candidate, so deterministic
identity, evidence, warning, and durable-schema behavior remains separate from
the model transport material.
## Prompt Construction ## Prompt Construction
D&D LLM-facing content lives beneath `assets/dnd/`. Each module contributes a D&D extractors assemble prompts from an ordered manifest of shared and
local `prompt.yaml` declaration and `instructions.md`; input-specific files module-owned assets. Reuse the shared D&D system, evidence, identity,
such as a catalog, registry, grounding projection, or candidate collection are reference, and transcript assets instead of copying their text into individual
local only when that module needs them. New extractor content uses its feature modules. A manifest’s declared sequence, including cache-control placement, is
subtree, while families with both extraction and normalization content use their part of the prompt behavior, and the chunk transcript is the final message.
`extract` and `normalize` subtrees. Shared visual-provenance fragments use Preserve that order when changing an extractor or its assets so prompt-cache
the `common-dnd-` prefix. Production lane code belongs with its D&D codec, behavior remains stable.
extractor, normalizer, and validator packages; registry projections and
identity helpers remain in their owning entity packages rather than in a
consumer lane.
The owning module’s manifest is the source of truth for which local and shared All extractors use the shared prompt-input preparation rules. The current chunk
assets are selected, their mount paths, their message order, cache controls, is copied into transcript material; player, party, glossary, and compatible
and the files included in its prompt fingerprint. Shared fragments belong to campaign references are context for disambiguation, not source evidence.
the D&D shared implementation and are selected by name rather than copied into Reference prompt material is canonically ordered before it is rendered, which
module directories. The root `assets` package is a content-only boundary; its keeps equivalent inputs stable across runs.
physical ownership and rationale are defined by
[ADR-0011](../adr/0011-centralize-llm-assets.md).
Put each rule at its narrowest owner:
- universal behavior belongs in the shared system asset;
- D&D-family behavior belongs in a selected `common-dnd-` asset;
- rules for an input projection belong with that input asset;
- lane-specific policy belongs in the module’s `instructions.md`; and
- transport-envelope shape belongs in the private response schema.
A rule is eligible for the system prompt only when every D&D LLM prompt needs
it regardless of lane, inputs, or response shape. Module instructions must not
repeat rules selected from shared assets or schemas. Reintroduce such repetition
only after observational evaluation with representative transcripts shows that
it improves results at the intended target models and cost; structural prompt
tests alone are not that evidence.
Every maintained D&D LLM prompt selects `dnd-extraction` as its default
profile. The D&D registrar registers the fallback, while an operator can
replace it with a complete profile of the same ID from the configured PromptKit
source. Deployment profile selection is documented in
[Configuration](../config.md#promptkit-profiles).
The D&D transcript assets have distinct consumers. Scene chunking consumes the
complete-session `common-dnd-transcript-full.md`, while extraction prompts
consume the current-chunk `common-dnd-transcript-chunk.md`. NPC, location, and
item normalization instead mount the generic semantic-reconciliation
candidate and transcript-window presentation assets. Player, party, glossary,
and compatible campaign references provide disambiguating context only when
declared by the active prompt; they never establish evidence. Reference
material is canonically ordered before rendering so equivalent inputs remain
stable.
Extraction prompts render the common system and identity messages first, then
cached campaign references and the cached chunk transcript. Evidence policy and
any lane-specific registry, catalog, or grounding projection follow that
prefix. The final module instructions message is ephemeral. This keeps the
reusable extraction prefix identical while preserving the lane-specific suffix.
Scene chunking intentionally uses a different order: system, cached campaign
references, uncached module instructions, then the final ephemeral full
transcript. Entity normalization also has its own order: D&D system, mandatory
generic protocol, ephemeral domain semantic instructions, generic candidate
presentation, and final ephemeral generic transcript windows. These orders and
cache controls are prompt behavior; change them only through the owning
manifest and prompt declaration.
NPC, item, and location registry reconciliation translate shared proposal
safety categories into bounded prose that references only the response-local
duplicate-group ordinals and candidate handles. Item reconciliation appends its
typed rule that currency may be consolidated only with aliases of the same
denomination and never with non-currency items. The next normalize attempt
receives that prose with the exact defective proposal under the shared
replacement-request protocol. Structurally invalid output has no valid proposal
candidate and receives a fresh feedback-free attempt instead. If the stage
budget is exhausted, safe groups stay applied, unsafe groups stay separate, and
one fallback warning summarizes the final defect without raw model content.
## Evidence, Candidates, And Normalization ## Evidence, Candidates, And Normalization
The current transcript is the only durable evidence source. Extractors assign The current transcript is the only durable evidence source. Extractors assign
the current source identity and losslessly order any reversed range whose two the current source identity, preserve candidate evidence ranges for validators,
endpoints resolve in that source, using transcript position rather than numeric and canonically order or remove exact duplicate ranges without asking the
unit-ID order. They then canonically order ranges and remove exact duplicates. model to repair semantic errors. Campaign context and generated artifacts may
This routine canonicalization does not request a retry or emit a warning. ground names or control routing, but they never establish evidence for a D&D
Unresolvable or wrong-source ranges remain unchanged for validators to reject. result.
Campaign context and generated artifacts may ground names or control routing,
but they never establish evidence for a D&D result.
Default chains keep responsibilities separate: structural validators assess the Default chains keep responsibilities separate: structural validators assess the
candidate, source-reference validators resolve cited ranges against the current candidate, source-reference validators resolve cited ranges against the current
source and require extraction evidence to stay within the current chunk, source, durable-schema validation checks an approved representation, and
durable-schema validation checks an approved representation, and
relatedness validators report advisory evidence concerns. The configured order relatedness validators report advisory evidence concerns. The configured order
is documented in is documented in
[Configuration](../config.md#production-validator-keys-and-default-chains). [Configuration](../config.md#production-validator-keys-and-default-chains).
The optional `extract/dnd/scene-descriptions/combat_semantics` validator is the Normalizers are deterministic for spells, combat turns, item events, NPC
D&D family's LLM-backed review of only combat versus non-combat classification. interactions, and scene descriptions. They canonicalize display values and
It selects the shared combat-policy prompt fragment and asks the model to evidence, use source-document order for stable output, and issue bounded
classify the current chunk independently as `combat` or `non_combat` without warnings for changes or collapsed duplicates. The NPC normalizer is the
receiving the proposed scene kind. Deterministic code compares that intentional exception: it first produces a deterministic candidate set, then
classification with the proposed kind and either approves it or produces the uses a bounded structured-LLM proposal to reconcile identity groups. Invalid
appropriate correction guidance. This keeps every schema-valid classification or unusable proposals retain the deterministic result and surface retry or
interpretable and avoids anchoring the reviewer on the producer's answer. It fallback diagnostics; the model does not directly replace durable records.
does not assess titles, summaries, non-combat subtype, or scene boundaries;
deferred boundary-coherence review remains separate. It is opt-in;
[Configuration](../config.md) owns selection and retry/failure behavior.
### Combat-semantics provider evaluation
The human-reviewed corpus at
`internal/modules/dnd/validate/scenedescriptions/combat_semantics/testdata/evaluation_cases.json`
owns the proposed kind, expected combat classification, and reviewer rationale
for each synthetic case. Its package test validates the fixture contract only.
Provider evaluation remains an explicit maintainer operation and must not be
added to the default offline test suite.
Use the following protocol before proposing default-chain inclusion:
1. Record the Notarius commit, prompt and schema fingerprints, provider, model,
profile settings, reasoning effort, structural-repair setting, number of
repetitions, and evaluation date before collecting results. Do not revise
expected classifications merely to agree with provider output; a substantive
corpus correction requires independent human review.
2. Exercise the production validator construction and prompt assets from an
explicitly invoked, disposable evaluation driver or test in the validator
package. For each corpus case, construct transcript source units from the
listed IDs and text, assign matching per-unit source references, and use
`source.MaterializeChunkPlan` with one range spanning those units. Construct
exactly one scene whose ID and source range match that chunk and whose kind
is the case's `proposed_kind`; title and summary may use fixed placeholders
because the validator neither receives nor evaluates them. Invoke the typed
validator at extract stage through the production registry and scheduled LLM
client. Do not commit provider credentials, generated source material, or an
always-on live test.
3. Compare the model classification with `expected_classification`, then verify
that its deterministic comparison with `proposed_kind` yields the expected
approval or rejection direction.
Record an unexpected approval of an expected rejection as a false
acceptance, an unexpected rejection of an expected approval as a false
rejection, and any validator execution failure separately from semantic
accuracy. Retain per-case results so repeated trials and systematic failure
modes remain visible.
4. Evaluate producer correction separately with representative complete
scene-description runs configured as shown in
[Configuration](../config.md#production-validator-keys-and-default-chains).
For every initial semantic rejection, record whether the next producer
attempt returns the requested combat status and is approved. Do not count a
PromptKit structural repair as a producer-correction attempt.
5. Run the correction evaluation with debug capture enabled and without reused
extraction checkpoints. Record added validator and producer calls, elapsed
latency, and cumulative prompt, completion, cached, and total token usage
from the debug attempt records. Compare these values with an otherwise
identical run whose scene-description chain omits the semantic validator.
The default-chain review must consider classification error, false acceptance,
false rejection, execution failure, producer-correction success, added calls,
latency, and token use together. A structurally successful provider run alone
is not evidence that the validator should become a default.
Every producer-correctable D&D rejection describes all currently detectable
corrections in transcript-grounded domain terms, using contextual names,
model-owned artifact fields, and source segment ranges when useful. Validators
collect independent record defects in one pass so one retry does not merely
reveal the next issue. Shared D&D diagnostic helpers keep repeated rules and
record descriptions stable, de-duplicated, and bounded; each artifact family
continues to own the semantic rule and its prose.
Operator diagnostics and model guidance are separate products of the same
assessment. Operator messages may use typed paths, reason details, and opaque
application identities. Correction guidance must not copy those messages or
ask the model to reproduce durable entity IDs, hashes, validator module keys,
reason codes, or Go field paths. A registry-normalization rejection instead
speaks in terms of the duplicate-group proposal response the normalizer can
actually revise. Normalization-only deterministic invariants retain useful
operator detail but do not imply that a model controls derived ordering or
identity. Only bounded actionable semantic guidance is eligible for a
correction prompt.
Private LLM schemas use simple enums for closed categorical fields when the
provider-compatible shape can express the rule directly. Deterministic typed
validators retain the same checks as defense in depth and for non-LLM
producers. Private schemas keep every property required and avoid optional
properties, `uniqueItems`, and conditional cross-field logic.
Item-occurrence shape validation groups repeated holder mistakes by occurrence
kind and gives the producer the required JSON null/non-null relationship. It
identifies affected records by contextual item name and cited transcript range,
never by the deterministically attached durable item ID. Holder mistakes remain
semantic rejections rather than silent rewrites because changing a holder can
also change the meaning of the occurrence kind.
Enemy-event extraction additionally rejects a second `engaged` observation for
the same comparison identity within one scene-scoped result. Normalization may
combine results from distinct scenes, so it intentionally does not apply that
rule. Configuration owns the exact validator key and chain position.
Extraction source-reference validators share one full-span chunk-containment
policy. After ordinary reference validity succeeds, the policy resolves both
endpoints through document order and requires every source unit in the
inclusive range to be present in the current chunk. It does not assume numeric
unit-ID ordering, mutate input, or weaken wrong-source and unresolved-reference
validation. Scene descriptions remain separate because their validator owns an
exact one-scene range contract rather than general extraction containment.
Normalizers are deterministic for spells, combat turns, item occurrences, NPC
occurrences, scene descriptions, enemy events, and location occurrences. They
canonicalize display values and evidence, use source-document order for stable
output, and emit bounded normalization observations for changes or collapsed duplicates. NPC,
item, and location registry normalizers are intentional exceptions: each first
produces a deterministic candidate set, then may use a bounded structured-LLM
proposal to reconcile identity groups.
## Semantic Registry Reconciliation
The three registry normalizers instantiate the domain-neutral
`internal/framework/semanticreconcile` engine with default bounds. Each
eligible candidate receives a contiguous, one-based `candidate_number` for that
request, and candidate material declares the exact inclusive range. The model
sees that handle, the candidate label and source-free evidence ranges, plus
bounded transcript windows; it returns only duplicate
groups of supplied handles and one supplied canonical handle per group. It
never returns names, evidence, durable IDs, or replacement records. Identical
labels and evidence remain independently selectable because their handles are
distinct. Transcript unit `id` values and evidence `start_unit_id` and
`end_unit_id` values are source positions in a separate namespace and are
never valid candidate numbers.
The generic core owns the mandatory handle protocol, candidate and transcript
presentation, the private response schema, source-reference validation,
candidate and combined-material limits, structured completion, proposal
assessment, stable group ordering, and typed plan-application mechanics. The
D&D prompt contributes its system message and registry-specific semantic
instructions. The generic registrar registers the shared prompt and schema;
the D&D registrar registers each consuming prompt and the fallback profile.
Fewer than two eligible candidates skips the LLM without a semantic warning.
An exceeded bound also skips the call and preserves the deterministic
preprocessed registry, adding the registry's bounded fallback warning. Invalid
structured output or discarded proposal groups use the normalizer's existing
retry contract; retry exhaustion preserves the safe deterministic or
partially applied result and emits its bounded fallback warning. Provider,
transport, cancellation, and context-material failures remain execution
errors.
Application remains typed and registry-owned. All three policies select the
canonical member's normalized display name, union member evidence in source
order, preserve ungrouped records, and derive durable identity only after
consolidation. NPC IDs derive from the final name. Item IDs also derive from
the final name, and a typed guard prevents currency aliases from crossing
denominations or mixing currency with non-currency records. Location IDs
derive from the final name and final evidence, preserving same-name,
parent/child, and distinct physical-place identities. Registry warning scopes,
reason codes, and postconditions remain outside the generic core.
## Generated References And Grounding ## Generated References And Grounding
@@ -318,55 +82,30 @@ producer provenance; consumers resolve the handed-off artifact into an
immutable, validated projection for each operation. External files are checked immutable, validated projection for each operation. External files are checked
during preparation, while generated artifacts are resolved at the handoff. during preparation, while generated artifacts are resolved at the handoff.
NPC and item registry consumers receive names-only grounding. Location NPC registries are names-only grounding projections: they may canonicalize
consumers receive a contextual selector containing the canonical name and the actors for spells and combat turns and are required for NPC interactions, but
registry references needed to distinguish same-name places. The calling module they do not supply evidence. Scene-description registries are eligibility-only
resolves those supplied selections locally and maps them into the unchanged projections: they retain the current chunk’s classification data, not scene
durable ID/name pair; an unknown or ambiguous selection rejects the complete prose or evidence, and exist to route combat extraction.
occurrence result rather than accepting a partial mapping. The NPC registry
additionally supplies names-only actor grounding to spells, combat turns, and
enemy events.
Registry references establish a registry identity and may disambiguate a
selection, but never become occurrence evidence. Each occurrence keeps its own
current-transcript source references, even when it was grounded through the
same registry record.
Scene descriptions are eligibility-only projections: they retain current-chunk
classification data, not scene prose or evidence, and exist to route combat
extraction. Enemy-event extraction also projects combat turns to `actor` and
`turn_kind` and filters NPC occurrences to `combat_opponent` names and kinds.
These projections are guidance only and never event evidence.
## Lane-Specific Rules ## Lane-Specific Rules
The following differences are intentional and should remain explicit when a The following differences are intentional and should remain explicit when a
shared helper changes. shared helper changes.
Shared D&D text comparison is identified by `dnd.text_comparison.v1`. Any
semantic change requires an explicit policy-version review for every affected
identity, mapping, normalization, and validator policy; helper source is not a
checkpoint fingerprint.
| Lane | Intentional behavior | | Lane | Intentional behavior |
| --- | --- | | --- | --- |
| Spells | May use a spell-catalog overlay and optional NPC grounding; the catalog validator supplies domain-specific semantic checks. | | Spells | May use a spell-catalog overlay and optional NPC grounding; the catalog validator supplies domain-specific semantic checks. |
| NPC registry | Establishes transcript-grounded NPC identities, including factual third-party mentions, without assigning occurrence categories. It does not consume an NPC registry, and its normalizer is the LLM-assisted reconciliation exception described above. | | NPCs | Does not consume an NPC registry. Its normalizer is the LLM-assisted reconciliation exception described above. |
| Combat turns | Requires a scene-description artifact. It calls the LLM only for an exact `combat` classification; exact non-combat classifications return an accepted empty result, while missing or mismatched classifications return an empty result with a bounded warning. Optional NPC grounding never becomes evidence. | | Combat turns | Requires a scene-description artifact. It calls the LLM only for an exact `combat` classification; exact non-combat classifications return an accepted empty result, while missing or mismatched classifications return an empty result with a bounded warning. Optional NPC grounding never becomes evidence. |
| Item occurrences | Requires the normalized item registry for exact deterministic grounding at extraction and normalization. Campaign context may disambiguate, but the registry never becomes occurrence evidence. | | Item events | Uses campaign context for disambiguation but has no NPC-registry or scene-description dependency. |
| Item registry | Produces source-grounded item types and unique designations. Its LLM-assisted reconciliation is proposal-only, preserves distinct currency denominations and item types, and does not create per-instance identities. | | NPC interactions | Requires the normalized NPC registry at extraction and normalization, using it for canonical actor grounding only. |
| NPC occurrences | Requires the normalized NPC registry at extraction and normalization, using it for canonical actor grounding only. It separately emits cited current-transcript occurrence facts, including `mentioned`, rather than deriving them from registry provenance. |
| Scene descriptions | Produces the classifications consumed by combat routing; it does not consume an NPC registry or provide evidence for combat artifacts. | | Scene descriptions | Produces the classifications consumed by combat routing; it does not consume an NPC registry or provide evidence for combat artifacts. |
| Enemy events | Requires NPC, scene-description, combat-turn, and NPC-occurrence artifacts. It calls the LLM only for an exact `combat` classification, records ordered observations rather than terminal state, and normalizes recognized names through the NPC registry while preserving grounded collective labels. |
| Location registry | Produces a source-anchored, session-scoped registry from stable proper names or unique in-world designations. Its LLM-assisted reconciliation is proposal-only and never collapses same-name places without validated identity and evidence rules. |
| Location occurrences | Requires the normalized location registry for both extraction and normalization. Its [durable occurrence categories](../integrations/dnd-location-occurrence-artifacts.md#occurrence-categories) distinguish explicit speculation from unsupported inference; the deterministic normalizer enforces exact registry grounding and never turns registry provenance into occurrence evidence. |
The combat and scene-description contracts describe their exact handoff and The combat and scene-description contracts describe their exact handoff and
empty-result behavior in more detail: empty-result behavior in more detail:
[combat turns](../integrations/dnd-combat-turn-artifacts.md) and [combat turns](../integrations/dnd-combat-turn-artifacts.md) and
[scene descriptions](../integrations/dnd-scene-description-artifacts.md). [scene descriptions](../integrations/dnd-scene-description-artifacts.md).
The [enemy-event contract](../integrations/dnd-enemy-event-artifacts.md)
defines its durable semantics; [Configuration](../config.md) owns its
selectable bindings and validation chains.
## Focused Verification ## Focused Verification

View File

@@ -1,12 +1,12 @@
# LLM Runtime Internals # LLM Runtime Internals
`internal/framework/llm` is Notarius’s provider-independent structured `internal/framework/llm` is Notarius’s provider-independent structured
completion boundary. It adapts framework requests to PromptKit, bounds completion boundary. It adapts framework requests to Scriptorium, bounds
provider calls, assembles registered prompt and schema assets, records selected provider calls, assembles registered prompt and schema assets, records selected
profiles, and redacts provider errors. The architectural boundary is defined in profiles, and redacts provider errors. The architectural boundary is defined in
[Architecture](../policy/architecture.md#llm-boundary); profile sources, [Architecture](../policy/architecture.md#llm-boundary); profile sources,
credentials, and concurrency settings belong in credentials, and concurrency settings belong in
[Configuration](../config.md#promptkit-profiles) and [Configuration](../config.md#scriptorium-profiles) and
[Configuration](../config.md#concurrency-output-cache-and-debug). [Configuration](../config.md#concurrency-output-cache-and-debug).
## Structured Completion Boundary ## Structured Completion Boundary
@@ -24,102 +24,25 @@ adapter does not own source evidence, artifact conversion, normalization, or
durable schemas. Those responsibilities remain with the module and its durable schemas. Those responsibilities remain with the module and its
[integration contract](../integrations/). [integration contract](../integrations/).
The calling module also resolves contextual entity selections and attaches any `ScriptoriumClient` validates the request target and prompt identity, maps each
application identity; PromptKit and this adapter do not own entity identity. named material to a Scriptorium inline artifact while preserving its origin URI,
forwards session and profile selection, then prepares and runs the prompt. It
returns Scriptorium’s validated raw bytes rather than re-encoding the decoded
target. An empty optional material is represented as one space so its named
input is retained by Scriptorium.
`PromptKitClient` validates the request target and prompt identity, maps each An empty request profile lets the prompt select its configured default. The CLI
named material to a PromptKit inline artifact while preserving its origin URI, prepares every explicitly selected binding profile before a run begins, so a
passes the supplied request session through to PromptKit's direct per-run missing explicit profile fails before stage execution. Calls record the profile
session field, retains the same value as the `session_id` prompt variable for actually selected by Scriptorium; the recorder deduplicates non-secret profile
maintained prompt compatibility, and forwards profile selection. It does not identity, provider, and model values for manifest use.
derive or replace session values; the CLI owns that policy. It then creates one
frozen prepared execution, captures its caller-owned credential-redacted
details for debug material, and executes that exact snapshot through
PromptKit's prepared-execution boundary. The direct field
is authoritative for provider session behavior. A session ID is a stable,
non-secret correlation identifier and may be exposed to providers and provider
observability. The adapter returns PromptKit’s validated raw bytes rather than
re-encoding the decoded target. An empty optional material is represented as
one space so its named input is retained by PromptKit.
When a request includes semantic correction material, the adapter validates and
defensively copies it before preparation, then appends exactly two messages
after the ordinarily rendered prompt: the prior response as an assistant
message and the correction guidance as a user message. Requests without a
correction do not add messages or introduce caller roles. Ordinary request
summaries record correction byte counts and digests only; complete messages are
available solely in an explicitly requested debug trace.
The adapter leaves the ordinary rendered message prefix, named inputs,
variables, session, profile, execution overrides, prepared-execution path, and
PromptKit repair policy unchanged for a corrected request. It never imports a
PromptKit message type into a module or pipeline contract. PromptKit reports
actual structural repair count and cumulative token usage per completion; the
pipeline's safe terminal debug record projects those values without copying
message content.
Client construction may also receive a run-wide reasoning-effort override from
the CLI factory boundary. The adapter copies the caller-owned pointer and
creates a fresh PromptKit execution override for each request: a nil pointer
inherits the selected profile, a non-empty value replaces it, and an empty
value clears inherited reasoning. The CLI's mutually exclusive
`--reasoning-effort` and `--clear-reasoning-effort` controls select those
states. With neither flag, profile behavior remains unchanged. Because
production constructs one shared client, the selected state applies uniformly
to module calls, retries, and LLM-backed validators for the whole run.
An empty request profile lets the prompt select its configured default. Before a
run begins, the CLI asks the adapter to inspect every explicit profile on the
resolved selected LLM-backed bindings and validators, including inherited
pipeline profiles. Inspection resolves the profile and its selected backend and
target without loading a prompt, reading credentials, admitting capacity, or
contacting a provider, so a missing or invalid explicit profile fails before
stage execution while a valid `api_key_env` may remain unset. Calls record the
profile actually selected by PromptKit. The recorder trims and deduplicates
non-secret profile identity, provider, model, selected backend ID, and
effective reasoning values for manifest use. Entries that differ in backend or
reasoning remain distinct and deterministically ordered. Endpoint-only profiles
retain an empty backend ID, which the published JSON omits. Successful
completion responses and recorded profile manifests identify the adapter
provider as `promptkit`.
The CLI's profile-inspection engine and the production adapter use the same
profile-source construction to apply the configured profile directory or file,
the optional registered fallback profile assets, and the optional conventional
`local` backend. Preflight therefore resolves the same profile sources and
backend membership as runtime without performing generation. Fallback assets
are mounted only when at least one source is registered. The production D&D
registrar contributes its `dnd-extraction` fallback, and the maintained D&D
prompts select that logical ID by default. PromptKit owns source precedence and
profile parsing and inheritance: an operator-provided matching profile takes precedence over a
fallback profile without Notarius merging either document.
When the registration is absent, a profile selecting `backend: local` fails
inspection instead of falling back to a built-in or endpoint-only target.
Before execution, the adapter also contributes a non-secret checkpoint
fingerprint for the effective PromptKit profile source. It combines the
identity of PromptKit's compiled-in profile catalog with a deterministic digest
of every YAML profile in the configured profile directory, or of the configured
profile file, and a deterministic digest of the flattened fallback profile
assets. The fingerprint contains neither profile content nor source paths. It
covers inherited pipeline profiles, explicit binding profiles, and
prompt-selected defaults, so changing a model or other profile setting cannot
reuse checkpoints created under the
prior profile source. This cache identity is independent of durable
profile provenance: run manifests continue to list only profiles actually
observed during LLM calls. When the local backend is registered, a second
fingerprint hashes its trimmed endpoint behind a stable marker. Changing that
semantic execution target invalidates checkpoint reuse. The raw endpoint is not
stored in checkpoint identity, and the local concurrency limit is excluded
because it changes scheduling rather than execution semantics.
## Shared Provider-Call Limit ## Shared Provider-Call Limit
Production construction creates one PromptKit client and wraps it in one Production construction creates one Scriptorium client and wraps it in one
scheduled client. The scheduler has a fixed, positive permit limit, serves scheduled client. The scheduler has a fixed, positive permit limit, serves
queued calls in FIFO order, and removes a queued call when its context is queued calls in FIFO order, and removes a queued call when its context is
cancelled. It rechecks the caller context after admission and before dispatch. cancelled. A granted permit is released exactly once on every completion path.
A granted permit is released exactly once on every completion path.
The scheduled wrapper surrounds every `CompleteStructured` call, so concurrent The scheduled wrapper surrounds every `CompleteStructured` call, so concurrent
lanes, pipeline retries, and LLM-backed validators share the same provider-call lanes, pipeline retries, and LLM-backed validators share the same provider-call
@@ -128,57 +51,22 @@ worker counts cannot exceed the configured LLM limit. The configuration field
and its effective default are owned by and its effective default are owned by
[Configuration](../config.md#concurrency-output-cache-and-debug). [Configuration](../config.md#concurrency-output-cache-and-debug).
PromptKit applies a second, independent admission limit when the selected
profile names a limited backend. It sits beneath the Notarius scheduled client,
so it may narrow but cannot expand the application-wide limit. Built-in
OpenRouter profiles select PromptKit's reserved backend and its upstream
capacity policy. A positive configured local-backend limit bounds active local
generations inside PromptKit; zero leaves that backend unlimited there.
Endpoint-only profiles do not select a PromptKit backend and remain limited
only by the Notarius scheduler.
## Prompt And Schema Assets ## Prompt And Schema Assets
An `AssetRegistry` collects prompt, schema, and optional fallback-profile An `AssetRegistry` collects prompt and schema filesystems from production module
filesystems from production module families. It flattens registered roots into families. It flattens registered roots into the Scriptorium filesystems and
the corresponding PromptKit filesystems and rejects invalid roots, unreadable rejects invalid roots, unreadable assets, duplicate paths, and missing prompt
assets, duplicate paths, and missing prompt or schema files during preparation. or schema files during preparation. The framework’s `promptfs` helper combines
Fallback assets receive a safe content digest for checkpoint identity; raw module-owned prompt files with reusable domain fragments without making the
paths and bytes are never included. The framework’s `promptfs` helper combines
module-selected prompt files with reusable domain fragments without making the
framework depend on D&D content. framework depend on D&D content.
LLM-facing content is embedded once by the root `assets` package. Each consumer Each LLM-backed module owns its prompt declaration, package-specific assets,
uses only its scoped subtree, while the module retains ownership of its prompt and private response schema. Shared D&D wording is owned by the D&D shared
declaration, ordered manifest, private response-schema identity, and asset package; the detailed D&D conventions are in
registration. Shared D&D fragments are selected by D&D's shared implementation; [D&D Module Internals](dnd.md). The mounted prompt assets used by a module also
the detailed convention is in [D&D Module Internals](dnd.md). This physical determine its prompt fingerprint. Schema loaders validate JSON, attach identity
arrangement and its data-only boundary are defined by and digest metadata, make defensive copies, and expose diagnostics without raw
[Architecture](../policy/architecture.md) and schema bytes.
[ADR-0011](../adr/0011-centralize-llm-assets.md), rather than by this runtime
guide.
The generic registrar is the sole production registration owner for the
semantic-reconciliation default prompt and private response schema. The
domain-neutral reconciliation package also exposes only its mandatory protocol
and candidate/transcript presentation files for domain prompt manifests. D&D
registry normalizers mount those files while retaining ownership and hashing
of their D&D system message, semantic instructions, and complete prompt
declaration. The response schema is therefore registered once even though
several typed normalizers select it.
Mounted prompt assets determine a module's fingerprint. The fingerprint hashes
only the module and shared files explicitly selected by its manifest, so an
unrelated asset does not invalidate a checkpoint. Schema loaders validate JSON,
attach identity and digest metadata, make defensive copies, and expose
diagnostics without raw schema bytes.
Semantic-reconciliation normalizers extend this identity with the shared
response-schema digest, framework policy version, and complete limit-policy
digest. Their manifest metadata records the same content-free prompt, schema,
policy, and limit identities together with domain identity and normalization
policies. Request-local handles, source material, proposal content, and raw
asset bytes are not checkpoint metadata.
Private response schemas validate a model transport envelope. They are not the Private response schemas validate a model transport envelope. They are not the
durable artifact schema and should not be documented as an external wire durable artifact schema and should not be documented as an external wire
@@ -188,136 +76,54 @@ contract. Durable formats and compatibility rules remain in the
## Prompt Maintenance And Backend Caching ## Prompt Maintenance And Backend Caching
Prompt message order and shared asset bytes are runtime behavior. Backend cache Prompt message order and shared asset bytes are runtime behavior. Backend cache
reuse depends on identical preceding roles, rendered bytes, and cache-control reuse depends on the same preceding messages and content, not merely equivalent
metadata—not merely equivalent meaning. Keep reusable shared assets meaning. Keep reusable shared assets byte-identical and keep stable material
byte-identical and preserve each prompt’s declared ordering and cache controls before the inputs that vary per request wherever a prompt’s declared sequence
when editing it. supports caching. Preserve the existing manifest order and cache-control hints
when editing a prompt.
For sibling prompts that can reuse the same source material, order universal D&D extraction manifests place the changing chunk transcript at the end of the
shared context first, request source material next, and module-specific prompt after their reusable context. Scene chunking and NPC normalization use
suffixes last. Put a cache boundary at a reusable prefix that is useful to the their own declared message sequences because their inputs and work differ. The
backend. Redundant intermediate cache boundaries do not extend that reusable family-specific asset and ordering rules belong in [D&D Module Internals](dnd.md).
prefix and add no value. Do not add tests that enforce a fixed message-prefix length; prompt-asset tests
should instead verify the meaningful asset sequence, inputs, and cache controls
Prompt-family owners may choose a different sequence when their inputs and of the prompt being changed.
reuse pattern differ. The D&D family’s extraction, scene-chunking, and NPC
normalization policies are maintained in [D&D Module Internals](dnd.md#prompt-construction).
Do not add tests that enforce prompt prose; prompt tests should verify the
meaningful input placement and cache controls of the prompt being changed.
## Validation, Repair, And Retries ## Validation, Repair, And Retries
PromptKit performs prompt rendering, provider execution, and the prompt’s Scriptorium performs prompt rendering, provider execution, and the prompt’s
structured-output validation. The adapter reports an empty result, validation structured-output validation. The adapter reports an empty result, validation
failure, empty structured body, or decode failure as failure, empty structured body, or decode failure as
`ErrInvalidStructuredOutput`, while retaining the returned raw bytes and debug `ErrInvalidStructuredOutput`, while retaining the returned raw bytes and debug
material when they exist. Provider failures remain operational errors rather material when they exist. Provider failures remain operational errors rather
than output-validation failures. Apart from documented context, capacity, and than output-validation failures.
invalid-output categories, provider error values and types do not cross the
adapter error chain; callers receive only a credential-redacted diagnostic.
`LLMGenerationError` preserves the provider-neutral failure kind (`transport`, Prompt-declared repair is executed within Scriptorium’s structured-output flow.
`provider_status`, `malformed_response`, or generic `generation`) and any HTTP The current production D&D prompt manifests set repair attempts to zero. That
status. When PromptKit exhausts an eligible transient-status retry budget, it setting does not replace pipeline retry behavior: a binding’s configured retry
also records that exhaustion and the attempts in that generation. Those fields count reruns its stage attempt after an error or rejection, and an exhausted
do not include repair generations, and zero attempts means unavailable rather rejection is a recorded output rather than a provider error. The pipeline owns
than proof that no request was sent. attempt lifecycle, validation chains, and retry diagnostics; see
Every execution that reaches PromptKit returns an optional request-local
`execution` summary. It contains only terminal aggregate attempts, retry and
repair counts, durations, outcome and failure classification, status, and
presence-aware token usage. Preparation failures have no summary; later local
decode failures retain the upstream terminal summary without changing it.
When PromptKit rejects backend admission before generation, the adapter maps
`promptkit.ErrCapacityExceeded` to
`contracts.ErrLLMCapacityExceeded`, retaining prompt context and a redacted
upstream diagnostic without exposing the PromptKit sentinel or capacity-error
type as a framework contract. When supplied, the normalized selected backend
ID appears only in that safe application-owned diagnostic context. A canceled
caller context takes precedence. The adapter does not retry capacity failures;
the pipeline's existing binding attempt policy sees the operational error and
decides whether to rerun the complete operation.
PromptKit executes structural repair within its structured-output flow. The
maintained production prompt manifests declare one additional repair attempt.
When a resolved binding supplies a repair value, the adapter inspects the
prompt, copies its complete output contract, changes only the repair limit, and
passes that complete replacement contract to PromptKit. This preserves the
prompt's output format, validation mode, schema, and provider structured-output
settings.
A successful repair is an ordinary successful completion, not a warning. The
adapter reports PromptKit's actual repair count and its cumulative usage
directly, without adding the initial and corrective counts again. Debug prompt
material records the configured complete contract; debug response material
records the repaired response and actual validation result. If the repair
budget is exhausted, the adapter retains the final raw bytes and debug material
and reports `ErrInvalidStructuredOutput`. Generation failures during an initial
or corrective call remain provider-neutral operational errors with the same
redaction boundary.
Structural repair does not replace pipeline retry behavior: a binding's
configured retry count reruns its complete stage attempt after an operational
or structural error, module-requested retry, or actionable semantic rejection.
An actionable module-requested retry and a validator rejection both use the
same correction payload when the producer exposes an exact latest response;
feedback-free module retries reconstruct the ordinary request without appended
messages. The pipeline owns attempt lifecycle, validation chains, and retry
diagnostics; see
[Pipeline Internals](pipeline.md#validation-retries-and-output) and the [Pipeline Internals](pipeline.md#validation-retries-and-output) and the
[binding reference](../config.md#module-bindings-and-validators). [binding reference](../config.md#module-bindings-and-validators).
## Timeout Ownership
The caller context remains the outer cancellation authority. PromptKit applies
a positive effective generation timeout as an inner request deadline for each
built-in generation attempt; an explicit zero disables only that generation
deadline. The HTTP client timeout is a separate transport-wide cap. Notarius
forwards the caller context and does not install another timeout wrapper around
PromptKit.
The selected PromptKit profile owns generation settings. The resolved global
`promptkit.transient_retries` configuration is passed to the shared PromptKit
engine and controls additional built-in attempts for eligible transient HTTP
failures in one generation. It defaults to two; explicit zero disables those
attempts. Notarius binding retries remain outside the adapter and repeat the
complete module operation and validation chain.
Operator-facing behavior is summarized in
[Operations](../operations.md#operational-limits), and the pinned upstream
contract is identified in
[PromptKit Integration](../integrations/pkg-promptkit.md).
## Observability And Redaction ## Observability And Redaction
When debug recording is enabled, the pipeline decorates the shared client. The When debug recording is enabled, the pipeline decorates the shared client. The
wrapper records prepared prompt and response material, timing, selected profile wrapper records prepared prompt and response material, timing, selected profile
and backend, effective model parameters, and call identifiers in the run’s and model, and call identifiers in the run’s debug bundle, including material
debug bundle, including material available from a failed structured completion. available from a failed structured completion. For a successful completion, a
Effective parameters use PromptKit's stable lower-case JSON field names and may debug-write failure is surfaced; when the completion already failed, its call
include `backend_id`. For a successful completion, a debug-write failure is error remains the result. Debug-bundle location, retention, and handling are
surfaced; when the completion already failed, its call error remains the operational concerns documented in [Operations](../operations.md#debug-bundles).
result. Debug-bundle location, retention, and handling are operational concerns
documented in [Operations](../operations.md#debug-bundles).
The attempt-terminal summary is a separate safe trace record: it contains Run manifests receive selected profile summaries and component identities, not
attempt kinds, validator outcome counts and reason codes, effective policy, prompt, schema, source, reference, or response content. Provider error text is
terminal action, and repair/usage references. It excludes raw assistant wrapped with prompt context and bearer credentials are redacted before it
responses and correction text. Those values can appear only in the explicitly crosses the runtime boundary. Known-secret redaction is available to other
requested detailed prompt and response artifacts, which require sensitive-data runtime collaborators; it does not make prompt or response contents safe for
handling. general logging.
Run manifests receive selected profile summaries, including optional effective
backend and reasoning provenance, and component identities—not prompt, schema,
source, reference, or response content. The published field semantics belong
to the [JSON output contract](../integrations/json-output.md#manifestjson).
Provider error text is wrapped with prompt context and bearer credentials are
redacted before it crosses the runtime boundary. Known-secret redaction is
available to other runtime collaborators; it does not make prompt or response
contents safe for general logging.
Generation failures expose an application-owned category and optional HTTP
status. Provider code, type, and message remain debug-only, after redaction.
## Failure Boundaries ## Failure Boundaries
@@ -325,8 +131,6 @@ status. Provider code, type, and message remain debug-only, after redaction.
sources, invalid asset registration, or a non-positive scheduler limit. sources, invalid asset registration, or a non-positive scheduler limit.
- Preparation failures, unavailable explicit profiles, provider failures, and - Preparation failures, unavailable explicit profiles, provider failures, and
context cancellation propagate to the calling stage with context. context cancellation propagate to the calling stage with context.
- Backend admission exhaustion is a provider-neutral operational error and is
not classified as invalid structured output or validator rejection.
- Malformed or schema-invalid provider output is classified separately as - Malformed or schema-invalid provider output is classified separately as
invalid structured output so the module or pipeline can apply its own retry invalid structured output so the module or pipeline can apply its own retry
and rejection policy. and rejection policy.

View File

@@ -12,24 +12,9 @@ exceptions. See [D&D Module Internals](dnd.md) rather than adding them here.
A module is a typed implementation registered for one pipeline stage. Its A module is a typed implementation registered for one pipeline stage. Its
`ModuleSpec` is the public-to-the-framework declaration of its stable key, `ModuleSpec` is the public-to-the-framework declaration of its stable key,
stage, execution class, required and provided capabilities, artifact kind, and stage, required and provided capabilities, artifact kind, and accepted
accepted reference slots. The execution class states whether a module is reference slots. The framework uses that declaration to resolve a configured
`deterministic` or `llm_backed`; registries retain it for catalog inspection and binding before it builds the implementation.
resolved-pipeline debug data without constructing the module. The framework
uses the declaration to resolve a configured binding before it builds the
implementation. After selection, the resolver applies profile inheritance only
to bindings whose declared execution class is `llm_backed` and rejects a
binding-specific profile on a deterministic module. The user-facing precedence
contract belongs in [Configuration](../config.md#pipelines).
An eligible LLM-backed chunk, extract, merge, or normalize producer may also
declare correction protocol `single_response_v1`. That declaration is a
promise that the implementation accepts one attempt-local semantic correction
and returns an owned copy of the exact one model response that directly
controlled the candidate. It must forward correction only to its structured
completion request; it must not manufacture prior-response material by
serializing a normalized artifact or expose opaque application IDs. Input,
output, validator, and deterministic specs cannot declare the protocol.
Implementations that accept options must provide both an option validator and Implementations that accept options must provide both an option validator and
a builder. The validator is used while resolving configuration; the builder a builder. The validator is used while resolving configuration; the builder
@@ -45,118 +30,45 @@ they need, register each leaf implementation, and add any family-owned assets
or default validator chains. They return contextual errors so production or default validator chains. They return contextual errors so production
composition fails at startup rather than at the first run. composition fails at startup rather than at the first run.
A validator that returns a completed rejection must supply two separate
bounded values: a stable `ReasonCode` for provenance and actionable
`CorrectionGuidance` for the producer. Guidance identifies the semantic defect
and the constraints on one complete corrected replacement. It must not contain
validator keys, diagnostic paths, opaque application IDs, or other internal
identifiers. An operator-facing `Message` may explain the same event, but the
framework never copies it into a model request. Missing or invalid guidance is
a validator contract failure.
An artifact family can register an optional typed evidence projector alongside An artifact family can register an optional typed evidence projector alongside
its codec. The projector returns defensive copies of the artifact's direct its codec. The projector returns defensive copies of the artifact's direct
generic source references and must use the codec's exact Go type. It does not generic source references and must use the codec's exact Go type. It does not
interpret surrounding context or publish files; the pipeline validates the interpret surrounding context or publish files; the pipeline validates the
capability during preparation and the output boundary owns publication. See capability during preparation and the output boundary owns publication. See
the [Published Evidence Context contract](../integrations/evidence-context.md) the [Published Evidence Context contract](../integrations/evidence-context.md)
for the durable source-unit excerpt. Lane artifacts retain citation and lane for the durable result.
provenance; the framework does not add either to that published excerpt.
An artifact family is broader than a module: it owns the cohesive domain
feature across its artifact type, codec, stage modules, validators, prompt
policy, schemas, identity helpers, and reference projections. An extractor and
normalizer in one artifact family remain independently registered modules in
their respective pipeline stages. This ownership vocabulary does not create a
new registry or change the fixed pipeline.
## Production Composition ## Production Composition
Production composition is intentionally split by family: Production composition is intentionally split by family:
- The generic registrar provides the unit chunker, generic JSON validators, - The generic registrar provides the unit chunker, generic JSON validators,
JSON output encoder, and shared semantic-reconciliation prompt and response and JSON output encoder.
schema assets.
- The Seriatim registrar provides the transcript input adapter. Its external - The Seriatim registrar provides the transcript input adapter. Its external
input behavior is defined by the [Seriatim contract](../integrations/seriatim.md). input behavior is defined by the [Seriatim contract](../integrations/seriatim.md).
- The D&D registrar provides its codecs, extractors, mergers, normalizers, - The D&D registrar provides its codecs, extractors, mergers, normalizers,
validators, prompt assets, fallback profile asset, and default chains. Its behavioral conventions validators, prompt assets, and default chains. Its behavioral conventions
are documented in [D&D Module Internals](dnd.md). are documented in [D&D Module Internals](dnd.md).
The CLI owns the composition that invokes these registrars. A module package The CLI owns the composition that invokes these registrars. A module package
may register its own family but must not assemble the CLI or make framework may register its own family but must not assemble the CLI or make framework
packages depend on production extensions. packages depend on production extensions.
## Semantic Reconciliation
`internal/framework/semanticreconcile` is a domain-neutral strategy used by a
typed normalize module; it is not itself a selectable stage module. A
source-backed artifact-family normalizer projects its deterministic records
into contextual candidates and owned typed record envelopes, supplies its
chosen prompt identity and resolved LLM profile, and constructs an engine with
explicit limits. The core filters invalid evidence, assigns contiguous
one-based request-local candidate numbers, renders bounded candidate and
transcript materials, invokes the structured-completion boundary, and assesses
the returned duplicate groups into a stable non-overlapping plan. Candidate
material declares the exact inclusive number range for the request. Transcript
unit IDs and evidence range endpoints remain source positions in a separate
namespace and are never valid candidate numbers.
The normalizer then applies that plan through a typed `ApplicationPolicy`. The
core preserves ungrouped records, contribution order, and provenance while the
artifact family owns group guards, field and evidence consolidation, durable
ID derivation, retry and fallback presentation, classified diagnostics, and
postconditions.
These candidate numbers are the concrete private representation of ADR-0013's
request-local handles; they do not enter the typed value or durable artifact.
Fewer than two eligible candidates skips model invocation; exceeding a
candidate or combined-material bound preserves the deterministic result under
the family's fallback policy. Provider, transport, cancellation, and
context-construction failures remain execution errors.
When the engine actually makes a proposal call, its typed result carries the
owned exact proposal response under the same correction contract as other
eligible producers. Deterministic skip, limit, and fallback outcomes carry no
model candidate, so a later rejection applies terminal policy without spending
an ineffective semantic retry.
When proposal assessment or typed group application rejects a structurally
valid group, the normalizer may return its safe partial value with a
module-requested retry. A feedback-capable directive supplies bounded
model-facing correction guidance separately from operator diagnostics and
retains the exact proposal response as its candidate. The shared stage retry
mechanism appends that response and guidance to a fresh complete request. A
structurally invalid completion has no valid candidate and therefore requests a
feedback-free fresh attempt. On exhaustion, only the final safe fallback and
its bounded process diagnostic advance to validation.
The core supplies a conservative generic prompt and the single private
response schema. A domain prompt may substitute its semantic instructions but
mounts the core-owned protocol and candidate/transcript presentation assets.
Prompt, schema, policy, and limit identities participate in manifest metadata
and checkpoint fingerprints. The generic registrar owns production
registration of those shared assets; a consuming domain registrar owns only
its domain prompt.
## Adding Or Changing A Module ## Adding Or Changing A Module
1. Choose the pipeline stage and the typed artifact boundary. Put external 1. Choose the pipeline stage and the typed artifact boundary. Put external
input or durable artifact formats in the relevant integration contract, input or durable artifact formats in the relevant integration contract,
not in this guide or in a private LLM response type. not in this guide or in a private LLM response type.
2. Define a stable `ModuleSpec` with an explicit execution class, the exact 2. Define a stable `ModuleSpec` with the exact capabilities and reference
capabilities, and reference slots needed for the operation. Model a slots needed for the operation. Model a producer/consumer handoff as an
producer/consumer handoff as an artifact-compatible slot; configuration artifact-compatible slot; configuration then chooses an external file or a
then chooses an external file or a generated binding. generated binding.
3. Implement strict option decoding, construction, and the typed stage 3. Implement strict option decoding, construction, and the typed stage
interface. Preserve caller ownership: do not retain mutable request data interface. Preserve caller ownership: do not retain mutable request data
and return defensive copies where an implementation exposes stored data. and return defensive copies where an implementation exposes stored data.
If declaring correction capability, forward the request correction and
retain only the exact validated response that controlled the result.
4. Register the module through its typed registry helper and add it to the 4. Register the module through its typed registry helper and add it to the
owning family registrar. Add a default validator chain only when that owning family registrar. Add a default validator chain only when that
family owns the behavior; otherwise require an explicit compatible chain. family owns the behavior; otherwise require an explicit compatible chain.
Every rejection path in a validator must provide actionable correction
guidance while retaining its stable internal reason code.
5. Update the selectable-key and chain reference in 5. Update the selectable-key and chain reference in
[Configuration](../config.md#production-module-keys), the applicable [Configuration](../config.md#production-module-keys), the applicable
integration contract, and focused tests. Keep the configuration document integration contract, and focused tests. Keep the configuration document

View File

@@ -25,13 +25,10 @@ physical state roots.
| Area | Implemented owners | Responsibility | | Area | Implemented owners | Responsibility |
| --- | --- | --- | | --- | --- | --- |
| Executable and command boundary | **cmd/notarius**, **internal/cli** | Process entry, command dispatch, configuration discovery, production composition, runtime collaborator setup, durable file placement, and user-facing reporting. | | Executable and command boundary | **cmd/notarius**, **internal/cli** | Process entry, command dispatch, configuration discovery, production composition, runtime collaborator setup, durable file placement, and user-facing reporting. |
| Build information | **internal/buildinfo** | Resolves a stable linked release tag or build metadata for the diagnostic root version command. | | Configuration | **internal/core/config** | Defaults, strict YAML parsing, environment overrides, structural validation, effective resolution, redaction, and resolved-composition summaries. |
| Configuration | **internal/core/config** | ConfigKit document definition, source-aware normalization, runtime conversion, structural validation, safe inspection, effective resolution, and resolved-composition redaction. |
| Generic models | **internal/core/source**, **internal/core/artifacts**, **internal/framework/contracts** | Source documents and chunks, manifests and provenance, plus typed artifact, reference, validation, output, and structured-completion contracts. | | Generic models | **internal/core/source**, **internal/core/artifacts**, **internal/framework/contracts** | Source documents and chunks, manifests and provenance, plus typed artifact, reference, validation, output, and structured-completion contracts. |
| Pipeline framework | **internal/framework/pipeline** | Registries, profile and reference resolution, typed preparation, validation, retry coordination, ordered execution, handoff, and result assembly. | | Pipeline framework | **internal/framework/pipeline** | Registries, profile and reference resolution, typed preparation, validation, retry coordination, ordered execution, handoff, and result assembly. |
| LLM and prompt runtime | **internal/framework/llm**, **internal/framework/promptfs** | Provider-neutral structured completions, scheduling, profile recording, prompt assets, schema registration, and credential-shaped-value redaction. | | LLM and prompt runtime | **internal/framework/llm**, **internal/framework/promptfs** | Provider-neutral structured completions, scheduling, profile recording, prompt assets, schema registration, and credential-shaped-value redaction. |
| Semantic reconciliation | **internal/framework/semanticreconcile** | Bounded source-backed candidate preparation, request-local handle proposals, deterministic assessment, typed plan application, and reconciliation identity metadata; see [Module Internals](modules.md#semantic-reconciliation) and [D&D Module Internals](dnd.md#semantic-registry-reconciliation). |
| Embedded LLM content | **assets** | Read-only centralized LLM-facing content, scoped by its consuming package; see [LLM Runtime](llm.md#prompt-and-schema-assets) and [D&D Module Internals](dnd.md#prompt-construction). |
| Runtime state | **internal/core/fileio**, **internal/core/debugbundle**, **internal/framework/checkpoint**, **internal/framework/chunkplan**, **internal/framework/chunkmap**, **internal/framework/debug** | Confined atomic files, debug bundles, checkpoint and chunk-plan state, accepted chunk maps, and pipeline-facing debug recording. | | Runtime state | **internal/core/fileio**, **internal/core/debugbundle**, **internal/framework/checkpoint**, **internal/framework/chunkplan**, **internal/framework/chunkmap**, **internal/framework/debug** | Confined atomic files, debug bundles, checkpoint and chunk-plan state, accepted chunk maps, and pipeline-facing debug recording. |
| Production extensions | **internal/modules/generic**, **internal/modules/seriatim**, **internal/modules/dnd** | Domain-neutral extensions, Seriatim input support, and D&D extraction families registered into the production catalog. | | Production extensions | **internal/modules/generic**, **internal/modules/seriatim**, **internal/modules/dnd** | Domain-neutral extensions, Seriatim input support, and D&D extraction families registered into the production catalog. |
@@ -51,9 +48,8 @@ the CLI composition boundary.
composition, and path safety. composition, and path safety.
- [LLM Runtime](llm.md): structured completion, scheduling, prompt assets, - [LLM Runtime](llm.md): structured completion, scheduling, prompt assets,
profiles, and secret handling. profiles, and secret handling.
- [Module Internals](modules.md): generic extension registration, artifact - [Module Internals](modules.md): generic extension registration, module
families, module construction, semantic reconciliation, validation, and construction, validation, and reference mechanics.
reference mechanics.
- [D&D Module Internals](dnd.md): shared D&D extractor conventions, generated - [D&D Module Internals](dnd.md): shared D&D extractor conventions, generated
reference projections, and lane-specific exceptions. Durable D&D and reference projections, and lane-specific exceptions. Durable D&D and
Seriatim data shapes remain in the [integration contracts](../integrations/). Seriatim data shapes remain in the [integration contracts](../integrations/).

View File

@@ -10,11 +10,11 @@ own durable output shapes. Concrete production extensions are covered by
## Boundary ## Boundary
The pipeline framework accepts a resolved composition, registries, shared The pipeline framework accepts a resolved composition, registries, shared
dependencies, input bytes, a supplied prompt session, and state/debug dependencies, input bytes, and state/debug collaborators. It returns logical
collaborators. It returns logical output files, normalized artifacts, recorded output files, normalized artifacts, recorded rejections and warnings, manifest
rejections, grouped diagnostics, manifest provenance, and checkpoint decisions. The provenance, and checkpoint decisions. The CLI owns process arguments,
CLI owns process arguments, configuration discovery, session resolution, configuration discovery, physical roots, and placement of returned output
physical roots, and placement of returned output files. files.
The framework has one fixed shape: The framework has one fixed shape:
@@ -32,47 +32,25 @@ Resolution turns a configured pipeline profile into a **ResolvedPipeline**.
It normalizes the pipeline and lane identities, applies stage defaults, selects It normalizes the pipeline and lane identities, applies stage defaults, selects
requested lanes where that is supported, resolves validator chains, checks requested lanes where that is supported, resolves validator chains, checks
module capabilities and typed artifact compatibility, validates options, and module capabilities and typed artifact compatibility, validates options, and
assigns a deterministic resolved-composition digest. A correction protocol is assigns a deterministic resolved-composition digest. The resolved pipeline
selected from each eligible LLM-backed producer specification and becomes part contains bindings and declared reference targets, not external reference bytes.
of that resolved identity; only `single_response_v1` is currently supported. Configuration resolution supplies the selected profile and catalog; see
Preparation rejects an LLM-backed producer that combines a non-empty validator
chain with positive producer retries unless it declares that protocol. Producers
without validators or without retries remain valid without correction support.
The resolved pipeline contains bindings and declared reference targets, not
external reference bytes.
After selection, the resolver applies command, binding, and pipeline profile
precedence to LLM-backed bindings and validators only; prompt defaults remain
an empty resolved binding profile. It resolves structural output repair
separately: a binding's `structured_output_repair_attempts` value wins, then a
pipeline value applies to LLM-backed bindings and validators, and omission
leaves the prompt-owned policy intact. An explicit repair value on a
deterministic binding is rejected. Resolved bindings own copied repair values,
and these effective values are part of the digest, so execution and checkpoint
consumers do not repeat profile inheritance or configuration resolution.
Each LLM request receives its own copy of that resolved value. PromptKit spends
it only for structural correction inside one completion; the runner's binding
retry policy remains the separate outer budget for complete stage attempts.
Configuration resolution supplies the selected overlay and catalog; see
[Configuration Internals](configuration.md). [Configuration Internals](configuration.md).
External reference materialization happens before preparation. The materializer External reference materialization happens before preparation. The materializer
checks that each slot is declared by the selected module, resolves a file path checks that each slot is declared by the selected module, resolves a file path
relative to the correct configuration or working-directory origin, reads relative to the correct configuration or working-directory origin, reads
UTF-8 text, verifies media type and size limits, and retains bounded UTF-8 text, verifies media type and size limits, and retains bounded
provenance. For a positive slot limit, it reads at most the limit plus one byte provenance. A generated-artifact selector remains declared but has no bytes
and rejects overflow before retaining content. A generated-artifact selector until its producing step completes.
remains declared but has no bytes until its producing step completes.
Preparation is the construction boundary. It validates the resolved shape and Preparation is the construction boundary. It validates the resolved shape and
registry set, clones the resolved data, then constructs the input adapter, registry set, clones the resolved data, then constructs the input adapter,
chunker, stage-local validators, every typed lane, and output encoder. The chunker, stage-local validators, every typed lane, and output encoder with
prepared producer metadata preserves each selected correction protocol, and cloned options, references, and shared dependencies. It also collects stable
the resolved digest carrying that metadata participates in checkpoint identity. checkpoint fingerprints. Missing registrations, incompatible typed entries,
Each registered builder receives its own cloned build request immediately nil implementations, and constructor failures are reported before source
before its module-owned code runs. Preparation also collects stable checkpoint parsing or any stage operation begins.
fingerprints. Missing registrations, incompatible typed entries, nil
implementations, and constructor failures are reported before source parsing
or any stage operation begins.
An output encoder can opt into source-evidence publication through its output An output encoder can opt into source-evidence publication through its output
policy. Preparation keeps the configured lane allowlist and active lanes policy. Preparation keeps the configured lane allowlist and active lanes
@@ -97,21 +75,10 @@ codec, checks its complete schema and media identity, and records a content
digest plus bounded producer provenance. A missing, ambiguous, invalid, or digest plus bounded producer provenance. A missing, ambiguous, invalid, or
incompatible producer prevents the consumer step from starting. incompatible producer prevents the consumer step from starting.
Resolution receives only exact stage-and-lane operational reference overrides.
The CLI may offer broader pipeline- or lane-scoped selectors, but expands and
arbitrates those before entering the framework. External overrides are applied
after configured external defaults and local bindings. They cannot coexist
with a generated binding for the same target and slot, and external unbinds do
not remove generated handoffs.
## Execution And Ordering ## Execution And Ordering
The runner validates its input, installs no-op state collaborators when none The runner validates its input, installs no-op state collaborators when none
were supplied, and serially performs source parsing and chunk-plan selection. were supplied, and serially performs source parsing and chunk-plan selection.
It transports the supplied session unchanged to prompt-facing operations and
run-manifest metadata; it neither derives a session nor substitutes a parsed
source document identifier. The public session contract is owned by the
[CLI reference](../cli.md#run).
An accepted plan is materialized into source-addressed chunks and passes the An accepted plan is materialized into source-addressed chunks and passes the
configured chunk validators before any lane runs. A chunk rejection is a configured chunk validators before any lane runs. A chunk rejection is a
recorded pipeline outcome: lanes do not start, but the output stage can encode recorded pipeline outcome: lanes do not start, but the output stage can encode
@@ -139,59 +106,18 @@ for started workers, and prevents output encoding.
Every chunk, extract, merge, and normalize candidate passes its resolved Every chunk, extract, merge, and normalize candidate passes its resolved
validator chain. Validators receive immutable canonical input appropriate to validator chain. Validators receive immutable canonical input appropriate to
their target: chunks, codec-decoded typed candidates, or serialized codec their target: chunks, typed values, or serialized codec bytes. They may
bytes. Each typed validator receives a newly decoded value from the one approve, approve with warnings, reject, or fail. A rejection is an ordinary
candidate serialization for that attempt, while serialized validators receive pipeline result; a validator error is a framework error.
separately owned representation bytes and schema metadata. They may approve,
approve with warnings, reject, fail, or be skipped when a runtime prerequisite
is unavailable. The shared executor settles every configured validator in
order. A failed LLM-backed validator retries only itself against the same
immutable candidate; it does not regenerate the producer or alter the
validator request. Rejections stop that validator, while other configured
validators still run. The executor retains ordered results, bounded
deduplicated correction guidance from rejections, and only the final exhausted
failure outcome for each validator. The correction builder keeps first
occurrence order, omits internal reason codes, validator names, and operator
messages, and requests one complete replacement. Missing guidance or an
oversized aggregate is a framework contract error; guidance is never inferred
or truncated.
The runner applies the binding's retry policy around a stage operation and its The runner applies the binding's retry policy around a stage operation and its
complete validation chain. It preserves terminal diagnostics only from the final accepted complete validation chain. It preserves warnings only from the final accepted
or rejected attempt, plus one fixed validation-incomplete warning per validator whose execution or rejected attempt. Cancellation stops retries. Normalizer-specific retry
budget was exhausted under `warn_continue`. Cancellation stops retries. directives consume this same budget and validate any final safe fallback through
Normalizer-specific retry directives consume this same budget and validate any the normalizer chain.
final safe fallback through the normalizer chain. A directive may carry bounded
correction guidance only when it also exposes the exact latest
`single_response_v1` candidate. The state machine then uses the same replacement
request shape as validator correction. A directive without guidance clears any
prior correction and starts a fresh attempt, which preserves structural retry
behavior when no valid response exists.
The artifact-neutral producer-attempt state machine owns that shared budget,
attempt provenance, semantic-correction material, and terminal-policy
selection. It accepts producer and complete-validation closures, so artifact
materialization, cache handling, checkpoints, and debug output stay at the
operation boundary. It distinguishes operational, structural, module-requested,
and validator-semantic retries. Model feedback from either semantic source is
available only for a valid latest `single_response_v1` candidate. A
deterministic or no-model validator rejection instead settles the semantic
policy immediately, while a feedback-free module directive remains an ordinary
fresh retry. Structural-output errors alone use the structural policy, and
validation failure without rejection settles the validator-failure policy
without regenerating the producer.
Chunk planning uses this state machine for generated plans. A rejected or
validation-incomplete automatic cache hit is not model material and therefore
falls through to a fresh initial generation at producer attempt one; it neither
receives a correction, consumes retry budget, promotes cached-candidate
diagnostics, nor overwrites the stored record. An incomplete cache validation
under `fail_run` terminates instead. Only a newly generated, completely
validated plan is published to the chunk-plan store. Rejected plans never
advance, and validation-incomplete plans remain unpublishable.
After terminal lane work, the runner assembles manifest provenance, normalized After terminal lane work, the runner assembles manifest provenance, normalized
artifacts, rejections, final grouped diagnostics, and an optional accepted chunk map. When an artifacts, rejections, warnings, and an optional accepted chunk map. When an
output policy selected evidence lanes, it decodes accepted serialized normalize output policy selected evidence lanes, it decodes accepted serialized normalize
outputs through their registered codecs and invokes the prepared typed outputs through their registered codecs and invokes the prepared typed
projectors. Rejected or absent lanes contribute nothing. This reconstruction is projectors. Rejected or absent lanes contribute nothing. This reconstruction is
@@ -202,15 +128,6 @@ The CLI publishes those files only after the runner returns without a framework
error. Logical file names and schemas are defined by the [output integration error. Logical file names and schemas are defined by the [output integration
contracts](../integrations/). contracts](../integrations/).
For every completed producer disposition, the runner projects one bounded
validation summary to the manifest, the affected rejection when present, and
the CLI result receipt. The summary records status, configured-order rejecting
validators and reason codes, incomplete validators, producer-attempt count,
and terminal action. It contains no operator message, correction guidance, or
model response. `complete`, `rejected`, and `incomplete` describe the final
candidate disposition; a run-level `incomplete` status indicates at least one
current-run output advanced under `warn_continue`.
## Checkpoint And Debug Hooks ## Checkpoint And Debug Hooks
The runner receives checkpoint and debug interfaces rather than roots. It The runner receives checkpoint and debug interfaces rather than roots. It
@@ -220,26 +137,10 @@ handoff. Generated-reference dependencies participate in checkpoint decisions.
Selective recomputation can require a canonical accepted normalized predecessor Selective recomputation can require a canonical accepted normalized predecessor
before a dependent lane starts. before a dependent lane starts.
The runner writes successful checkpoint artifacts only after complete accepted
validation. Chunk plans follow the same rule for publication. A rejection,
invalid structured response, or incomplete validation is never reusable state;
the current run may still hand off an otherwise valid `warn_continue` result
according to its terminal policy. The runner carries private reuse eligibility
through extract, merge, normalize, and generated-reference handoff. Any stage
derived from incomplete validation skips both checkpoint lookup and all
checkpoint publication even when that stage's own validation completes.
External references and fully validated generated references remain eligible.
Attempt debug records retain safe kind,
validator, repair-usage, policy, and terminal-decision provenance. Full
assistant and correction content remains confined to the requested detailed
LLM trace.
Debug recording is attempt-scoped and application-owned. A failure to persist Debug recording is attempt-scoped and application-owned. A failure to persist
required debug data is a framework error. State roots, persistence, reason-code required debug data is a framework error. State roots, persistence, reason-code
meanings, resume, and cleanup are intentionally owned by meanings, resume, and cleanup are intentionally owned by
[Run State Internals](state.md) and [Operations](../operations.md). [Run State Internals](state.md) and [Operations](../operations.md).
Extract-validator trace scopes include the current chunk ordinal so concurrent
chunks cannot overwrite one another's validator attempts or LLM artifacts.
## Invariants To Preserve ## Invariants To Preserve

View File

@@ -1,9 +1,5 @@
# Run State Internals # Run State Internals
LLM debug response artifacts and their safe per-call references retain an
optional terminal execution summary when one is available. This metadata is
diagnostic only and does not change receipts, checkpoints, or aggregate usage.
This document describes the implementation collaborators behind output, cache, This document describes the implementation collaborators behind output, cache,
and debug state. User-visible fields belong in [Configuration](../config.md), and debug state. User-visible fields belong in [Configuration](../config.md),
and physical layout, retention, recovery, reason codes, and cleanup belong in and physical layout, retention, recovery, reason codes, and cleanup belong in
@@ -36,31 +32,19 @@ owned by the [Accepted Chunk Map contract](../integrations/chunk-map.md).
`internal/framework/chunkplan` owns source-addressed plan storage, validation, `internal/framework/chunkplan` owns source-addressed plan storage, validation,
and atomic publication. Its store is constructed only when the selected mode is and atomic publication. Its store is constructed only when the selected mode is
not `bypass`. It preserves rooted entry checks while creating new cache roots not `bypass`.
and source-digest directories with requested mode `0777`, then creates an
atomic temporary file through rooted `O_CREATE|O_EXCL` with requested mode
`0666` before sync and rename.
`internal/framework/checkpoint` owns checkpoint identity, manifests, payload `internal/framework/checkpoint` owns checkpoint identity, manifests, payload
codecs, loader, and recorder. The CLI constructs a recorder whenever checkpoint codecs, loader, and recorder. The CLI constructs a recorder whenever checkpoint
recording is enabled and constructs a loader only for a `--resume` invocation. recording is enabled and constructs a loader only for a `--resume` invocation.
The recorder writes manifest and payload JSON through
`fileio.WriteJSONWithProcessPermissions`, whose confined atomic writer creates
new directories with requested mode `0777` and temporary files with requested
mode `0666` before publication.
Identity incorporates explicit stable semantic fingerprints collected from Identity incorporates explicit stable semantic fingerprints collected from
prepared modules and validators in addition to configuration, input, prepared modules and validators in addition to configuration, input,
references, runtime overrides, observed LLM profiles, and the LLM runtime's references, runtime overrides, and LLM profiles.
non-secret effective profile-source identity. A profile source change therefore
causes a cold miss even when the configured profile ID remains unchanged.
The serialized The serialized
`workspace_schema_version` identifiers are frozen wire-compatibility fields; `workspace_schema_version` identifiers are frozen wire-compatibility fields;
they do not describe a current public state surface. they do not describe a current public state surface.
Ordered-step lane checkpoints include the step identity in their storage scope. Ordered-step lane checkpoints include the step identity in their storage scope.
Accepted step and lane identities are encoded injectively before becoming
filesystem path components, while ordinary safe identifiers retain their
readable paths.
When a later lane consumes a generated artifact, its dependency fingerprints When a later lane consumes a generated artifact, its dependency fingerprints
include the producer's artifact kind, complete schema identity, media type, include the producer's artifact kind, complete schema identity, media type,
canonical content digest, and size. Ordinary resume compares those fingerprints canonical content digest, and size. Ordinary resume compares those fingerprints
@@ -75,13 +59,12 @@ Ordinary resume loads extract, merge, and normalize checkpoints progressively
and may execute later lane stages after an earlier cache miss. Selective and may execute later lane stages after an earlier cache miss. Selective
recomputation instead asks the loader for the required producer's accepted recomputation instead asks the loader for the required producer's accepted
normalize artifact. That lookup reuses the existing normalize files, requires normalize artifact. That lookup reuses the existing normalize files, requires
workspace schema v4 plus an exact non-empty invocation identity, and deliberately workspace schema v3 plus an exact non-empty invocation identity, and deliberately
does not require extract or merge checkpoint files or dependency fingerprints. does not require extract or merge checkpoint files or dependency fingerprints.
The runner performs canonical codec and producer-provenance validation before The runner performs canonical codec and producer-provenance validation before
cloning the artifact into normal step output. Success restores only stored cloning the artifact into normal step output. Success restores only stored
normalize diagnostics and emits one normalize decision; failure retains the normalize warnings and emits one normalize decision; failure retains the files,
files, records the decision, and stops without executing the producer or records the decision, and stops without executing the producer or consumer.
consumer.
The loader assigns a typed category and reason code at each validation site; The loader assigns a typed category and reason code at each validation site;
diagnostic prose is not classified after the fact. The runner then applies diagnostic prose is not classified after the fact. The runner then applies
@@ -97,10 +80,7 @@ and debug-summary JSON.
canonical operator-facing reason-code reference. canonical operator-facing reason-code reference.
`internal/core/fileio` provides confined atomic file writes used by state `internal/core/fileio` provides confined atomic file writes used by state
collaborators. Its exact-mode writer remains available to output and debug collaborators. The chunk-plan store retains its stronger entry validation.
callers; its process-permission writer is used by checkpoints. The chunk-plan
store retains its stronger rooted entry validation. [Operations](../operations.md#cache-provisioning)
owns cache mode and provisioning guidance.
The CLI constructs selective-recomputation policy from resolved generated The CLI constructs selective-recomputation policy from resolved generated
artifact dependencies. It forces the selected step and transitive consumers, artifact dependencies. It forces the selected step and transitive consumers,
@@ -112,7 +92,7 @@ owns the operator workflow and stable reason-code meanings.
`internal/core/debugbundle` allocates an explicitly requested per-run bundle `internal/core/debugbundle` allocates an explicitly requested per-run bundle
with `summary/` and `trace/` roots. `SummaryWriter` persists redacted command, with `summary/` and `trace/` roots. `SummaryWriter` persists redacted command,
resolution, run, final grouped diagnostic, and failure artifacts. `internal/framework/debug` resolution, run, warning, and failure artifacts. `internal/framework/debug`
implements the pipeline-facing trace recorder under the trace root. implements the pipeline-facing trace recorder under the trace root.
The CLI allocates a bundle before pipeline resolution and treats requested The CLI allocates a bundle before pipeline resolution and treats requested

View File

@@ -5,23 +5,6 @@ This is the canonical guide for operating Notarius runtime state. The
[Configuration](config.md) owns fields, defaults, and precedence. Maintainers [Configuration](config.md) owns fields, defaults, and precedence. Maintainers
who need implementation mechanics should read [Run State Internals](internal/state.md). who need implementation mechanics should read [Run State Internals](internal/state.md).
## Source Deployment
Linux is the supported deployment platform. Install a pinned source release
with the Go version declared in `go.mod` (currently Go 1.25.5):
~~~sh
GOWORK=off go install \
gitea.maximumdirect.net/eric/notarius/cmd/notarius@vMAJOR.MINOR.PATCH
~~~
Pin the exact tag in deployment automation rather than following a branch.
Use [`notarius --version`](cli.md#command-summary) as a diagnostic after
installation; its syntax and semantics are owned by the [CLI reference](cli.md).
The maintainer publication process, including tag guards and verification,
belongs to [Source Releases](release.md). macOS builds are best-effort for
development, and Windows is unsupported.
## State Surfaces ## State Surfaces
Each run can use independent roots with different retention and access-control Each run can use independent roots with different retention and access-control
@@ -50,88 +33,15 @@ directory selects a per-user root:
The field definitions and configuration examples are in [Configuration](config.md). The field definitions and configuration examples are in [Configuration](config.md).
On supported Unix systems, output directories and files are created with On supported Unix systems, output directories and files are created with
requested modes **0755** and **0644**; debug directories and files use requested modes **0755** and **0644**. Chunk-plan, checkpoint, and debug
**0700** and **0600**. The operating system's umask may impose stricter output directories and files use **0700** and **0600**. The operating system's umask
or debug modes. Chunk-plan and checkpoint caches instead request maximum modes may impose stricter output modes. Cache and debug roots may contain sensitive
of **0777** for new directories and **0666** for new files, including atomic source-derived data, so provision them for one trusted account or service. An
temporary files. The kernel applies the service's umask and applicable default output bundle can also contain source content when its JSON output enables
ACLs when those entries are created.
## Cache Provisioning
Cache locations select where Notarius writes, not who can access them. A
private service can use a restrictive umask:
~~~sh
umask 0077
notarius run example --config /etc/notarius/config.yml --input /srv/input.txt
~~~
A trusted-group deployment can provision a setgid cache root and use a
group-oriented umask (or an equivalent default ACL):
~~~sh
install -d -m 2770 -g notarius-operators /srv/notarius/chunk-plans /srv/notarius/checkpoints
umask 0007
notarius run example --config /etc/notarius/config.yml --input /srv/input.txt
~~~
Setgid and default ACLs control inherited group ownership and access policy;
Notarius does not copy a parent directory's literal mode bits. Cache state is
reconstructible but can contain source-derived data, so operators own its
access and retention policy. Existing cache inodes retain their prior modes
after an upgrade. Recreate a cache beneath correctly provisioned parents, or
stop writers and perform a deliberate administrative migration when a
whole-tree change is required.
Cache publication remains atomic and stored records remain validated. Those
guarantees do not add cross-process locking or transactional coordination for
simultaneous writers to one checkpoint identity.
Output bundles can also contain source content when JSON output enables
evidence publication. Apply an appropriate umask and output-root access policy evidence publication. Apply an appropriate umask and output-root access policy
before enabling that option; the requested output modes alone may not be before enabling that option; the requested output modes alone may not be
suitable for transcript-bearing bundles. suitable for transcript-bearing bundles.
## PromptKit Profile Deployment
Profile deployment has four distinct layers:
| Layer | Owner | Operational role |
| --- | --- | --- |
| Prompts and schemas | Notarius module families | Embedded request and structured-output definitions. They are not deployment profile files. |
| Fallback profiles | Notarius module families | Embedded application defaults, including D&D's `dnd-extraction` profile. |
| Built-in profiles | PromptKit | Upstream catalog entries available when no higher-precedence source defines an ID. |
| Operator profiles | Deployment filesystem | Complete environment-specific definitions selected by `promptkit.profile_file` or `promptkit.profile_dir`. |
The maintained D&D pipeline uses the workload ID `dnd-extraction`. The
embedded fallback makes that ID usable without an operator file. Production,
development, and local deployments can each install a different complete
definition for the same ID, retaining the pipeline while choosing their own
model, backend, timeout, or reasoning policy. An operator definition wins over
the fallback; it is not merged with it. The configuration field and full
precedence rules are owned by [Configuration](config.md#promptkit-profiles).
Use a profile source owned by the service account, keep it readable only by
the intended operator, and supply provider credentials through the service
environment—not in the Notarius configuration or profile YAML. The maintained
[LLM profile](../examples/llm-profiles/dnd-extraction.yml) is secret-free and
can be copied as a format starting point. Validate a deployment without a
provider call or credentials:
~~~sh
notarius config validate --config /etc/notarius/config.yml --pipeline dnd-session
~~~
An unset optional `api_key_env` reaches the provider without authorization and
may receive a 401 or 403 response.
Configured PromptKit profile paths resolve from the file that supplies their
surviving declaration. The complete example's
`./llm-profiles/dnd-extraction.yml` path is therefore relative to that example.
Use absolute paths such as `/etc/notarius/profiles/dnd-extraction.yml` for
services and containers when that is clearer for the deployment.
## Run Lifecycle ## Run Lifecycle
Use the [run command](cli.md#run) to start a pipeline. A valid invocation loads Use the [run command](cli.md#run) to start a pipeline. A valid invocation loads
@@ -139,59 +49,10 @@ and resolves configuration before module preparation and source parsing. It
then performs any permitted cache lookup, executes the pipeline, and publishes then performs any permitted cache lookup, executes the pipeline, and publishes
logical output files only after a successful runner result. logical output files only after a successful runner result.
On success, the command reports the output bundle path. A run with actionable On success, the command reports the output bundle path. A warning-bearing run
process warnings still succeeds and reports warning-group and occurrence counts still succeeds and reports its warning count on standard error. Errors and
on standard error; advisory and observation findings do not produce a warning
line. Errors and
their exit classes are defined in the [CLI reference](cli.md#output-streams-and-exit-statuses). their exit classes are defined in the [CLI reference](cli.md#output-streams-and-exit-statuses).
## Validation Retries And Terminal Outcomes
When debug recording is enabled, per-call response artifacts include the
terminal execution summary for retry, repair, timing, and presence-aware usage
diagnosis. Normal output, receipts, and warnings do not aggregate or publish
this diagnostic metadata.
Each producer binding has one outer **retries** budget. It covers complete
producer attempts for operational failures, invalid structured output,
normalizer fallback retry, and semantic correction. It is independent from
PromptKit's structural-repair calls inside one completion, its configured
`promptkit.transient_retries` built-in HTTP attempts, and an LLM-backed
validator's own retry budget. The transient setting defaults to two additional
attempts for eligible provider statuses; set it explicitly to zero when an
operator needs one HTTP attempt per generation. A semantic correction rebuilds
the ordinary producer request and supplies only the latest rejected model
response plus aggregated validator guidance; it is not a conversation replay.
These budgets multiply only by their declared scopes. With two outer producer
retries, two transient retries, and one structural repair, a single producer
can issue up to 18 HTTP attempts: three producer attempts, each with an initial
generation and one repair generation, each allowing three HTTP attempts. This
is an illustration, not a total-deadline or exactly-once guarantee. Every
built-in HTTP attempt has its own generation timeout; caller cancellation
remains the overall authority and interrupts Retry-After waiting. The shared
Notarius scheduler keeps its permit through those waits, so retry budgets can
increase latency and provider cost. Observed usage can remain incomplete when
providers omit accounting fields.
After the applicable budgets are exhausted, the resolved
[`validation_policy`](config.md#pipelines) determines the result. Structural
failure and semantic rejection normally fail the run; an explicit
`reject_output` records a rejection and allows unrelated work to finish. A
validator execution failure normally uses `warn_continue`, which keeps an
otherwise accepted result in the current run with `incomplete` validation
provenance. It emits one bounded warning for every validator whose execution
budget was exhausted. A corrected result that later passes validation does not
retain diagnostics from abandoned attempts.
Treat a successful process exit as a completed run, not as proof that every
candidate was fully validated. Inspect the receipt's `validation_status`,
`validation_summaries`, rejection count, and warning group and occurrence
counts when an orchestrator requires complete validation. The durable fields
and their meanings are owned by the
[run-result receipt](integrations/run-result.md) and
[published JSON output contract](integrations/json-output.md).
## Output Bundles ## Output Bundles
Each successful run receives a generated safe run identifier and writes beneath: Each successful run receives a generated safe run identifier and writes beneath:
@@ -213,9 +74,9 @@ are defined in [Accepted Chunk Map](integrations/chunk-map.md). An optional
[evidence context](integrations/evidence-context.md) contains source-unit text [evidence context](integrations/evidence-context.md) contains source-unit text
and metadata. It is not a cache or debug artifact: retain it with the output and metadata. It is not a cache or debug artifact: retain it with the output
bundle only for as long as consumers need it, and apply source-content access bundle only for as long as consumers need it, and apply source-content access
controls to the entire bundle. Its selected source-unit excerpt may include controls to the entire bundle. Selected lanes may collectively cite most of a
every source unit once when coverage is broad or its configured window is transcript, so a broad allowlist can make the evidence artifact nearly as
large, so do not assume a byte or token reduction or reduced sensitivity. sensitive and large as the source itself.
## Chunk-Plan Cache ## Chunk-Plan Cache
@@ -241,9 +102,7 @@ The configured cache mode controls one invocation:
A reused plan is still materialized and validated against the current source. A reused plan is still materialized and validated against the current source.
If a prior plan no longer gives acceptable results, use a refresh run rather If a prior plan no longer gives acceptable results, use a refresh run rather
than editing cache files. Deleting a plan is recoverable but can repeat costly than editing cache files. Deleting a plan is recoverable but can repeat costly
chunking work. A plan accepted only under incomplete validation is not chunking work.
published, and a rejected cache hit falls through to ordinary generation rather
than becoming a correction candidate.
## Checkpoint Recording, Resume, And Recompute ## Checkpoint Recording, Resume, And Recompute
@@ -258,21 +117,9 @@ compatible recorded work. A resume request fails when checkpoint recording is
disabled. Without **--resume**, a recording-enabled run executes normally and disabled. Without **--resume**, a recording-enabled run executes normally and
does not load checkpoint state. Compatibility includes the resolved pipeline, does not load checkpoint state. Compatibility includes the resolved pipeline,
input, selected lanes, runtime overrides, reference provenance, LLM-profile input, selected lanes, runtime overrides, reference provenance, LLM-profile
provenance, the effective PromptKit profile-source fingerprint, and provenance, and prepared-component fingerprints. A changed identity produces a
prepared-component fingerprints. When a local PromptKit backend is configured, cold miss; Notarius does not migrate, rewrite, or delete older checkpoint
compatibility also includes a non-secret fingerprint of its endpoint. Changing directories.
profile content or the local endpoint causes a cold miss; changing only the
local concurrency limit does not. A changed identity produces a cold miss;
Notarius does not migrate, rewrite, or delete older checkpoint directories.
Reasoning-effort inheritance, replacement, and explicit clearing are distinct
runtime identities, so checkpoints created under one state are not reused by
either of the others.
Only accepted, completely validated chunk, extract, merge, and normalize
results are checkpointed for reuse. Rejected, structurally invalid, and
validation-incomplete producer results remain non-reusable, even when a
`warn_continue` result advanced during its original run. A resumed invocation
therefore reruns that producer rather than treating degraded state as accepted.
Checkpoint state is confined below an identity-specific path: Checkpoint state is confined below an identity-specific path:
@@ -327,19 +174,12 @@ Only a [debug-enabled run](cli.md#run) creates a bundle:
trace/ trace/
~~~ ~~~
The summary contains redacted invocation and resolution information, including The summary contains redacted invocation and resolution information plus run,
`effective-config.yml` and the separately redacted resolved pipeline, plus run, warning, checkpoint, chunk-plan, and terminal reporting artifacts. The trace
final grouped diagnostic, checkpoint, chunk-plan, and terminal reporting artifacts. Attempt contains allowlisted application diagnostic records and can include source or
terminal records contain bounded attempt kinds, validator outcomes, policy, derived application data. Neither surface is a cache input. Do not treat a
decision, PromptKit repair count, and usage; they do not contain assistant debug bundle as safe to share merely because its configuration summary is
responses or complete correction messages. The trace contains allowlisted redacted.
application diagnostic records and can include source, model, and correction
content. Neither surface is a cache input. Do not treat a debug bundle as safe
to share merely because its configuration summary is redacted. Invocation
metadata omits reasoning effort when it is inherited, records the replacement
value when one is supplied, and records an empty value when inherited reasoning
was explicitly cleared. It records `config_overlay` only when a non-base
configuration overlay was selected.
Notarius never creates debug state without an explicit request and never Notarius never creates debug state without an explicit request and never
automatically deletes a requested bundle. If allocation succeeds, the command automatically deletes a requested bundle. If allocation succeeds, the command
@@ -367,74 +207,10 @@ or automatic cleanup command.
## Operational Limits ## Operational Limits
Provider execution settings and the generation timeout come from the selected Provider retries and timeouts are supplied by the selected Scriptorium profile.
PromptKit profile. The invocation-only **--reasoning-effort** and Module retry settings and concurrency limits are configuration contracts; see
**--clear-reasoning-effort** controls may replace or clear that profile setting [module bindings](config.md#module-bindings-and-validators) and
for all LLM-backed calls in one run without changing the profile. PromptKit
v0.11.0 also retries eligible transient HTTP failures with its built-in default
of two additional attempts per generation. This retry work is separate from
structural output repair, which happens within one structured-completion call. Its
effective `structured_output_repair_attempts` limit is resolved from the
selected binding, then the pipeline, then the prompt declaration; see
[module bindings](config.md#module-bindings-and-validators). This is distinct
from Notarius binding **retries**, which rerun the complete module operation
and validation chain and do not consume or replenish the structural-repair
limit. The maintained production prompts declare one repair attempt, paid only
after a structural failure. One structured completion with repair budget **R**
contains at most **R + 1** serial generations. Each generation can now contain
more than one provider call when PromptKit retries an eligible transient
failure. LLM-backed validators have their own corresponding invocation and
repair budgets. These are bounds on attempted work, not a promise that every
call reaches a provider.
Timeouts are layered. Caller cancellation is the outer authority. A positive
effective generation timeout adds an inner request deadline to each built-in
generation attempt, while zero disables only that generation deadline. The HTTP
client timeout remains a transport-wide cap. Notarius does not add another
timeout around PromptKit. Retries and repairs are serial within the same caller
context, so their worst-case latency and cost can exceed the structural-repair
bound above; provision run deadlines and provider budgets accordingly.
Credentials remain optional unless the selected
PromptKit profile requires one, in which case preparation fails before a
provider call when its configured credential is unavailable.
The pinned upstream boundary and profile-format links are in
[PromptKit Integration](integrations/pkg-promptkit.md).
Concurrency has two independent layers. Notarius **total_llm** defaults to 16
and is the application-wide provider-call limit shared by all backends,
modules, retries, and validators. PromptKit may impose a narrower admission
limit for the selected backend. The effective active-generation bound is the
intersection of the Notarius limit, any PromptKit backend limit, and work made
available by the pipeline. Built-in OpenRouter profiles use PromptKit's
upstream backend limit; endpoint-only profiles have no PromptKit backend limit
and remain bounded by Notarius. For the configured local backend, a zero
**concurrency_limit** leaves only the Notarius scheduler as a call limit. A
positive value makes the effective active local-generation bound the smaller
of **total_llm** and that local limit, so a local limit of four permits no more
than four active local generations.
The Notarius scheduler admits one logical structured completion and holds that
permit while PromptKit performs its serial corrective calls. PromptKit applies
its selected-backend admission to each provider call; Notarius does not
reacquire a permit or add another scheduler for a repair.
For a positive local limit, PromptKit owns its default waiting capacity and
admission behavior. When a PromptKit backend has admitted all active and queued
work, a new call fails as capacity exhaustion before generation. The adapter
maps that failure to Notarius's existing provider-neutral capacity error and
does not retry it. The calling stage's configured retry policy applies
normally, and the run fails if those attempts are exhausted. Caller
cancellation remains authoritative. Configuration contracts are documented
under [PromptKit profiles](config.md#promptkit-profiles) and
[concurrency](config.md#concurrency-output-cache-and-debug). Extract-worker [concurrency](config.md#concurrency-output-cache-and-debug). Extract-worker
limits and actual provider-call limits are independent. Notarius writes local limits and actual provider-call limits are independent. Notarius writes local
filesystem state only; remote storage, archival, and retention automation are filesystem state only; remote storage, archival, and retention automation are
outside the implemented CLI. outside the implemented CLI.
Every run has an effective prompt session used for provider routing and run
provenance. The generated default is stable for the same input module and raw
input bytes; use [**--session-id**](cli.md#run) only when intentionally grouping
different invocations. Both generated and explicit values can be visible to
providers, manifests, checkpoints, and requested debug bundles. Do not put
credentials or other secrets in an explicit session identifier; command-line
values are not a credential mechanism.

View File

@@ -1,90 +0,0 @@
# Agent Policy
**Base Policy Version:** 1.1
**Last Updated**: 2026-09-25
**Upstream Source:** `policy` commit `45591a79004c6811950f14dca554f6ff69b7f58e`.
Notarius adaptations preserve operator choice, existing explicit contracts, and
the deployment-permissions example below.
## Purpose
Agents working in this repository must read and adhere to the following house rules.
## Implementation Discipline
Simplify the mechanism, not the user's choices.
**KEY PRINCIPLES:** Write the simplest idiomatic code that satisfies the user's
request. Do not implicitly strengthen requirements, restrictions, or guarantees
beyond what the user actually requested.
**Keep implementation machinery proportionate.** Use straightforward,
idiomatic code and introduce only the machinery needed to support the requested
behavior and flexibility. Optimize for conceptual simplicity and
maintainability, not for the fewest lines, fewest options, or narrowest product
surface.
**Do not invent restrictions.** Do not reject, prohibit, constrain, cap,
harden, or otherwise narrow behavior unless required by the user, an explicit
repository contract, or an unavoidable correctness constraint. Preserve
operator and caller choice where policy has not been established. If an
additional restriction appears advisable, present it as a decision rather than
silently implementing it.
**Do not invent guarantees.** Do not strengthen public contracts, guarantees,
or invariants beyond those expressly requested by the user.
**Use the standard library.** For example: in almost every case, the best
approach to copying files is to use the standard library. Writing custom code to
implement power-loss protection at the application level, or to guard against
bad actors with privileged and concurrent access to the host's filesystem,
is wasteful, unnecessary, and not idiomatic. Stick to the simple, standard, and
idiomatic approach unless another path is expressly requested by the user.
**An implementation detail is not repository policy.** Tests and documentation
must not elevate incidental behavior into a durable guarantee or restriction
unless that behavior was expressly selected as part of the contract. Existing
explicit contracts remain authoritative; changing or weakening one requires a
deliberate decision rather than being inferred from these principles.
[ADR-0017](../adr/0017-defer-cache-access-policy-to-deployment.md) illustrates
why Unix deployments own cache access policy through permissions, umask, setgid,
and default ACLs. Notarius retains its separately adopted correctness boundaries.
## Git Commit Policy
By default, create a commit after completing each coherent unit of work, using a
concise, plain-English message describing what changed. This policy is standing
authorization to create those local commits; a separate user request is not
required. An explicit user instruction to leave work uncommitted takes precedence.
A unit of work is a focused, reviewable outcome, including its relevant tests and
documentation. Do not commit after every intermediate edit or create empty commits.
Complete the appropriate validation before committing. If work is incomplete or
validation is blocked, report that state rather than presenting it as completed.
Before committing:
1. Run `git status --short` and review the relevant diff, including any staged
changes. Identify changes that predate the current unit of work.
2. Exclude secrets, local environment files, generated junk, and unrelated changes.
3. Stage only the files or hunks belonging to the completed unit. Use `git add .`
only when every included change belongs to that unit. Do not disturb unrelated
staged or unstaged work.
4. Review `git diff --staged` and run appropriate checks for the change. Do not
repeat checks already passed for unchanged content merely to create a commit.
5. Review `git log --oneline -n 10` and use
`git commit -m "<concise plain-English commit message>"`.
6. Report the commit identifier and any remaining uncommitted work.
If the unit cannot be safely separated from pre-existing work, leave it uncommitted
and explain the overlap rather than silently including that work. If a commit
fails, preserve the changes and report the failure; do not bypass hooks.
Default commit authorization does not authorize pushing, publishing, amending,
resetting, reverting, or rewriting history. Those actions require a separate user
instruction. Do not use `git commit -a` or `git commit --no-verify` to bypass the
staging and validation discipline above.

View File

@@ -1,25 +1,11 @@
# Architecture # Architecture
**Base Policy Version:** 1.1
**Last Updated**: 2026-09-25
**Upstream Source:** `policy` commit `45591a79004c6811950f14dca554f6ff69b7f58e`.
Notarius adapts the base template to its fixed pipeline, artifact families,
external adapters, state boundaries, and source-only distribution.
This document defines the intended high-level architecture of Notarius and the This document defines the intended high-level architecture of Notarius and the
invariants that changes must preserve. Implemented component details belong in invariants that changes must preserve. Implemented component details belong in
[Internal Overview](../internal/overview.md) and its linked documents. The [Internal Overview](../internal/overview.md) and its linked documents. The
reasoning behind significant architectural choices belongs in reasoning behind significant architectural choices belongs in
[ADRs](../adr/). [ADRs](../adr/).
An adopted constraint can govern future changes without implying that every
component it could govern exists. Descriptions of implemented structure must
remain accurate; proposals and transition status belong in roadmaps and ADRs.
Constraints reflect selected requirements, not incidental implementation choices.
## System Shape ## System Shape
Notarius is a small, dependency-light Go application for extracting structured Notarius is a small, dependency-light Go application for extracting structured
@@ -38,12 +24,6 @@ DAGs or a general workflow language. Every stage remains explicit; general
chunking, merging, or normalization behavior must not be hidden inside an chunking, merging, or normalization behavior must not be hidden inside an
extractor. extractor.
A stage module is one configured implementation of one pipeline stage. An
artifact family is the cohesive domain feature that owns an artifact across
the explicit stages and supporting codecs, validators, prompts, identity
rules, and reference projections. Artifact-family ownership does not combine
stages or alter the fixed pipeline.
Input and chunking are pipeline-wide. Each selected artifact lane owns its Input and chunking are pipeline-wide. Each selected artifact lane owns its
extract, merge, and normalize stages, and the output stage aggregates the run's extract, merge, and normalize stages, and the output stage aggregates the run's
lane outcomes. lane outcomes.
@@ -52,8 +32,6 @@ Notarius is contract-first without being abstraction-heavy. Interfaces and
extension points should protect demonstrated boundaries. New abstraction is not extension points should protect demonstrated boundaries. New abstraction is not
itself an architectural goal. itself an architectural goal.
Implementation discipline is owned by [Agent Policy](agents.md).
## Layers And Dependency Direction ## Layers And Dependency Direction
The application boundary is the composition root and may depend on concrete The application boundary is the composition root and may depend on concrete
@@ -61,24 +39,11 @@ implementations. Domain-neutral model and framework layers provide reusable
policy, contracts, and orchestration. Concrete input, pipeline, output, and policy, contracts, and orchestration. Concrete input, pipeline, output, and
validation extensions depend inward on those generic layers. validation extensions depend inward on those generic layers.
Semantic reconciliation is one such domain-neutral framework mechanism. It
prepares bounded source context, invokes a shared model-judgment protocol,
validates proposals, and applies safe plans through typed policies supplied by
the consuming artifact family. It does not own domain identity, durable IDs,
warning semantics, or artifact construction rules.
Generic layers must not depend on production extensions. Concrete extensions Generic layers must not depend on production extensions. Concrete extensions
must not compose the application or take ownership of process behavior. The must not compose the application or take ownership of process behavior. The
current packages implementing these layers are inventoried in current packages implementing these layers are inventoried in
[Internal Overview](../internal/overview.md). [Internal Overview](../internal/overview.md).
The root `assets` package is a content-only dependency leaf. It may expose a
read-only embedded filesystem, but it must contain no business logic and must
not depend on `internal` packages or PromptKit. Consumers scope that filesystem
to the content they own; the root package is not a behavioral registry or a
public extension contract. The rationale and compatibility consequence are
recorded in [ADR-0011](../adr/0011-centralize-llm-assets.md).
The following dependency boundaries are mandatory: The following dependency boundaries are mandatory:
- extractors and validators do not depend on concrete input adapters; - extractors and validators do not depend on concrete input adapters;
@@ -108,12 +73,6 @@ Extract modules own artifact semantics, prompt use, response schemas, and
domain interpretation. Domain-specific concepts remain in the relevant module, domain interpretation. Domain-specific concepts remain in the relevant module,
validator, shared domain helper, and artifact contract. validator, shared domain helper, and artifact contract.
Physical centralization of LLM-facing content does not transfer semantic
ownership from those modules. Modules retain their manifests, response-schema
identities, prompt ordering, and registration, while reading only their scoped
content subtree. Generic framework code remains domain-neutral when it reads
its own scoped generic assets from the shared content container.
Typed artifact registrations declare one stable artifact kind and exact Go Typed artifact registrations declare one stable artifact kind and exact Go
type from extraction through merge, normalization, and semantic validation. type from extraction through merge, normalization, and semantic validation.
Pipeline resolution requires a compatible codec and matching kind-specific Pipeline resolution requires a compatible codec and matching kind-specific
@@ -162,9 +121,8 @@ lanes, validators, and LLM profile: the canonical source digest selects the
plan, while the current run still applies its configured chunk validators to plan, while the current run still applies its configured chunk validators to
the materialized chunks. the materialized chunks.
The framework owns orchestration, origin enrichment, aggregation, and handoff The framework owns orchestration and handoff provenance. Modules return logical
provenance. Modules return logical results and classified diagnostics; they do results and warnings; they do not own CLI reporting, physical output, cache, or
not own CLI reporting, physical output, cache, or
debug roots, durable file placement, or checkpoint and debug lifecycle. debug roots, durable file placement, or checkpoint and debug lifecycle.
After pipeline-wide chunking, extraction uses bounded framework concurrency. After pipeline-wide chunking, extraction uses bounded framework concurrency.
@@ -176,53 +134,26 @@ may overlap. The framework must not create unbounded goroutines per lane or
chunk. chunk.
Completion timing does not choose public ordering or errors. The coordinator Completion timing does not choose public ordering or errors. The coordinator
orders accepted artifacts, grouped diagnostics, rejections, checkpoint events, and orders accepted artifacts, warnings, rejections, checkpoint events, and
framework errors by stable pipeline scope. Rejections do not cancel unrelated framework errors by stable pipeline scope. Rejections do not cancel unrelated
work. A framework error cancels derived work, prevents undispatched work from work. A framework error cancels derived work, prevents undispatched work from
starting, waits for started work, and prevents output encoding. starting, waits for started work, and prevents output encoding.
Warnings are process-only signals: configuration degradation, approved fallback,
or incomplete configured validation. Quality uncertainty and grounding findings
are advisories; successful canonicalization and cleanup are observations.
Modules choose that semantic classification, while the framework attaches
origin, aggregates groups, enforces bounds, and presents final collections.
An ordinary successful run therefore has zero warnings. See
[ADR-0015](../adr/0015-separate-process-warnings-from-quality-diagnostics.md)
for the decision rationale.
## Validation ## Validation
Validation is a framework-managed boundary around outputs from chunk, extract, Validation is a framework-managed boundary around outputs from chunk, extract,
merge, and normalize stages. Validators receive immutable stage output and merge, and normalize stages. Validators receive immutable stage output
make an explicit whole-output decision: approve, reject, fail, or skip when a and make an explicit whole-output decision: approve, approve with warnings, or
runtime prerequisite is unavailable. reject.
Typed artifact validators receive the domain value directly. Chunk validators Typed artifact validators receive the domain value directly. Chunk validators
receive source-zone chunks, while serialized validators receive immutable receive source-zone chunks, while serialized validators receive immutable
representation bytes and declared schema metadata. A validator registered for representation bytes and declared schema metadata. A validator registered for
one target or artifact kind cannot satisfy an incompatible selection. one target or artifact kind cannot satisfy an incompatible selection.
The framework runs every applicable validator sequentially in configured order. Rejection is a recorded pipeline outcome, not a framework execution error.
It aggregates rejections, exhausted validator failures, and skips before the Validator execution failures are framework errors. Rejected output does not
producer policy chooses a disposition. A completed rejection never advances. advance to the next stage.
With no rejection, an exhausted validator failure may fail the run or, under
the configured `warn_continue` policy, advance a structurally valid candidate
with explicit incomplete-validation provenance. Validators report findings;
they do not choose candidate disposition.
A completed rejection supplies a stable reason code for internal provenance
and bounded actionable correction guidance for the candidate producer. Reason
codes, validator keys, and operator-facing messages remain diagnostic data;
they are not model instructions. The framework constructs model-facing retry
text only from the semantic guidance and fails the contract rather than
inventing or truncating missing guidance.
An LLM-backed module may also request a feedback-aware retry when its own
deterministic translation or typed safety policy rejects a structurally valid
model response. It must supply model-facing guidance separately from its
reason code, operator message, and fallback diagnostics, together with the
exact `single_response_v1` candidate. A feedback-free module retry remains
valid when no exact candidate exists.
Default validator chains are production composition policy and are registered Default validator chains are production composition policy and are registered
centrally by stage and module. Configuration may replace a stage-local default, centrally by stage and module. Configuration may replace a stage-local default,
@@ -239,32 +170,6 @@ The caller of the LLM owns prompt selection, prompt inputs, response schema,
and interpretation of structured output. Provider adapters do not own source- and interpretation of structured output. Provider adapters do not own source-
or domain-specific prompt logic. or domain-specific prompt logic.
PromptKit owns bounded structural correction within one structured completion.
Notarius owns outer stage attempts, semantic validation, and acceptance policy;
the two budgets must remain separate.
An LLM-backed producer can participate in semantic correction only when it
declares `single_response_v1` and returns the exact one response that directly
controlled its candidate. On an actionable validator rejection or
feedback-capable module retry, the framework rebuilds the ordinary request and
appends only the latest defective response as an `assistant` message plus one
aggregated `user` correction message. This is a fresh replacement request, not
a growing conversation. The retry budgets, terminal policy, and sensitive-data
rationale are recorded in
[ADR-0014](../adr/0014-feedback-aware-validation-retries.md) and
[ADR-0016](../adr/0016-feedback-aware-module-requested-retries.md).
When a model selects an application entity, callers must supply a contextual
selection and deterministically attach the opaque application identity whenever
the selection resolves exactly. Models do not receive or reproduce opaque
application identifiers. Semantic reconciliation may instead expose
contiguous, one-based candidate handles that exist only for one request;
deterministic code resolves them before typed application, and they never
become durable identity. This is the approved request-local-label application
of [ADR-0012](../adr/0012-resolve-opaque-entity-identifiers-deterministically.md)
recorded by
[ADR-0013](../adr/0013-use-request-local-candidate-handles-for-semantic-reconciliation.md).
LLM calls and other external operations accept cancellation and respect LLM calls and other external operations accept cancellation and respect
timeouts. Concurrency control belongs in shared runtime plumbing rather than in timeouts. Concurrency control belongs in shared runtime plumbing rather than in
individual modules. individual modules.
@@ -272,16 +177,13 @@ individual modules.
The application-wide LLM scheduler bounds actual provider calls independently The application-wide LLM scheduler bounds actual provider calls independently
of framework worker limits. Every LLM-backed module, retry, and validator uses of framework worker limits. Every LLM-backed module, retry, and validator uses
the single injected scheduled client, including work performed by overlapping the single injected scheduled client, including work performed by overlapping
lanes. Provider runtime adapters may enforce a narrower backend-specific limit lanes.
beneath this mandatory application-wide scheduler.
## Configuration And Provenance ## Configuration And Provenance
Configuration loading, precedence, defaults, redaction, and validation are Configuration loading, precedence, defaults, environment overrides, redaction,
centralized through the ConfigKit-backed boundary. `NOTARIUS_CONFIG` is limited and validation are centralized. Named pipeline definitions make structural
to discovery; field-level environment overlays are not configuration input. composition explicit and discoverable. Operational overrides are permitted
Named pipeline definitions make structural composition explicit and discoverable.
Explicit command overrides are permitted
when they do not obscure the configured pipeline structure. when they do not obscure the configured pipeline structure.
Run preparation fails before stage execution when statically discoverable Run preparation fails before stage execution when statically discoverable
@@ -290,18 +192,10 @@ invalid or incompatible.
Run manifests record enough resolved pipeline, module, source, reference, and Run manifests record enough resolved pipeline, module, source, reference, and
LLM provenance to make a run auditable after configuration changes. Manifests LLM provenance to make a run auditable after configuration changes. Manifests
record identities and bounded validation summaries rather than secret, raw record identities and summaries rather than secret or large payload content.
model, correction, or large payload content.
## State, Output, And Safety ## State, Output, And Safety
Source material and model responses cross application validation boundaries.
Host access policy belongs to the deployment; hypothetical privileged actors
concurrently changing the filesystem do not justify new application defenses
without an explicit decision. This scope statement does not remove the adopted
correctness boundaries below or claim excluded scenarios cannot occur. Follow
[Agent Policy](agents.md) when a new threat or guarantee needs a decision.
Notarius exposes three filesystem surfaces with independent roots and Notarius exposes three filesystem surfaces with independent roots and
lifecycle: lifecycle:
@@ -318,49 +212,18 @@ an invocation that explicitly requests resume. Debug is never a cache input and
is never created without an explicit request. Pipeline modules receive is never created without an explicit request. Pipeline modules receive
collaborator interfaces and never physical roots. collaborator interfaces and never physical roots.
Deployments own cache access policy. Cache writers request conventional maximum
creation modes and let the process umask, setgid inheritance, and default ACLs
determine effective access. Notarius owns the separate safety boundary:
confined paths, symlink rejection, atomic publication, validation, and
recovery. It does not chmod, chown, chgrp, or migrate cache entries to impose
an application policy. This rule applies to chunk-plan and checkpoint cache
only; output and debug retain their separate contracts. [Operations](../operations.md)
owns deployment provisioning and retention guidance.
Only accepted, completely validated producer output is reusable checkpoint or
chunk-plan state. Rejected, structurally invalid, and validation-incomplete
results cannot become cache or checkpoint inputs, even when a
`warn_continue` result is allowed to advance in the current run. This
ineligibility follows derived merge and normalize results and generated
references for the remainder of the run: current-run handoff remains allowed,
but no dependent cache or checkpoint may be loaded or published.
Writes are atomic where practical. Paths for writes, moves, overwrites, and Writes are atomic where practical. Paths for writes, moves, overwrites, and
deletion must be narrow and explicit. Notarius never automatically deletes deletion must be narrow and explicit. Notarius never automatically deletes
output or requested debug bundles; cache cleanup is explicit and recoverable. output or requested debug bundles; cache cleanup is explicit and recoverable.
Secrets must not appear in errors, logs, output, cache, debug summaries, Secrets must not appear in errors, logs, output, cache, debug summaries,
traces, manifests, documentation, examples, or redacted configuration. Raw traces, manifests, documentation, examples, or redacted configuration. Debug
assistant responses and complete correction messages are attempt-local and are
excluded from ordinary durable records and summaries; the requested detailed
debug trace is the sole diagnostic surface allowed to retain them. Debug
collection is allowlisted to application-owned payloads and must not capture collection is allowlisted to application-owned payloads and must not capture
unrelated process environment values or filesystem content. Trace data may unrelated process environment values or filesystem content. Trace data may
contain application data and therefore inherits its sensitivity; operators own contain application data and therefore inherits its sensitivity; operators own
access controls and retention. Physical layout and operation are defined in access controls and retention. Physical layout and operation are defined in
[Operations](../operations.md). [Operations](../operations.md).
## Platform And Distribution
Linux is the supported deployment platform. macOS is supported only as a
best-effort development and compilation environment, while Windows is
unsupported. Notarius distributes source releases only: an immutable source
tag and its checked-in release note identify a release. The project does not
publish executable binaries, archives, installers, container images,
checksums, signatures, or package-manager entries. [Release Policy](release.md)
owns publication requirements and completion; maintainer commands and guards
belong to [Source Releases](../release.md).
## Architectural Non-Goals ## Architectural Non-Goals
Notarius does not aim to provide: Notarius does not aim to provide:

View File

@@ -1,14 +1,5 @@
# Documentation Policy # Documentation Policy
**Base Policy Version:** 1.3
**Last Updated**: 2026-09-25
**Upstream Source:** `policy` commit `45591a79004c6811950f14dca554f6ff69b7f58e`.
Notarius retains subprocess-consumer ownership, the distinction between
descriptive behavior and policy, and no placeholders for absent interfaces.
## Purpose ## Purpose
This policy assigns each documentation topic to one canonical owner. Its goal is This policy assigns each documentation topic to one canonical owner. Its goal is
@@ -33,28 +24,12 @@ contract, maintain it only in the owning document.
Outside `docs/roadmap/`, documentation describes implemented behavior only. Outside `docs/roadmap/`, documentation describes implemented behavior only.
Partial features may be described only to their implemented boundary. Partial features may be described only to their implemented boundary.
Architectural constraints may govern future changes without implying that the ADRs are the narrow exception: an ADR may record an accepted architectural
components or capabilities they would govern already exist. This does not permit decision before implementation, but acceptance must not be presented as proof
describing a proposed system structure as implemented.
ADRs may record an accepted architectural decision before implementation, but
acceptance must not be presented as proof
that the behavior exists. The roadmap owns implementation status and sequencing that the behavior exists. The roadmap owns implementation status and sequencing
until the decision is implemented. Current architecture, user, operator, until the decision is implemented. Current architecture, user, operator,
integration, and internal documentation are updated when the behavior lands. integration, and internal documentation are updated when the behavior lands.
### Implementation Details And Policy
Documentation must distinguish an explicit contract from an incidental
implementation choice. Do not describe a restriction, guarantee, security
policy, default, or limit as authoritative merely because the current code
implements it. Documentation may accurately describe current behavior without
making it a permanent requirement; distinguish descriptive behavior from
normative policy. The
[implementation-discipline rules](agents.md#implementation-discipline)
own the criteria for adopting product restrictions and for elevating behavior
into repository policy.
### Audience And Detail ### Audience And Detail
Write for the document's stated audience and include only the detail needed for Write for the document's stated audience and include only the detail needed for
@@ -84,18 +59,14 @@ secret values.
| --- | --- | --- | --- | | --- | --- | --- | --- |
| Product orientation and minimal end-to-end quickstart | `README.md` | What Notarius is, why it is useful, one shortest successful invocation, and links onward. | Complete command reference, configuration reference, operational procedures, implementation detail. | | Product orientation and minimal end-to-end quickstart | `README.md` | What Notarius is, why it is useful, one shortest successful invocation, and links onward. | Complete command reference, configuration reference, operational procedures, implementation detail. |
| Contributor entry point | `docs/development.md` | Task-oriented reading guide, minimal contributor orientation, baseline validation commands, and links to canonical docs. | Package inventory, architecture rules, subsystem behavior, detailed change recipes. | | Contributor entry point | `docs/development.md` | Task-oriented reading guide, minimal contributor orientation, baseline validation commands, and links to canonical docs. | Package inventory, architecture rules, subsystem behavior, detailed change recipes. |
| Application architecture policy | `docs/policy/architecture.md` | System shape, normative ownership, dependency direction, architectural boundaries, invariants, safety properties, and non-goals. | Concrete package inventory, implementation mechanics, contributor procedures, decision history, future work. | | Current application architecture | `docs/policy/architecture.md` | System shape, normative ownership, dependency direction, architectural boundaries, invariants, safety properties, and non-goals. | Concrete package inventory, implementation mechanics, contributor procedures, decision history, future work. |
| Agent policy | `docs/policy/agents.md` | Implementation discipline, rules against invented restrictions or guarantees, and local commit authorization. | Application contracts, architecture, testing policy, and contributor procedures. |
| Documentation organization | `docs/policy/documentation.md` | Documentation ownership, audience boundaries, maintenance rules, and ADR/document lifecycle. | Application architecture or product behavior. | | Documentation organization | `docs/policy/documentation.md` | Documentation ownership, audience boundaries, maintenance rules, and ADR/document lifecycle. | Application architecture or product behavior. |
| Testing policy | `docs/policy/testing.md` | Test philosophy, risk-based sufficiency, test boundaries, doubles, coverage guidance, regression-test policy, and criteria for adding, rewriting, or deleting tests. | Subsystem behavior, application contracts, subsystem-specific test inventories, and implementation plans. | | Testing policy | `docs/policy/testing.md` | Test philosophy, risk-based sufficiency, test boundaries, doubles, coverage guidance, regression-test policy, and criteria for adding, rewriting, or deleting tests. | Subsystem behavior, application contracts, subsystem-specific test inventories, and implementation plans. |
| CLI contract | `docs/cli.md` | Commands, arguments, flags, invocation semantics, and exit codes. | End-to-end operating procedures, configuration field definitions, runtime filesystem layout, module implementation details. | | CLI contract | `docs/cli.md` | Commands, arguments, flags, invocation semantics, and exit codes. | End-to-end operating procedures, configuration field definitions, runtime filesystem layout, module implementation details. |
| Configuration contract | `docs/config.md` | Discovery and precedence, file schema, fields, defaults, environment overrides, validation rules, and user-selectable module or validator keys. | Complete example files, CLI syntax, runtime state lifecycle, module implementation details. | | Configuration contract | `docs/config.md` | Discovery and precedence, file schema, fields, defaults, environment overrides, validation rules, and user-selectable module or validator keys. | Complete example files, CLI syntax, runtime state lifecycle, module implementation details. |
| Operations | `docs/operations.md` | Runtime workflows, physical filesystem and state layout, output, cache, and debug handling, resume, cleanup, permissions, recovery, and operational limits. | CLI flag syntax, configuration field definitions, logical output schemas, implementation mechanics. | | Operations | `docs/operations.md` | Runtime workflows, physical filesystem and state layout, output, cache, and debug handling, resume, cleanup, permissions, recovery, and operational limits. | CLI flag syntax, configuration field definitions, logical output schemas, implementation mechanics. |
| Release policy | `docs/policy/release.md` | Versioning, compatibility, integrity, immutability, proportionate gate selection, publication controls, and release completion. | Concrete commands, product installation, historical notes, and CI implementation detail. |
| Repository-specific release procedure | `docs/release.md` | Prerequisites, candidate preparation, publication command, automated guards, optional diagnostics, and failure recovery. | General release requirements, product installation guidance, historical summaries, and current compatibility contracts. |
| Release-note history | `docs/releases/` | One checked-in historical summary for each source release made under the procedure. The note at the immutable tag is that release's record. | Current commands, behavior, contracts, and compatibility definitions. |
| Public HTTP contract, if introduced | `docs/api.md` | Routes, authentication, media types, request and response schemas, status codes, pagination, caching, idempotency, rate limits, and HTTP retry semantics. | Client walkthroughs, upstream or downstream integration internals, implementation detail. | | Public HTTP contract, if introduced | `docs/api.md` | Routes, authentication, media types, request and response schemas, status codes, pagination, caching, idempotency, rate limits, and HTTP retry semantics. | Client walkthroughs, upstream or downstream integration internals, implementation detail. |
| Consumer guidance | `docs/consumers/` | Task-oriented subprocess use, minimal client examples, consumer responsibilities, and guidance for any future public package or API. | Wire semantics, external protocol contracts, internal implementation detail. | | Consumer guidance, if a public package or API is introduced | `docs/consumers/` | Task-oriented use of the public interface, minimal client examples, and consumer responsibilities. | HTTP wire semantics, external protocol contracts, internal implementation detail. |
| External and durable integration contracts | `docs/integrations/` | External file formats and protocols, upstream and downstream contracts, logical output bundle paths and schemas, media types, and compatibility behavior. | Physical runtime placement and lifecycle, internal transformations, CLI syntax, configuration defaults. | | External and durable integration contracts | `docs/integrations/` | External file formats and protocols, upstream and downstream contracts, logical output bundle paths and schemas, media types, and compatibility behavior. | Physical runtime placement and lifecycle, internal transformations, CLI syntax, configuration defaults. |
| Implemented component inventory | `docs/internal/overview.md` | Current packages and components, their implemented responsibilities, and links to focused internal docs. | Normative architecture, contributor reading policy, external contracts. | | Implemented component inventory | `docs/internal/overview.md` | Current packages and components, their implemented responsibilities, and links to focused internal docs. | Normative architecture, contributor reading policy, external contracts. |
| Internal component behavior | Other files under `docs/internal/` | Implementation flow, internal collaborators and state transitions, package-local guarantees and failures, and relevant tests. | Global architecture invariants, configuration definitions and defaults, external schemas, operator procedures. | | Internal component behavior | Other files under `docs/internal/` | Implementation flow, internal collaborators and state transitions, package-local guarantees and failures, and relevant tests. | Global architecture invariants, configuration definitions and defaults, external schemas, operator procedures. |
@@ -124,16 +95,6 @@ runtime state and how to operate or recover the application. When a workflow
crosses these topics, choose the document that owns the task and link to the crosses these topics, choose the document that owns the task and link to the
other contracts. other contracts.
### Releases
The [release policy](release.md) owns requirements and invariants; the
[repository procedure](../release.md) owns the commands that satisfy them.
Release notes are
historical summaries, not current-state contract owners: the checked-in note at
an immutable tag records that release, while current canonical documentation
must change with the behavior it describes. Do not use a release procedure or
note to weaken policy, redefine current contracts, or defer documentation updates.
### Contracts And Implementation ### Contracts And Implementation
Integration and API documents define externally observable shapes and Integration and API documents define externally observable shapes and

View File

@@ -1,204 +0,0 @@
# Release Policy
**Base Policy Version:** 1.1
**Last Updated**: 2026-09-25
**Upstream Source:** `policy` commit `45591a79004c6811950f14dca554f6ff69b7f58e`.
Notarius adaptations: source-only delivery, lightweight tags, branch-before-tag
publication, no prereleases, and the justified native command check below.
## Purpose
This policy defines a proportionate workflow for validating and publishing an
intended source revision under a fresh version. Releases should require little
routine intervention, avoid repeated work, and preserve published contents.
The repository-specific commands belong in `docs/release.md`. Historical release
summaries belong in `docs/releases/`. The procedure must agree with the adopted
policy and link to it rather than introduce additional release requirements.
## Release Model And Completion
A source release identifies one exact committed source revision through an
immutable version tag and a version-matched, checked-in release note. The
primary release branch is `main` and the authoritative remote is `origin`.
Tags are lightweight. Publish the candidate to `main` before pushing its tag.
A successful push of the intended tag, with its source commit available on the
authoritative primary branch, completes the release. Notarius uses
branch-before-tag publication; it does not require an atomic push.
Release commands must finish at successful publication. They must not wait for
or poll CI, require successful CI status, inspect downloadable artifacts, or
check consumer installation or Go module availability as completion gates.
A failed or ambiguous push requires resolving the authoritative remote state
before declaring success or retrying; successful pushes do not require a
separate remote read-back gate.
Notarius publishes source only: no binaries, archives, images, packages,
checksums, signatures, or Gitea release objects. Tag CI may provide advisory
validation but publishes no deliverables and is not a completion gate.
## Versioning And Compatibility
Stable releases use Semantic Versioning in the form `vMAJOR.MINOR.PATCH`, with
no leading zeroes. Notarius does not support prereleases.
The selected version must be unused locally and on the authoritative remote.
Do not require queries to every configured remote or publication service.
A version known to have already been published must not be reused, even if the
normal guards do not inspect that channel. Adding a distribution channel
requires a deliberate update to the policy and procedure where needed.
For releases at or after `v1.0.0`:
- a major release may make intentional incompatible changes;
- a minor release may add backward-compatible behavior; and
- a patch release may make backward-compatible fixes.
Before `v1.0.0`, a minor release may change a documented public, operational,
configuration, data, integration, or compatibility contract when the release
note explains the impact and required consumer action. A patch release must
not intentionally break those contracts within its minor line.
Additional compatibility promises and deliberate exceptions belong in the
adopted policy or canonical contract documentation. Release notes explain
changes; they do not redefine current compatibility contracts.
## Release Documentation And Preparation
Every new release must include `docs/releases/<version>.md` in the tagged
source tree. Use the complete version tag as the filename and begin with a
title containing the product name and that exact version.
Include a short summary and any relevant compatibility impact or upgrade
instructions. Keep the note proportionate to the change. Fixed sections and
explicit no-action boilerplate are not required. Existing historical notes
need not be rewritten to adopt this format.
Keep exact current contracts in their canonical documents and update affected
documentation with the behavior change. Link to those documents from the note
where useful. A release note must not substitute for current documentation.
Do not create retrospective notes for the pre-procedure v0.1.0, v0.2.0, or
v0.3.0 tags.
Version selection, release-note content, and retirement of completed roadmaps
are preparation responsibilities, not separate human approval gates. Preserve
planned and deferred work and repair affected links when retiring roadmaps.
Ordinary change review covers unintended files, credentials, and private data;
do not repeat it as a release-specific sign-off. Ignored local files are not
prohibited merely because they exist: they are not included in the tagged source.
## Candidate Validation
Provide one publication command that validates the final committed candidate,
checks publication state, and publishes the intended refs. A validation-only
mode or command may be provided for diagnosis, but must not be a required step
before publication repeats the same checks.
Run the complete deterministic, offline default test suite once for the final
candidate, following the [testing policy](testing.md). For repositories that
publish builds, verify the required build targets. Select those targets for
the application's actual support commitments; native testing on each platform
and access to particular hardware are not default release prerequisites.
Additional gates must address a concrete risk that the existing checks do not
adequately cover. Do not automatically require linting, race testing, module
hygiene audits, documentation audits, credential scans, example validation, or
expanded platform matrices solely because they are possible. Retain such
checks when their distinct protection justifies their cost. A one-time migration
check must not silently become a permanent release requirement.
Release tests must not require real credentials, mutable application services,
paid APIs, provider probes, or inference calls. The procedure may describe dependency
preparation separately from offline validation.
Avoid duplicate validation across release phases. Successful checks for the
same final candidate need not be repeated solely to satisfy another phase;
any reuse must clearly apply to that exact candidate. Persistent validation
caches are not required. If the candidate changes, validate the new candidate.
Cheap publication-state checks may be repeated immediately before pushing
because local or remote state may have changed during validation.
Automate mechanical guards. Human review gates are strongly disfavored: add one
only through an explicit project-specific policy decision explaining the
concrete risk and why ordinary change review or automation is insufficient.
Do not add routine confirmations for checks the command can perform itself.
### Notarius Gate Selection
Run the default offline Go suite once and build the native `cmd/notarius` with
release version injection, checking its `--version` output. The command check
protects the actual executable entry point and linker-supplied version, which
package tests alone do not verify. Build products are temporary, not published.
Use `GOWORK=off` so a sibling checkout cannot substitute for declared dependencies.
Race tests, standalone vet, formatting/whitespace checks, module tidy audits,
standalone example validation, and Linux/Darwin cross-builds remain useful
change-specific development checks, not mandatory release gates. Default tests
already cover maintained examples. Source-only publication does not require a
binary distribution matrix. No new platform or hardware prerequisites are added.
## Publication Guards
Automate the checks needed to publish the intended source under a fresh version:
- the version has the adopted format and is unused locally and on the
authoritative remote;
- the worktree and index are clean on the primary release branch;
- the source being published is the exact candidate that passed validation,
including its matching release note;
- the branch push advances authoritative `main` to the candidate without
discarding remote history, before the tag is published; and
- the tag identifies the validated candidate and only the intended refs are
published.
Fetch the authoritative branch and tag state as needed for these checks.
Do not force branch or tag updates to bypass a failed guard. Publish explicit
refs rather than all local tags. Do not require CI success, publication-service
availability checks, or a separate manual confirmation for each guard.
## Immutability And Failure Recovery
Published version tags and published contents are immutable. Do not move,
overwrite, recreate, or delete a published tag, or replace a published artifact,
checksum, signature, provenance statement, or release note under that version.
The checked-in note at the immutable tag is that release's historical record.
If validation fails before publication, correct and commit the candidate and
rerun the publication command. If a push fails or its outcome is ambiguous,
inspect the authoritative remote before retrying. Remove an unpublished local
tag only after confirming it was never published or used by a release channel.
An unavailable remote leaves publication status unknown; do not overwrite a tag
to resolve uncertainty.
Transient network, authentication, CI, or consumer-availability failures do not
by themselves require a new version. Advisory CI or optional installation checks
may be retried against the same immutable tag. Notarius has no asynchronous
artifact delivery to recover.
A defect requiring changes to published source or contents requires a corrected
commit, a new semantically appropriate version, a matching release note, and
the release procedure again. Leave the original release intact. A channel's
yank, deprecation, or advisory mechanism may warn consumers away from a defective
release without replacing its contents or reusing its version.
## Repository Release Procedure
Maintain `docs/release.md` as a short, executable maintainer runbook. It must
identify:
1. prerequisites, primary branch, and authoritative remote;
2. version and release-note preparation;
3. one publication command with its selected local checks and automated guards;
4. declared deliverables, required build targets, and automation responsibilities;
5. the successful-push completion point and separately reported delivery status;
and
6. failure handling and ambiguous-push recovery.
Use copyable commands where practical. Keep optional diagnostics and consumer
verification clearly optional. The procedure must not turn preparation work,
ordinary change review, or post-publication investigation into additional
release gates.

View File

@@ -1,14 +1,5 @@
# Testing Policy # Testing Policy
**Base Policy Version:** 1.1
**Last Updated**: 2026-09-25
**Upstream Source:** `policy` commit `45591a79004c6811950f14dca554f6ff69b7f58e`.
Adopted without substantive deviations; Notarius's established contracts remain
subject to [Agent Policy](agents.md).
## Purpose ## Purpose
Our tests exist to make **incorrect changes expensive and correct changes cheap**. Our tests exist to make **incorrect changes expensive and correct changes cheap**.
@@ -17,16 +8,29 @@ We do not optimize for test count, line coverage, exhaustive isolation, or the f
## Every test has a cost ## Every test has a cost
Every test has an immediate and continuing cost: writing, review, execution, Testing is not an unqualified good. Every test imposes both an immediate cost and a continuing lifetime cost.
diagnosis, fixture maintenance, and updates when legitimate behavior changes.
Tests can also constrain refactoring, duplicate policy, and add failure noise. A test must be:
- written and reviewed;
- understood by future maintainers and coding agents;
- executed in local and CI workflows;
- diagnosed when it fails;
- updated when legitimate behavior changes;
- maintained as fixtures, APIs, and dependencies evolve; and
- removed or rewritten when it becomes redundant, brittle, misleading, or obsolete.
Tests also create cognitive and architectural friction. They can constrain refactoring, duplicate policy, slow feedback loops, add noise to failures, and cause harmless implementation changes to require unrelated edits across the suite.
A test is warranted only when the confidence it provides justifies these costs. A test is warranted only when the confidence it provides justifies these costs.
Evaluate that tradeoff at two levels:
Apply this cost-benefit analysis at two levels:
1. **Per test:** What realistic defect does this test detect, how consequential would that defect be, and is that protection worth the test's lifetime cost? 1. **Per test:** What realistic defect does this test detect, how consequential would that defect be, and is that protection worth the test's lifetime cost?
2. **Across the suite:** Does this collection provide materially more confidence than a smaller, simpler suite would? 2. **Across the suite:** Does this collection provide materially more confidence than a smaller, simpler suite would?
The preferred test suite is a **lean suite that provides sufficient confidence in the risks that matter, without redundant or low-value tests**. We seek sufficient confidence with the least unnecessary testing friction, not the fewest possible tests.
Some friction is intentional. Tests should make dangerous changes—such as breaking compatibility, corrupting data, violating security boundaries, or reintroducing subtle bugs—require deliberate review. They should not make ordinary internal changes needlessly expensive. Some friction is intentional. Tests should make dangerous changes—such as breaking compatibility, corrupting data, violating security boundaries, or reintroducing subtle bugs—require deliberate review. They should not make ordinary internal changes needlessly expensive.
The cost of a test is not a reason to omit testing by default. Do not cite maintenance cost abstractly. When omitting a plausible test, be able to state why the protected failure is low-risk, already covered, obvious, reversible, or cheaper to detect elsewhere. For consequential, subtle, or difficult-to-observe behavior, the presumption should favor testing. The cost of a test is not a reason to omit testing by default. Do not cite maintenance cost abstractly. When omitting a plausible test, be able to state why the protected failure is low-risk, already covered, obvious, reversible, or cheaper to detect elsewhere. For consequential, subtle, or difficult-to-observe behavior, the presumption should favor testing.
@@ -51,7 +55,7 @@ Control clocks, randomness, environment variables, and other process-global or m
## What deserves tests ## What deserves tests
To the extent the following features are present in the application, prioritize tests for: Prioritize tests for:
1. Public and package-level contracts. 1. Public and package-level contracts.
2. Domain rules and important invariants. 2. Domain rules and important invariants.
@@ -63,18 +67,7 @@ To the extent the following features are present in the application, prioritize
A package-level contract is behavior relied upon by another package or major collaborator, not every observable detail of a package implementation. A package-level contract is behavior relied upon by another package or major collaborator, not every observable detail of a package implementation.
Choose tests for requested behavior and established contracts. Tests must not For behavior involving **data integrity, destructive operations, compatibility, security, concurrency, idempotency, or recovery**, presume that durable tests are required unless the behavior is already credibly protected at another layer.
introduce new restrictions, guarantees, or threat scenarios merely because they
could be asserted. Follow the [agent policy](agents.md) when deciding whether a
behavior belongs to the contract; current implementation behavior alone is not
sufficient authority.
For established behavior involving **data integrity, destructive operations,
compatibility, security, concurrency, idempotency, or recovery**, favor durable
tests for realistic failures whose consequences warrant protection. Identify
the failure and the existing contract it would violate. These categories call
for attention, not an automatic test inventory or permission to strengthen the
application's guarantees. Account for protection already provided at other layers.
Do not add tests merely because a function, branch, or line exists. Do not add a test when the same meaningful risk is already adequately protected elsewhere. Do not add tests merely because a function, branch, or line exists. Do not add a test when the same meaningful risk is already adequately protected elsewhere.
@@ -92,7 +85,11 @@ Do not force all behavior through oversized end-to-end tests. Do not test every
## Test behavior, not implementation ## Test behavior, not implementation
A test should protect a decision, contract, or invariant. A test must not memorialize the current implementation. A test should protect a decision, contract, or invariant—not memorialize the current implementation.
Before adding or retaining a test, ask:
> What realistic defect would this test catch?
A test is suspect when its main purpose is to detect that someone: A test is suspect when its main purpose is to detect that someone:
@@ -105,6 +102,8 @@ A test is suspect when its main purpose is to detect that someone:
Refactoring should normally require no test edits unless the refactored structure is itself part of the contract. Refactoring should normally require no test edits unless the refactored structure is itself part of the contract.
A test can be factually correct and still have negative value. Accurately describing current behavior is not enough; the protected behavior must be important enough to justify the future friction.
## Expected effects of different changes ## Expected effects of different changes
Use the following expectations when evaluating test failures and test maintenance: Use the following expectations when evaluating test failures and test maintenance:
@@ -122,11 +121,10 @@ A test failing is not the same as a test needing to be edited. Many tests may co
Configurable thresholds and defaults must not be duplicated throughout the test suite. Configurable thresholds and defaults must not be duplicated throughout the test suite.
For this example, assume the selected contract rejects a request to start more For example, do not encode an internal concurrency limit indirectly:
processes than a configured limit. Do not encode its production default indirectly:
```go ```go
// Production default: // Production policy:
const maxConcurrency = 4 const maxConcurrency = 4
// Brittle test: // Brittle test:
@@ -144,30 +142,25 @@ require.NoError(t, runner.Start(limit))
require.ErrorIs(t, runner.Start(limit+1), ErrTooMuchConcurrency) require.ErrorIs(t, runner.Start(limit+1), ErrTooMuchConcurrency)
``` ```
For this selected contract, the test should prove: The test should prove:
- the configured limit is accepted; and - the configured limit is accepted; and
- one beyond the configured limit is rejected. - one beyond the configured limit is rejected.
Rejection is not a general requirement for concurrency control. If the selected
contract queues excess work or limits active workers while accepting larger
batches, test that behavior instead. Do not add rejection or a configurable
limit solely to reproduce this example.
The production default should be tested exactly only when its literal value is itself a public, operational, safety, protocol, or compatibility requirement. The production default should be tested exactly only when its literal value is itself a public, operational, safety, protocol, or compatibility requirement.
Apply the same rule to limits, timeouts, capacities, retry counts, and ranges: test relationships and behavior, not duplicated literals. Apply the same rule to limits, timeouts, capacities, retry counts, and ranges: test relationships and behavior, not duplicated literals.
For established concurrency limits, distinguish these behaviors where the contract requires them: For concurrency limits, test both kinds of behavior when relevant:
1. **Configuration handling:** values are accepted or rejected according to the selected contract. 1. **Configuration enforcement:** invalid or excessive requested values are handled correctly.
2. **Runtime enforcement:** observed peak concurrency never exceeds the configured limit. 2. **Runtime enforcement:** observed peak concurrency never exceeds the configured limit.
Use a test-controlled limit and measure the behavior relative to that limit. Do not merely assert today's default value. Use a test-controlled limit and measure the behavior relative to that limit. Do not merely assert today's default value.
## Avoid semantic duplication across layers ## Avoid semantic duplication across layers
Each behavior should have a clear test owner. Where these layers exist: Each behavior should have a clear test owner.
- Parser tests own parsing cases. - Parser tests own parsing cases.
- Validator tests own validation rules. - Validator tests own validation rules.
@@ -179,6 +172,8 @@ Each behavior should have a clear test owner. Where these layers exist:
Higher-level tests should not repeat every lower-level case. A single intentional policy change should not require unrelated edits across many test files. Higher-level tests should not repeat every lower-level case. A single intentional policy change should not require unrelated edits across many test files.
Tests that are individually reasonable may still be collectively redundant. Evaluate the marginal value of each additional test in light of the protection already provided by the rest of the suite.
## Use test doubles deliberately ## Use test doubles deliberately
Choose the least elaborate test double that provides the required control or observation. Choose the least elaborate test double that provides the required control or observation.
@@ -202,16 +197,15 @@ Do not use mocks merely to isolate every object or reproduce the implementation'
## Go-specific guidance ## Go-specific guidance
Select techniques according to the behavior and risk being tested. The following Use:
are options, not a checklist that every package or application must satisfy:
- table-driven tests for meaningful behavioral categories and boundaries; - table-driven tests for meaningful behavioral categories and boundaries;
- `t.TempDir()` for real filesystem behavior; - `t.TempDir()` for real filesystem behavior;
- `httptest.Server` for realistic HTTP interactions; - `httptest.Server` for realistic HTTP interactions;
- fuzz tests when exploring broad input spaces, such as parsers or path handling, adds useful protection beyond representative cases; - fuzz tests for parsers, normalization, path handling, and broad input spaces;
- golden files only when the complete output is intentionally stable; - golden files only when the complete output is intentionally stable;
- integration tests where correctness depends on component interaction; and - integration tests where correctness depends on component interaction; and
- a small number of representative end-to-end tests when assembled workflows expose meaningful risks not adequately covered at narrower boundaries. - a small number of representative end-to-end tests.
Avoid exact error-string assertions unless the wording is itself contractual. Prefer `errors.Is`, `errors.As`, typed errors, or structured error fields. Avoid exact error-string assertions unless the wording is itself contractual. Prefer `errors.Is`, `errors.As`, typed errors, or structured error fields.
@@ -229,6 +223,8 @@ Use it to find untested critical branches and unexpectedly weak packages. Do not
Pure domain logic will often warrant higher coverage than CLI wiring or external adapters. Uneven coverage is acceptable when it reflects risk. Pure domain logic will often warrant higher coverage than CLI wiring or external adapters. Uneven coverage is acceptable when it reflects risk.
Increasing coverage is valuable only when the newly covered behavior protects a meaningful risk at an acceptable cost.
## Regression tests ## Regression tests
A bug fix should normally include a regression test that fails before the fix and passes afterward. A bug fix should normally include a regression test that fails before the fix and passes afterward.
@@ -257,26 +253,44 @@ Strong candidates include tests that:
Several brittle tests may encode one genuine requirement. Replace them with one durable behavior-level or invariant test rather than preserving all of them. Several brittle tests may encode one genuine requirement. Replace them with one durable behavior-level or invariant test rather than preserving all of them.
Deleting a low-value test can improve the quality of the suite by reducing noise, maintenance burden, and friction around legitimate change.
## Reviewing a proposed test ## Reviewing a proposed test
Use the following questions when the value, boundary, or durability of a proposed test is not self-evident. Significant test additions should be reviewable against them, but written answers are not required for every routine test. Use the following questions when the value, boundary, or durability of a proposed test is not self-evident. Significant test additions should be reviewable against them, but written answers are not required for every routine test.
1. Which requested behavior or established contract does it protect, and what realistic defect would violate it? 1. What realistic defect would it catch?
2. How likely and consequential is that defect, and is it already protected elsewhere? 2. How likely is that defect?
3. What is the narrowest stable boundary that can test it clearly? 3. How consequential would it be?
4. Would a correct internal refactor or a change to an incidental default require test edits? 4. Is the behavior already protected elsewhere?
5. Does its confidence justify its maintenance, execution, and diagnostic cost, or would a smaller test suffice? 5. At which layer should this behavior be owned?
6. Does the test assert a durable contract or an incidental implementation detail?
7. Could the implementation be refactored without changing the behavior and without editing this test?
8. What should cause this test to fail?
9. What legitimate changes should not cause this test to fail?
10. What ongoing maintenance, execution, and diagnostic cost will the test impose?
11. Is there a smaller or more direct test that protects the same risk?
Do not add the test when its expected lifetime cost exceeds its expected protective value.
When deciding not to test plausible behavior, record or be able to explain why the risk is low, already protected, obvious, reversible, or cheaper to detect elsewhere.
## Definition of sufficient ## Definition of sufficient
A test suite is sufficient when: A test suite is sufficient when:
- important requested behavior and established contracts are protected against realistic, consequential failures; - important contracts and invariants are protected;
- boundaries and component interactions have coverage where they introduce meaningful risks not protected elsewhere; - meaningful boundaries and failure modes are exercised;
- realistic and consequential regressions are credibly protected against silent recurrence;
- behavior involving data integrity, destructive operations, compatibility, security, concurrency, idempotency, and recovery is credibly protected;
- important external boundaries have realistic integration coverage;
- representative complete workflows are tested;
- failures provide useful signal rather than redundant noise; - failures provide useful signal rather than redundant noise;
- legitimate internal changes usually do not require test edits; and - legitimate internal changes usually do not require test edits; and
- additional tests would mostly repeat existing protection or preserve inconsequential implementation details. - additional tests would mostly repeat existing protection or preserve inconsequential implementation details.
Sufficiency is a risk judgment, not a coverage percentage, test count, or required Sufficiency is a risk judgment, not a coverage percentage or test count. Reassess it as the application, its users, and the consequences of failure evolve.
mix of testing techniques. Reassess it as the application, its users, and the
consequences of failure evolve. The governing rule is:
> Test heavily where failure is consequential, subtle, or difficult to detect after the fact. Test lightly where failure is obvious, reversible, and inexpensive—and retain no test whose lifetime cost exceeds the confidence it provides.

View File

@@ -1,84 +0,0 @@
# Source Releases
This is the executable runbook for [Release Policy](policy/release.md).
Notarius publishes an immutable lightweight tag and its checked-in release note,
not binaries or Gitea release objects. Linux is supported; macOS is best-effort
for development, and Windows is unsupported. `main` on `origin` is authoritative.
## Prepare
Use Git, a POSIX shell, and Go 1.25.5 or newer. Configure ordinary Git credentials
for `origin` and private-module access without putting secrets in the repository.
Prepare dependencies before offline validation where necessary:
```sh
GOWORK=off GOPRIVATE=gitea.maximumdirect.net go mod download
```
Choose an unused stable version under the policy's compatibility rules. Add
`docs/releases/<version>.md`, starting with `# Notarius <version>`, with a concise
summary and relevant compatibility/upgrade information. No fixed section list is
required. Update current documentation with behavior changes; retire only
completed roadmaps and preserve planned/deferred work. Commit the candidate on
`main` after ordinary change review. There is no separate release sign-off.
## Publish
Run one command from the repository with the actual selected version:
```sh
./scripts/release.sh vMAJOR.MINOR.PATCH
```
The command checks version availability on `origin`, branch and worktree state,
and the matching note. It runs the complete offline default test suite once and
a native CLI build/version check with `GOWORK=off`. The
[gate rationale](policy/release.md#notarius-gate-selection) explains the selected
checks. It then rechecks candidate and publication state, pushes the exact
candidate to `main` without force, creates a lightweight tag, and pushes only
that tag. It neither rewrites history nor publishes other local tags.
A successful tag push completes the source release. The command does not wait
for CI or verify consumer installation/module availability. Tag CI is advisory
validation only; there are no asynchronous deliverables to await.
## Failure And Recovery
Before publication, correct failed checks, commit any changes, and rerun the
command. If the candidate changes during validation, validation must run again
on the new candidate. A failed branch push publishes no tag.
On a failed or ambiguous push, the command queries the affected remote ref. If
a tag push reports failure but the intended lightweight tag is confirmed on the
remote, publication succeeded. Otherwise it stops and reports failure or unknown
status, leaving any local tag intact for inspection. Do not blindly rerun, force
push, or delete a possibly published tag.
Inspect authoritative state before retrying:
```sh
git ls-remote origin refs/heads/main refs/tags/vMAJOR.MINOR.PATCH
git rev-parse vMAJOR.MINOR.PATCH
```
If the intended tag exists remotely, leave it immutable. If it is absent and
you have confirmed that the local tag was never published or used by a release
channel, remove only that unpublished local tag, then rerun the command. A
failed remote query is not proof of absence. Network/authentication failures
alone do not require a new version. A defect requiring changes to published
source requires a new semantically appropriate version and matching note.
## Optional Diagnostics
`./scripts/check-release-source.sh <version>` runs validation without publication.
It is useful for diagnosis, but is not a required preliminary step that must
repeat the publication command's checks.
Race tests, vet, module hygiene, formatting, example checks, and cross-builds may
be run when relevant to a change; they are not additional release gates. The
development guide and testing policy govern change-specific verification.
A fresh source install or exact-tag checkout/build may be used to investigate
consumer availability after release. Such checks are optional and do not delay
or reverse release completion. `notarius --version` is diagnostic; subprocess
compatibility is defined by the receipt and artifact contracts.

View File

@@ -1,83 +0,0 @@
# Notarius v0.4.0
This release strengthens LLM reliability and validation throughout the
configured pipeline, upgrades the PromptKit integration, and establishes the
source-release and downstream-consumer workflows needed for broader D&D
pipeline integration.
## Summary
Notarius now distinguishes PromptKit structural-output repair from
application-owned semantic validation retries. Producer candidates can run
through complete deterministic validator chains, receive bounded semantic
correction guidance, and retry under explicit stage policies. Final run
receipts and manifests preserve bounded validation provenance, while outputs
that advance with incomplete validation remain available to the current run
without entering reusable checkpoint state.
The release also adds a maintained complete D&D subprocess-consumer workflow,
diagnostic build versions, and the source-only release procedure used to
publish this version.
## Compatibility
- Configuration files must use schema version 4. Version 3 is not decoded or
rewritten; rename the top-level `scriptorium` section to `promptkit` when
migrating. See [Configuration](../config.md#migrating-version-3-configuration).
- PromptKit is pinned to v0.9.0. Operator profile files use PromptKit's v0.9.0
format and may use its profile-inheritance support. Notarius continues to
resolve operator profiles before embedded fallbacks.
- Structural-output repair and semantic stage retries are separate bounded
mechanisms. Maintained production prompts request one structural repair by
default; explicit configuration can override the supported repair count.
- Validation policy can now fail a run, reject an output, or permit an
otherwise valid candidate to advance with incomplete-validation provenance.
The application defaults are documented in
[Configuration](../config.md#pipelines).
- The `notarius.run-result.v1` receipt remains at schema version 1 and adds
optional validation summaries plus a required validation-status field.
Consumers of this pre-release contract should follow the current
[run-result receipt](../integrations/run-result.md).
- Existing D&D artifact schema identities remain unchanged. Validation and
producer-policy changes can nevertheless cause previously accepted weak
candidates to retry, reject, or fail instead.
## Upgrade
1. Migrate every Notarius configuration to version 4 and rename `scriptorium`
to `promptkit`.
2. Review deployed PromptKit profiles against the pinned v0.9.0 profile format
and ensure their credential environment variables are available at run
time.
3. Run `notarius config validate --config <path> --pipeline <id>` before the
first production invocation.
4. Review `structured_output_repair_attempts`, producer retry counts, and
`validation_policy` wherever the deployment needs behavior different from
the documented defaults.
5. Update subprocess consumers to inspect receipt `validation_status` and to
tolerate the optional bounded `validation_summaries` field. A consumer that
requires fully validated artifacts should require `approved`.
## Changes
- Upgraded PromptKit from v0.5.0 through v0.9.0 and adopted profile
inheritance, structured-output repair, typed error classification, and the
correction-aware completion protocol.
- Added pipeline and binding configuration for structural repair and terminal
validation policy, with strict startup validation and effective-setting
provenance.
- Added feedback-aware retries for chunking, extraction, merge, normalize, and
semantic reconciliation producers. Retry prompts contain the exact defective
response and actionable semantic correction guidance without exposing
internal reason codes or opaque entity identifiers.
- Added complete validator-chain execution, validator retry handling, bounded
warnings, terminal dispositions, and durable validation summaries.
- Prevented validation-incomplete artifacts and all derived lineage from
loading or publishing reusable checkpoints while preserving same-run
generated-reference handoff.
- Tightened cached chunk-plan validation so only completely validated plans are
reused or replace stored plans.
- Added a machine-readable subprocess receipt workflow and complete D&D
consumer documentation covering all maintained artifacts.
- Added source-release checks, immutable lightweight-tag guidance, Linux and
Darwin build verification, and diagnostic `notarius --version` output.

View File

@@ -1,85 +0,0 @@
# Notarius v0.5.0
This release separates actionable process warnings from extraction-quality
advisories and routine normalization observations, giving operators a quiet
warning channel without discarding durable diagnostic detail.
## Summary
Notarius now carries one validated, origin-aware diagnostic contract from
producers and validators through retries, reusable state, output publication,
debug summaries, run receipts, and CLI presentation. Warnings are reserved for
process degradation or incomplete configured work. Data-quality findings are
advisories, and successful deterministic cleanup is recorded as observations.
An ordinary successful run therefore reports zero warnings while retaining
bounded diagnostic provenance for later review.
The framework aggregates findings deterministically by their stable identity
and complete pipeline origin, preserves exact occurrence counts, and retains
bounded representative samples. Warning groups fail rather than truncate;
advisory and observation representation is bounded with explicit truncation
metadata and exact unrepresented-occurrence counts.
## Compatibility
- `warnings.json` now uses the incompatible grouped
`notarius.warnings.v2` envelope and contains process warnings only. Consumers
of the former flat warning payload must migrate to the current
[JSON output contract](../integrations/json-output.md).
- The new `diagnostics.json` file uses `notarius.diagnostics.v1` and contains
advisory and observation groups. Production `index.json` files always expose
both `warnings_file` and `diagnostics_file`.
- The machine-readable run receipt is now `notarius.run-result.v2`. It replaces
`warning_count` with exact warning group and occurrence counts and adds
advisory/observation group, occurrence, and truncation fields. See the
current [run-result receipt](../integrations/run-result.md).
- Custom output modules must return their complete logical file set or an
error. The former `OutputResult.Warnings` field has been removed; an output
module cannot report a warning after serializing its output.
- Reusable state now uses `notarius.workspace.v4` and chunk-plan records use
`notarius.chunk-plan.v3` so they can preserve structured diagnostics. Older
pre-release reusable state is not reused under these contracts; start with
clean state when deterministic continuity with an older workspace is not
required.
- Validation acceptance, semantic retry budgets, rejection policy, and D&D
artifact schema identities are unchanged by this release.
## Upgrade
1. Update subprocess consumers to require `notarius.run-result.v2` and read
`warning_group_count`, `warning_occurrence_count`,
`diagnostic_group_count`, `diagnostic_occurrence_count`, and
`diagnostics_truncated`.
2. Update output-bundle consumers to decode `notarius.warnings.v2`, discover
`diagnostics.json` through `index.json`, and treat diagnostics as review
information rather than process warnings.
3. Update any custom output module for the removal of
`OutputResult.Warnings`; return an error when encoding cannot complete.
4. Clear pre-release reusable state before the first upgraded production run
when deterministic continuity with an older workspace is not required.
5. Run `notarius config validate --config <path> --pipeline <id>` and perform
one representative run before promoting the release in an automated
pipeline.
## Changes
- Added validated diagnostic dispositions, categories, origins, stable reason
codes, exact occurrence counts, and bounded representative samples.
- Added deterministic run-level aggregation with separate limits for
actionable warning groups and advisory/observation groups.
- Reclassified D&D source-relatedness and unresolved-identity findings as
data-quality advisories and routine normalization changes as observations.
- Preserved structured diagnostics across producer retries, validation,
generated-reference handoff, checkpoints, chunk-plan reuse, and debug
summaries while discarding superseded-attempt findings.
- Added grouped `warnings.json`, a new grouped `diagnostics.json`, and the
corresponding production index entries.
- Upgraded the machine-readable run receipt and human CLI summary to report
exact warning and diagnostic counts without allowing advisory volume to
create warning output.
- Removed post-encoding output warnings and hardened diagnostic validation,
overflow handling, aggregate memory bounds, and warning-file path
presentation.
- Documented diagnostic ownership, classification, operator interpretation,
durable contracts, and architectural invariants in ADR-0015 and the
canonical CLI, operations, integration, and internal documentation.

View File

@@ -1,84 +0,0 @@
# Notarius v0.6.0
This release improves the reliability and ergonomics of unattended,
subprocess-driven D&D extraction pipelines.
## Summary
Notarius now gives models more precise, semantically useful correction guidance
when deterministic or model-backed validation rejects an otherwise structured
candidate. Semantic reconciliation retries identify candidates with compact,
request-local numbers, preserve valid candidates when a proposal cannot be
repaired, and report bounded process warnings when reconciliation is
incomplete. D&D extraction also canonicalizes safely resolvable reversed source
ranges and applies more consistent schema, prompt, and validator constraints,
improving successful operation with smaller models without weakening evidence
validation.
An optional LLM-backed combat-semantics validator can review whether scene
descriptions correctly consolidate combat and apply the `combat` kind. For
subprocess callers, command-line references now follow the pipeline-scoped
configuration model: a shared reference can be supplied once and automatically
reach every compatible selected target, while lane- and binding-specific forms
remain available for exceptions.
## Compatibility
- The meaning of an unqualified `--reference slot=path` or
`--without-reference slot` selector has changed. It now applies to every
selected pipeline target that declares the slot instead of requiring exactly
one matching target. Review callers that relied on ambiguity rejection or a
unique implicit target; the current selector contract is documented in the
[CLI reference](../cli.md).
- The stage-wide `merge.slot` CLI reference shorthand has been removed. Use
`lane.merge.slot` for an exact merge binding, `lane.slot` for all compatible
bindings in one lane, or an unqualified `slot` for pipeline scope.
- Command-line file references cannot replace, remove, or coexist with a
generated artifact handoff for the same concrete target and slot. Resolution
reports the conflict so the caller can narrow or remove the selector.
- The private LLM-facing semantic-reconciliation contract now uses contiguous
`candidate_number` values rather than application identities. This is not a
durable or operator-configurable contract and requires no operator action.
- Durable D&D artifact schemas, the `notarius.run-result.v2` receipt,
`notarius.warnings.v2`, and `notarius.diagnostics.v1` are unchanged from
`v0.5.0`.
## Upgrade
1. Update subprocess commands to provide shared inputs once with
`--reference slot=path`, and convert any stage-wide `merge.slot` selector to
an explicit supported scope. The [subprocess consumer guide](../consumers/subprocess.md)
and [complete D&D consumer guide](../consumers/dnd-pipeline.md) show the
current invocation pattern.
2. Review CLI reference overrides that overlap generated same-run references;
narrow or remove an external selector rather than attempting to replace the
generated handoff.
3. Run `notarius config validate --config <path> --pipeline <id>` for every
deployed pipeline configuration.
4. Optionally add
`extract/dnd/scene-descriptions/combat_semantics` to the scene-description
validator chain where the deployment wants LLM-backed combat-scene review;
see the [configuration reference](../config.md).
5. Perform a representative run with the deployed model profile and inspect
the machine-readable receipt, warnings, and diagnostics before promotion.
## Changes
- Added feedback-aware, module-requested retry support for normalizers while
keeping retry budgets bounded and preserving attempt diagnostics.
- Reworked semantic reconciliation around compact, contiguous candidate
numbers, explicit namespace guidance, complete proposal validation, and
model-facing retry feedback that omits opaque internal identities.
- Preserved candidates separately when an exhausted semantic proposal cannot
be safely applied, with a bounded process warning describing the fallback.
- Canonicalized safely resolvable reversed D&D source-reference endpoints
before deterministic coverage validation and clarified the shared evidence
prompt.
- Improved D&D schemas, prompts, and validators with closed-value constraints,
contextual correction guidance, shared source-range coverage logic, and
clearer item-holder transition rules.
- Added the optional D&D scene-description combat-semantics validator, shared
combat policy assets, evaluation fixtures, and retry-path coverage.
- Made unqualified CLI references pipeline-scoped, added hierarchical lane and
exact-binding selectors, defined deterministic override precedence, and
documented first-class subprocess use.

Some files were not shown because too many files have changed in this diff Show More