Compare commits

..

340 Commits

Author SHA1 Message Date
bef8ca263b Tighten evidence context source excerpts 2026-08-09 20:04:46 +00:00
f6d037b613 Document evidence context source unit excerpts 2026-08-09 19:47:27 +00:00
449b506804 Update evidence context output coverage 2026-08-09 19:45:21 +00:00
071a78ae22 Replace evidence context with source unit excerpts 2026-08-09 19:42:33 +00:00
ad1cba41c2 Fix OpenAI reconciliation schema 2026-08-09 18:49:29 +00:00
67338798aa Complete the semantic reconciliation roadmap 2026-08-09 18:30:32 +00:00
e95e2f2220 Finalize semantic reconciliation documentation 2026-08-09 17:05:17 +00:00
628b8d1800 Remove legacy reconciliation path 2026-08-09 16:57:44 +00:00
d24d4609b6 Migrate location reconciliation to shared engine 2026-08-09 16:51:29 +00:00
c7f79fb38e Migrate item reconciliation to shared engine 2026-08-09 16:46:43 +00:00
8c071800cf Migrate NPC reconciliation to shared engine 2026-08-09 16:38:43 +00:00
569e12c6f4 Add generic reconciliation plan application 2026-08-09 16:27:36 +00:00
5b6eb591b2 Add shared semantic reconciliation engine 2026-08-09 16:19:54 +00:00
b630384aa0 Add generic semantic reconciliation prompt assets 2026-08-09 16:08:53 +00:00
297d58f090 Add semantic reconciliation proposal validation 2026-08-09 16:00:01 +00:00
ee71dc4937 Add bounded semantic candidate preparation 2026-08-09 15:52:55 +00:00
65e5d65d14 Record semantic reconciliation architecture decision 2026-08-09 15:43:05 +00:00
b40b40aaf3 Plan semantic reconciliation improvements 2026-08-09 15:38:10 +00:00
f120be1cb4 Add proposed changes to shared LLM prompts 2026-08-09 09:18:36 -05:00
17673d74ea Archive the completed codebase audit 2026-08-09 13:24:12 +00:00
ef19a03cbf Reconcile remediation documentation 2026-08-09 02:45:22 +00:00
0546f6eb4f Simplify module cleanup paths 2026-08-09 02:40:25 +00:00
d28d1062e0 Reuse canonical item occurrence evidence 2026-08-09 02:36:53 +00:00
b3ebfcef37 Index enemy event duplicate identities 2026-08-09 02:31:28 +00:00
b70d9f77e3 Improve D&D registry normalization efficiency 2026-08-09 02:26:49 +00:00
2a75f40871 Bound D&D normalization diagnostics 2026-08-09 02:22:23 +00:00
a705ba74a1 Project spell aliases into extraction prompts 2026-08-09 02:17:15 +00:00
8d9c9e7c87 Align item occurrence evidence fields 2026-08-09 02:11:27 +00:00
0b5cc4f251 Require chunk-local extraction evidence 2026-08-09 02:08:11 +00:00
82ffe85f2d Enforce durable enemy event validation 2026-08-09 02:03:27 +00:00
b3644abc0e Enforce durable D&D evidence ranges 2026-08-09 01:54:43 +00:00
d653bf1b90 Share immutable in-memory filesystems 2026-08-09 01:47:44 +00:00
3e66127b94 Write durable outputs through confined file writer 2026-08-09 01:41:37 +00:00
5d086c13ca Cache compiled JSON schemas per validator 2026-08-09 01:38:42 +00:00
d36d4e7689 Avoid redundant decoded graph clones 2026-08-09 01:34:26 +00:00
1456aa51cc Index generated reference handoffs 2026-08-09 01:30:10 +00:00
ffc179c822 Centralize builder request cloning 2026-08-09 01:27:28 +00:00
8e669a1f14 Preserve terminal rejection warnings 2026-08-09 01:19:55 +00:00
14bfae216d Isolate typed validator candidates 2026-08-09 01:11:03 +00:00
5a58d87995 Require candidate decoders for artifact codecs 2026-08-09 01:03:50 +00:00
557809f364 Add candidate artifact codec decoding 2026-08-09 00:57:03 +00:00
ee600975f0 Prevent scheduler callbacks after cancellation 2026-08-09 00:52:31 +00:00
0d8017e23f Gate runner dispatches on cancellation 2026-08-09 00:50:02 +00:00
37b18edf3d Contain provider errors at the LLM adapter 2026-08-09 00:44:38 +00:00
cda7a61b47 Encode checkpoint and debug path identities 2026-08-09 00:41:28 +00:00
2ad9283148 Bound external reference reads and fingerprints 2026-08-09 00:33:33 +00:00
41a8a80dda Reject ambiguous configuration and reference bindings 2026-08-09 00:29:04 +00:00
90c7fa6381 Finalize codebase audit synthesis 2026-08-08 23:00:28 +00:00
e3839f8620 Audit combat and enemy event processing 2026-08-08 22:54:00 +00:00
0fc2f9ee01 Audit spell and scene processing 2026-08-08 22:42:49 +00:00
5d6305f21a Audit NPC item and location occurrences 2026-08-08 22:30:23 +00:00
551e4daea2 Audit NPC item and location registries 2026-08-08 22:18:29 +00:00
3589d33468 Audit shared D&D family conventions 2026-08-08 22:08:24 +00:00
a22c1a7f59 Audit generic and Seriatim modules 2026-08-08 21:59:06 +00:00
ad85d71b0f Audit LLM runtime and prompt assets 2026-08-08 21:47:51 +00:00
f3506240c2 Audit state persistence and file safety 2026-08-08 21:37:57 +00:00
70c199aa31 Audit runtime execution and concurrency 2026-08-08 21:27:37 +00:00
e2b82746ab Audit reference materialization and ordered handoffs 2026-08-08 21:16:30 +00:00
4235507f7b Audit pipeline composition and typed registries 2026-08-08 21:06:28 +00:00
b346670cc7 Audit configuration and CLI composition 2026-08-08 20:56:50 +00:00
7868c26be7 Establish codebase audit baseline 2026-08-08 20:50:15 +00:00
92e89076a2 Add an audit plan and staged audit sequence to identify opportunities for code quality improvement 2026-08-08 20:42:35 +00:00
d9b87347b8 Simplify contextual entity grounding 2026-08-08 15:47:41 +00:00
20397ef710 Document deterministic entity identity resolution 2026-08-08 15:08:57 +00:00
fc449863f2 Use contextual descriptors for entity reconciliation 2026-08-08 15:05:35 +00:00
51d62de1f3 Ground location occurrences with contextual selectors 2026-08-08 14:56:43 +00:00
fc76805075 Add contextual location grounding 2026-08-08 14:49:08 +00:00
8e680cf96e Ground item occurrences by canonical names 2026-08-08 14:43:04 +00:00
ece1bca460 Ground NPC occurrences by canonical names 2026-08-08 14:37:54 +00:00
516af12916 Finish D&D entity contract cleanup 2026-08-08 13:14:42 +00:00
1015d61b2d Clarify NPC mention eligibility 2026-08-06 13:40:42 +00:00
4192aa8584 Harden item occurrence grounding 2026-08-06 13:37:21 +00:00
8cf03a2a44 Reject unsafe currency reconciliation proposals 2026-08-06 13:31:15 +00:00
2ee6b495e1 Consolidate duplicate item registry records 2026-08-06 13:28:12 +00:00
f1b120b590 Complete D&D entity contract audit 2026-08-05 20:17:20 +00:00
2ec17f5b4f Consolidate D&D entity documentation 2026-08-05 20:12:16 +00:00
6de470d541 Verify complete D&D entity handoffs 2026-08-05 20:07:31 +00:00
a6e176e160 Move item occurrences to canonical namespace 2026-08-05 20:00:20 +00:00
3dfefd0e14 Adopt registry-backed item occurrences 2026-08-05 19:55:34 +00:00
f91237e9c0 Register item registry normalization 2026-08-05 19:39:06 +00:00
ad9ee076b5 Add item registry extraction and validation 2026-08-05 19:27:49 +00:00
ce04387dbc Add item registry domain foundation 2026-08-05 19:19:58 +00:00
e8965ebbbb Move location registry modules to canonical namespace 2026-08-05 19:13:37 +00:00
c006b163d5 Adopt location registry durable contract 2026-08-05 19:05:42 +00:00
2f61118e78 Move NPC occurrences to their canonical namespace 2026-08-05 19:00:55 +00:00
5e2ccffc0f Adopt registry-backed NPC occurrence artifacts 2026-08-05 18:55:58 +00:00
3f4a1f2647 Move NPC registry to canonical namespace 2026-08-05 18:38:22 +00:00
9653e06297 Migrate NPC registry durable contract 2026-08-05 18:25:04 +00:00
916a32195b Plan canonical D&D entity registries and occurrences 2026-08-05 18:16:37 +00:00
299c110267 Tighten D&D prompt contracts and tests 2026-08-05 17:38:51 +00:00
4093ff2e8b Document D&D prompt conventions and verification 2026-08-05 15:05:24 +00:00
39563f3ea0 Normalize private D&D response schema identities 2026-08-05 14:59:03 +00:00
4b52cace76 Consolidate grounded event extraction instructions 2026-08-05 14:55:40 +00:00
16d1b29b14 Consolidate spell and combat turn instructions 2026-08-05 14:52:26 +00:00
856b26b718 Consolidate NPC location and item event instructions 2026-08-05 14:48:55 +00:00
ab70347c5d Consolidate scene and normalization prompt instructions 2026-08-05 14:42:54 +00:00
8ed99ceaee Standardize D&D prompt asset filenames 2026-08-05 14:37:18 +00:00
257bca0dc2 Define D&D prompt transcript scopes 2026-08-05 14:33:00 +00:00
bad5db3ef3 Plan the D&D prompt asset cleanup 2026-08-05 14:28:05 +00:00
406ad1d362 Retire the completed asset migration roadmaps 2026-08-05 13:23:00 +00:00
5f207b2ab1 Document centralized LLM asset ownership 2026-08-05 00:49:15 +00:00
c613b306ae Centralize item and enemy event assets 2026-08-05 00:45:04 +00:00
1c7291e17d Centralize spell and combat-turn assets 2026-08-05 00:43:39 +00:00
2345da106a Centralize location LLM assets 2026-08-05 00:42:22 +00:00
03761c97dd Centralize NPC LLM assets 2026-08-05 00:40:47 +00:00
9cb9ee48a7 Centralize scene planning and description assets 2026-08-05 00:38:55 +00:00
08954f17e2 Centralize shared D&D LLM assets 2026-08-05 00:37:19 +00:00
a57f83e30d Centralize generic LLM schema assets 2026-08-05 00:34:56 +00:00
f4c05c34ef Make item event responses compatible with strict schemas 2026-08-04 19:52:47 +00:00
29fcad6e9b Improve D&D registry caching and retire the completed roadmap 2026-08-04 18:29:17 +00:00
f5fd115046 Migrate location registry to shared resolver 2026-08-04 13:30:12 +00:00
7f28899730 Migrate NPC registry to shared resolver 2026-08-04 13:26:06 +00:00
84c0758455 Add shared D&D registry resolver 2026-08-04 13:20:20 +00:00
5002864e88 Narrow location occurrence normalizer references 2026-08-04 13:12:46 +00:00
55b188fd84 Clarify hypothetical location occurrence classification 2026-08-04 13:09:30 +00:00
d1f43df88e Restore NPC canonical name selection 2026-08-04 13:08:00 +00:00
d52387c1f7 Document D&D location tracking contracts 2026-08-04 00:51:43 +00:00
9c5e3cff14 Add D&D location tracking to complete example 2026-08-04 00:45:50 +00:00
811d5b8bd9 Compose D&D location tracking modules 2026-08-04 00:39:00 +00:00
a168c13b85 Add D&D location occurrence validators 2026-08-04 00:33:39 +00:00
dd61a4efda Add D&D location validators 2026-08-04 00:26:50 +00:00
228cc6ee83 Add D&D location occurrence normalizer 2026-08-04 00:22:25 +00:00
06170e1f65 Add D&D location occurrence extractor 2026-08-04 00:18:16 +00:00
7715baa1f6 Add immutable D&D location registry 2026-08-04 00:11:31 +00:00
98506db1a9 Add D&D location normalizer 2026-08-04 00:07:47 +00:00
bb4855f0c6 Add D&D location extractor 2026-08-04 00:02:18 +00:00
c51934d5c6 Migrate NPC normalization to shared reconciliation 2026-08-03 23:56:48 +00:00
c3513da880 Add shared D&D entity reconciliation support 2026-08-03 23:47:42 +00:00
c7d853ea52 Add D&D location artifact codecs 2026-08-03 23:40:35 +00:00
da7fcdaffd Add D&D location identity contracts 2026-08-03 23:35:50 +00:00
b6aad4fa98 Plan D&D location tracking 2026-08-03 23:29:32 +00:00
9c6af28d02 Finish the enemy engagement cleanup 2026-08-03 23:11:44 +00:00
04eabdfcb9 Complete enemy event reference documentation 2026-08-03 23:00:09 +00:00
a8a99c1037 Make enemy event prompt tests resilient to refactoring 2026-08-03 22:57:17 +00:00
e15007fffb Simplify enemy event grounding ownership 2026-08-03 22:54:11 +00:00
e6b7c61f45 Enforce unique enemy engagements per scene 2026-08-03 22:50:41 +00:00
42973215fa Document D&D enemy event artifacts 2026-08-03 21:14:52 +00:00
ba1d112d1f Add D&D enemy event pipeline example 2026-08-03 21:10:21 +00:00
b722131d57 Compose D&D enemy event production family 2026-08-03 21:01:23 +00:00
a92d2c0885 Add D&D enemy event validators 2026-08-03 20:56:36 +00:00
02ec10d66b Add D&D enemy event normalizer 2026-08-03 20:49:32 +00:00
9dd57dbfa4 Add D&D enemy event extractor 2026-08-03 20:44:44 +00:00
c164a3fc69 Prepare D&D enemy event grounding references 2026-08-03 20:37:35 +00:00
8834df617f Add D&D enemy event artifact contract 2026-08-03 20:32:20 +00:00
db2adb52da Preserve prompt sessions and retire completed roadmaps 2026-08-03 19:45:27 +00:00
fc3c128171 Document prompt sessions and concurrency defaults 2026-08-03 19:16:18 +00:00
8a15b083a0 Raise default LLM concurrency 2026-08-03 19:12:19 +00:00
d9dae2b639 Record effective sessions in debug provenance 2026-08-03 19:09:49 +00:00
7a4fd7be7a Derive stable prompt sessions for CLI runs 2026-08-03 19:05:49 +00:00
39388e96d4 Make PromptKit profile handling safer and more consistent 2026-08-03 18:35:40 +00:00
12ac25bd63 Sanitize PromptKit profile fingerprint errors 2026-08-03 17:25:44 +00:00
394278e1f2 Document workload-oriented LLM profile deployment 2026-08-03 17:21:42 +00:00
5cd7f8e737 Expose pipeline LLM profile defaults 2026-08-03 17:15:07 +00:00
bf3fadf9ae Resolve pipeline LLM profile defaults 2026-08-03 17:09:16 +00:00
58815aaf33 Require explicit module execution classes 2026-08-03 17:00:08 +00:00
ce857966f1 Add execution metadata to module specifications 2026-08-03 16:49:51 +00:00
a3bd0c1867 Add D&D extraction fallback profile 2026-08-03 16:39:45 +00:00
b05634ee86 Add fallback PromptKit profile assets 2026-08-03 16:34:48 +00:00
4829f94157 Inspect PromptKit profiles during preflight 2026-08-03 16:26:15 +00:00
67b315099d Execute PromptKit requests from prepared snapshots 2026-08-03 16:17:40 +00:00
b5c86de4d7 Upgrade PromptKit to version 0.5.0 2026-08-03 16:12:29 +00:00
2eeca2ed5a Plan the PromptKit upgrade and profile workflow 2026-08-03 16:07:37 +00:00
b5aaeb1c78 Update future roadmap document with ideas for new feature developments 2026-07-30 16:27:36 +00:00
9171b66a41 Clarify PromptKit configuration and retire the completed plan 2026-07-30 15:38:31 +00:00
b4363b3b73 Document local PromptKit backend support 2026-07-30 05:22:41 +00:00
241e9d2a89 Include local backend target in checkpoint identity 2026-07-30 05:17:39 +00:00
715fff7b72 Register configured local PromptKit backend 2026-07-30 05:14:22 +00:00
d627b91b4f Add local PromptKit backend configuration 2026-07-30 05:09:54 +00:00
a67b3aa76d Upgrade PromptKit to version 0.3.0 2026-07-30 05:06:32 +00:00
a16dcdfa52 Clean up the PromptKit upgrade 2026-07-30 02:58:25 +00:00
46e4466d28 Translate LLM backend capacity failures 2026-07-30 02:23:51 +00:00
71a004bfc8 Publish effective LLM backend provenance 2026-07-30 02:18:14 +00:00
f8333f2c15 Expose run-wide reasoning effort controls 2026-07-30 02:11:35 +00:00
f603f7ac64 Support internal reasoning effort overrides 2026-07-30 02:03:04 +00:00
7a00e7049c Forward sessions through PromptKit requests 2026-07-30 01:58:40 +00:00
2a9db9a957 Upgrade PromptKit to version 0.2.0 2026-07-30 01:54:47 +00:00
de046a8f13 Keep checkpoints aligned with PromptKit profiles 2026-07-28 13:36:01 -05:00
f1a6574013 Document the PromptKit integration 2026-07-28 16:52:49 +00:00
4bca6d3103 Align run provenance with PromptKit 2026-07-28 16:47:07 +00:00
7c569a3d8c Make D&D prompt assets provider neutral 2026-07-28 16:42:41 +00:00
8e04ef9e2b Introduce version 4 PromptKit configuration 2026-07-28 16:38:05 +00:00
53a330587b Replace the Scriptorium adapter with PromptKit 2026-07-28 16:31:36 +00:00
7cfab8ada0 Add a staged implementation plan for migration to the promptkit LLM library 2026-07-28 11:20:31 -05:00
5c82b62856 Retire completed roadmap documents and add ideas for future feature work 2026-07-27 20:20:00 -05:00
de8ed41b34 Document D&D prompt cache ordering 2026-07-27 20:16:11 +00:00
d1eaec4dad Consolidate D&D prompt ordering tests 2026-07-27 20:14:57 +00:00
c0ec068f53 Reorder D&D scene and normalization prompts 2026-07-27 20:08:09 +00:00
5cbd9e56e4 Reorder D&D extraction prompt prefixes 2026-07-27 20:07:09 +00:00
53490cdb59 Add a new feature roadmap and implementation plan for D&D prompt ordering to improve LLM provider caching 2026-07-27 15:04:32 -05:00
7c94b5eeed Clarify evidence context and test lane filtering 2026-07-27 14:03:23 -05:00
4f2864fc96 Stop tracking generated codebase memory files 2026-07-27 13:56:04 -05:00
893b03fccf Document published evidence context 2026-07-27 18:28:02 +00:00
256cc98ddb Publish evidence context in JSON output bundles 2026-07-27 18:23:48 +00:00
e61e522662 Build evidence context for output encoders 2026-07-27 18:17:54 +00:00
a4c7eca87b Add evidence context artifact builder 2026-07-27 18:11:49 +00:00
224a8292c4 Add evidence context policy preparation 2026-07-27 18:04:12 +00:00
64d461fc18 Add a feature roadmap and implementation plan for a context evidence artifact 2026-07-27 12:09:14 -05:00
fb1134e591 Address code review findings from the subprocess enhancements 2026-07-26 18:52:37 -05:00
1da29e6788 Document subprocess run results 2026-07-26 17:22:27 +00:00
0d947549fb Test JSON run result behavior 2026-07-26 17:17:35 +00:00
950fba17ce Add JSON run result mode 2026-07-26 17:14:16 +00:00
678d2c6099 Add run result wire model 2026-07-26 17:12:57 +00:00
db8db5ffc5 Add a plan to implement improved documentation and support for running notarius as a subprocess 2026-07-26 12:08:53 -05:00
94b3eafb1a Finish the documentation refresh 2026-07-26 13:53:38 +00:00
f6981e2264 Harmonize D&D artifact contracts 2026-07-26 13:43:45 +00:00
2f506f4985 Harmonize foundational integration contracts 2026-07-26 13:38:55 +00:00
fdf8c4afd4 Rewrite LLM runtime documentation 2026-07-26 13:33:45 +00:00
74c793e6a1 Document D&D module conventions 2026-07-26 13:30:36 +00:00
b5835fbc37 Rewrite pipeline and component overview documentation 2026-07-26 13:26:20 +00:00
fd3f7b85cc Rewrite operations and state documentation 2026-07-26 13:24:02 +00:00
d86b74f485 Document configuration and CLI internals 2026-07-26 13:20:18 +00:00
59cbf1eb27 Rewrite configuration reference 2026-07-26 13:16:06 +00:00
ee43add75c Rewrite CLI orientation and reference 2026-07-26 13:11:40 +00:00
46761706a2 Add complete D&D example transcript 2026-07-26 13:07:59 +00:00
5a968b64eb Clean up completed roadmap work 2026-07-25 21:50:39 -05:00
7b077c269d Clarify normalize fallback ownership 2026-07-26 02:44:29 +00:00
80ec939383 Validate normalize retry diagnostics 2026-07-26 02:43:28 +00:00
d3c4d6f133 Simplify NPC normalization prompt guidance 2026-07-26 02:40:08 +00:00
5ad661f95f Redact NPC normalization context errors 2026-07-26 02:38:03 +00:00
d63e5c6852 Plan NPC normalization follow-up fixes 2026-07-25 21:34:42 -05:00
fbb8e0d241 Document NPC semantic normalization 2026-07-26 01:46:23 +00:00
8d9a496935 Integrate NPC semantic normalization 2026-07-26 01:41:35 +00:00
d1c48db4bc Add LLM-assisted NPC normalization 2026-07-26 01:36:32 +00:00
6bd781d344 Add NPC normalization prompt context 2026-07-26 01:27:34 +00:00
8a12c56971 Add retryable normalization fallbacks 2026-07-26 01:20:28 +00:00
26bd59a5a2 Plan LLM-assisted NPC normalization 2026-07-25 20:13:12 -05:00
927a7beb88 Clarify and streamline D&D item events 2026-07-25 23:59:56 +00:00
f7059607af Document D&D item event support 2026-07-25 22:16:29 +00:00
63de44c347 Add D&D item event integration coverage 2026-07-25 22:10:53 +00:00
f0ede9dacc Register D&D item event pipeline 2026-07-25 22:05:52 +00:00
5711f8b9e3 Add D&D item event normalization 2026-07-25 22:00:32 +00:00
da83510234 Add D&D item event validators 2026-07-25 21:52:59 +00:00
f51b22bea7 Add D&D item event extractor 2026-07-25 21:49:24 +00:00
f320c2fcee Add D&D item event artifact contract 2026-07-25 21:41:17 +00:00
4ba1e50a89 Plan D&D item event tracking 2026-07-25 21:33:43 +00:00
2a7e025251 Improve scene-aware combat gating 2026-07-25 21:08:24 +00:00
3da20e9d6a Document scene-aware combat extraction 2026-07-25 19:34:33 +00:00
b1c0faa748 Test scene-aware combat handoff behavior 2026-07-25 19:29:15 +00:00
989f2c220b Gate combat extraction on scene descriptions 2026-07-25 19:23:19 +00:00
7e35915b3e Add scene eligibility registry 2026-07-25 19:15:33 +00:00
24238d249e Plan scene-aware combat extraction 2026-07-25 18:59:57 +00:00
9614469b45 Consolidate the D&D configuration examples 2026-07-25 15:48:48 +00:00
29ee68824d Finish the D&D module cleanup 2026-07-25 13:30:23 +00:00
aeaaf44ae0 Remove unused NPC registry and identity aliases 2026-07-25 13:06:40 +00:00
d752c51aec Centralize D&D candidate JSON codecs 2026-07-25 13:05:02 +00:00
8199d95dc1 Reuse document indexes in D&D normalization 2026-07-25 13:01:31 +00:00
97cdb01357 Reuse document indexes in D&D validators 2026-07-25 12:57:31 +00:00
7a66095912 Reuse indexes for D&D citation validation 2026-07-25 12:53:48 +00:00
9d1356a20e Add D&D refactoring plan and isolate merger results 2026-07-25 12:51:36 +00:00
e4471fc300 Fix D&D extraction issues and retire the completed audit 2026-07-25 12:30:42 +00:00
84a2854b5e Close D&D extraction audit documentation 2026-07-24 14:48:13 +00:00
a1b76093ce Document D&D extractor contract 2026-07-24 14:46:19 +00:00
1aa30a73db Protect D&D citation prompt ordering 2026-07-24 14:44:10 +00:00
dc7c0e2f9e Move scene semantics to deterministic validation 2026-07-24 14:40:54 +00:00
e2cb0d901a Centralize D&D extraction request preparation 2026-07-24 14:38:42 +00:00
8e0b029f5f Order combat and NPC interaction extraction by document position 2026-07-24 14:33:34 +00:00
9bbf2535dd Order spell and NPC extraction by document position 2026-07-24 14:29:19 +00:00
83fde83a58 Preserve literal ordering for foreign source references 2026-07-24 14:25:02 +00:00
bef3d1359d Fingerprint D&D extractor mapping policies 2026-07-24 14:23:35 +00:00
1ff449435f Centralize D&D source reference ordering 2026-07-24 14:20:10 +00:00
6e21c83fd8 Add D&D module code audit implementation plan 2026-07-24 14:08:36 +00:00
6dc9d522b1 Audit the D&D modules 2026-07-24 13:43:53 +00:00
8adcf6840d Clean up completed D&D scene chunking work 2026-07-24 03:14:34 +00:00
f5ed30e455 Retire annotated D&D scene plans 2026-07-24 00:40:53 +00:00
cacf3f24e7 Simplify D&D scene chunking responses 2026-07-24 00:35:10 +00:00
f08ca4ddfa Plan simpler D&D scene chunking 2026-07-24 00:29:46 +00:00
e1c2f3c202 Harden D&D scene description extraction 2026-07-23 23:03:37 +00:00
9614eb540d Format shared D&D prompt assets 2026-07-23 20:30:47 +00:00
1b46596a39 Document D&D scene description artifacts 2026-07-23 20:29:01 +00:00
e043d61a99 Register D&D scene description pipeline 2026-07-23 20:22:05 +00:00
ad89782c9b Add D&D scene description validation 2026-07-23 20:17:18 +00:00
cd29265d5d Add D&D scene description extractor 2026-07-23 20:09:59 +00:00
2f36b7c3b6 Add D&D scene description codec 2026-07-23 20:02:37 +00:00
b8b3f3abfa Plan D&D scene description extraction 2026-07-23 18:59:34 +00:00
b490297cde Harden chunk map export and retire completed plans 2026-07-23 17:38:04 +00:00
06148074a2 Document accepted chunk map export 2026-07-23 15:06:33 +00:00
16a998055c Add opt-in JSON chunk map export 2026-07-23 15:02:01 +00:00
97c9a8e5ce Carry accepted chunk maps through the runner 2026-07-23 14:58:14 +00:00
66415fd1fa Add accepted chunk map contract 2026-07-23 14:52:25 +00:00
bfe25609a7 Tighten NPC interaction validation and consistency 2026-07-23 14:25:32 +00:00
36e0512454 Complete D&D NPC interaction integration 2026-07-23 13:59:02 +00:00
b02f667107 Register NPC interaction production lane 2026-07-23 13:49:34 +00:00
ed2b6f4580 Add NPC interaction normalization 2026-07-23 13:42:37 +00:00
cb7f145c76 Add NPC interaction validators 2026-07-23 13:35:15 +00:00
2b9d2eaeaa Add grounded NPC interaction extractor 2026-07-23 13:26:27 +00:00
61016671ab Add D&D NPC interaction artifact codec 2026-07-23 13:18:38 +00:00
b2c076946b Plan chunk map exports and scene descriptions 2026-07-23 04:33:18 +00:00
250c5c22b8 Align the spell response schema 2026-07-23 03:57:55 +00:00
4b0b166143 Complete minimal D&D extraction cutover 2026-07-22 19:23:10 +00:00
90481a0e4b Simplify D&D combat turn extraction contract 2026-07-22 19:14:14 +00:00
2cbaf20e55 Minimize D&D spell extraction contracts 2026-07-22 19:01:58 +00:00
a263a0840c Minimize D&D NPC extraction contracts 2026-07-22 18:51:26 +00:00
14991cf58b Plan simpler D&D extraction contracts 2026-07-22 18:35:25 +00:00
ab0b4e350c Relax private D&D response schemas 2026-07-22 14:38:35 +00:00
7b2fb0880d Align D&D validator ordering 2026-07-22 14:30:20 +00:00
748e02db80 Harden checkpoint reuse and combat validation 2026-07-22 14:24:03 +00:00
23c55f8925 Add implementation roadmap to clean up D&D extract module validation issues 2026-07-22 09:19:13 -05:00
906d97b391 Reconcile ordered pipeline documentation 2026-07-22 02:40:36 +00:00
f15fd4f9c1 Add recompute recovery acceptance coverage 2026-07-22 02:35:44 +00:00
7aadb088a6 Hydrate accepted producer checkpoints 2026-07-22 02:27:19 +00:00
9de399432e Add stable checkpoint decision diagnostics 2026-07-22 02:13:50 +00:00
5bdd56cfb1 Decompose pipeline runner orchestration 2026-07-22 02:03:27 +00:00
64ea23c21f Audit the implementation and add a roadmap with cleanup items and bugfixes 2026-07-22 01:30:17 +00:00
7071102ab7 Document ordered pipeline operations and retire sequential examples 2026-07-21 22:25:11 +00:00
9184072839 Implement operation-time D&D NPC artifact handoff 2026-07-21 22:12:01 +00:00
c437682407 Implement selective checkpoint recomputation 2026-07-21 21:59:48 +00:00
22d4f29670 Implement generated artifact handoff and provenance 2026-07-21 21:39:23 +00:00
afb7ed3cf1 Execute ordered pipeline steps with barriers 2026-07-21 21:19:00 +00:00
f846f252c0 Implement ordered pipeline step resolution 2026-07-21 21:05:28 +00:00
f5618d1f0c Plan the ordered pipeline steps implementation 2026-07-21 20:28:04 +00:00
f94ab0a6bf Remove completed roadmap documentation and add future work to the long-term roadmap 2026-07-21 20:01:46 +00:00
41b52aae74 Put stable D&D prompt content before transcripts 2026-07-21 18:59:59 +00:00
ed36f7d7fd Finish the D&D audit fixes 2026-07-21 16:08:28 +00:00
3ba2bfd7f6 Finalize D&D prompt integration and documentation 2026-07-21 14:58:26 +00:00
b344d16dc1 Clarify D&D registration composition 2026-07-21 14:48:05 +00:00
6e12c09952 Align D&D validator policies and diagnostics 2026-07-21 14:43:20 +00:00
07460341e3 Share D&D cited source traversal 2026-07-21 14:37:12 +00:00
732b13669f Centralize D&D chunk prompt material preparation 2026-07-21 14:27:39 +00:00
3a8a82ebc9 Reorder shared D&D extraction prompt messages 2026-07-21 14:22:46 +00:00
1c9819f08e Make D&D prompt asset manifests exact 2026-07-21 14:13:11 +00:00
447c4f73f9 Protect deterministic D&D prompt inputs 2026-07-21 14:04:52 +00:00
e01b8d1b6d Use shared assets instead of prompt prefix change detectors 2026-07-21 13:59:28 +00:00
3d70920f3d Add the D&D harmonization roadmap and implementation plan 2026-07-21 13:53:04 +00:00
110593ece1 Close out the D&D combat turn extraction roadmap 2026-07-21 07:49:57 -05:00
a1f5dce405 Compose production D&D combat pipeline 2026-07-21 05:42:33 +00:00
50aa60e0b8 Add combat turn normalization and invariants 2026-07-21 05:22:12 +00:00
2fbb3813aa Add standalone D&D combat turn extraction 2026-07-21 05:08:02 +00:00
6dd695611c Add durable D&D combat turn artifacts 2026-07-21 04:48:55 +00:00
92acb45775 Share the D&D NPC registry and prompt grounding 2026-07-21 04:34:56 +00:00
d3e171aa82 Add feature roadmap and implementation plan for a D&D combat turn extraction module 2026-07-20 23:22:03 -05:00
c6f330eb06 Implement NPC extraction follow-up fixes 2026-07-20 23:10:25 -05:00
20cfbfd311 Add NPC registry grounding for spell extraction 2026-07-21 03:12:03 +00:00
fb043325e1 Add production NPC pipeline composition 2026-07-21 02:50:37 +00:00
06c0259788 Add NPC normalization and identity validation 2026-07-21 02:36:59 +00:00
3d5fd9dc05 Add D&D NPC extraction and validation 2026-07-21 02:24:26 +00:00
3ba2c62cc1 Add D&D NPC artifact contract and identity codec 2026-07-21 02:08:46 +00:00
e2ab01f9d2 Add feature roadmap and implementation plan for a D&D NPC extraction module 2026-07-20 20:58:29 -05:00
5186e061a8 Minor additional improvements to the D&D spell extraction prompt. 2026-07-20 20:34:59 -05:00
c4c907d421 Hardened D&D spell extraction prompts 2026-07-20 20:25:23 -05:00
fa5076f5f1 Updated .gitignore to ignore local workspace for testing 2026-07-20 20:25:14 -05:00
636 changed files with 67706 additions and 7044 deletions

3
.gitignore vendored
View File

@@ -1,6 +1,8 @@
# build and testing artifacts # build and testing artifacts
notarius notarius
notarius-output notarius-output
workspace/
.codebase-memory/
# ---> Go # ---> Go
# If you prefer the allow list template instead of the deny list, see community template: # If you prefer the allow list template instead of the deny list, see community template:
@@ -72,4 +74,3 @@ Icon
Network Trash Folder Network Trash Folder
Temporary Items Temporary Items
.apdisk .apdisk

View File

@@ -1,33 +1,46 @@
# Notarius # Notarius
Notarius is a Go CLI for extracting structured artifacts from source material Notarius is a Go CLI for turning source material into structured artifacts with
with explicit, configurable pipeline modules. configured extraction pipelines. The implemented D&D workflow reads Seriatim
transcript JSON and can produce NPC, location, and item registries; their
source-grounded occurrences; scene descriptions, combat turns, enemy events,
and spell casts.
The current implementation reads Seriatim transcript JSON, chunks the source ## Quickstart
units, extracts D&D spell-cast artifacts with a Scriptorium-backed LLM runtime,
and writes JSON output. Add `--debug` when a per-run inspection bundle is
needed.
```sh Provide an OpenRouter API key through the environment, then run the maintained
OPENROUTER_API_KEY=... \ minimal example:
go run ./cmd/notarius run dnd-session \
--config examples/dnd-spells.config.yml \
--input examples/seriatim-minimal-transcript.json
```
This invocation uses the maintained example configuration and input. See the ~~~
configuration and operations references for profile selection, credentials, and OPENROUTER_API_KEY=your-api-key \
run artifacts. go run ./cmd/notarius run dnd-session \
--config examples/dnd-minimal.config.yml \
--input examples/seriatim-minimal-transcript.json
~~~
Useful references: The command publishes a JSON output bundle. Its command syntax and exit
behavior are documented in the [CLI reference](docs/cli.md); configuration,
credentials, and module selection are owned by the
[configuration reference](docs/config.md).
- [CLI reference](docs/cli.md) For the complete ordered D&D workflow, use
- [Configuration reference](docs/config.md) [the complete configuration](examples/dnd-complete.config.yml) with
- [Operations](docs/operations.md) [its synthetic transcript](examples/dnd-complete-transcript.json). It
- [Seriatim input contract](docs/integrations/seriatim.md) demonstrates all implemented D&D lanes and the supporting campaign references.
- [JSON output contract](docs/integrations/json-output.md)
- [D&D spell artifact contract](docs/integrations/dnd-spell-artifacts.md) ## Documentation
- [Developer guide](docs/development.md)
- [Internal implementation docs](docs/internal/overview.md) - [CLI reference](docs/cli.md) — commands, flags, output streams, and exits.
- [Maintained example config](examples/dnd-spells.config.yml) - [Configuration reference](docs/config.md) — configuration files, profiles,
- [Maintained example input](examples/seriatim-minimal-transcript.json) validation, and module selection.
- [Operations](docs/operations.md) — output, state, recovery, and debug
handling.
- [Integration contracts](docs/integrations/) — Seriatim input and published
artifact formats.
- [Subprocess consumer guide](docs/consumers/subprocess.md) — invoke Notarius
from an orchestrator and consume a published result.
- [Internal overview](docs/internal/overview.md) — implemented component map
for maintainers.
- [Developer guide](docs/development.md) — contributor orientation and
validation guidance.
- [Future work](docs/roadmap/future.md) — unimplemented ideas and priorities.

View File

@@ -0,0 +1,18 @@
Extract Dungeons & Dragons combat-turn artifacts from the supplied transcript.
Include a record only when the transcript establishes that an in-world
participant takes a combat turn or performs a discrete interrupting combat
event. Keep events in transcript chronology; place an interrupting event where
it occurs.
Exclude initiative setup without a turn or combat event, tactical planning,
table talk, rules lookup, hypothetical events, abandoned intentions, recaps
outside the current passage, and downstream consequences. Do not infer combat
events from Dungeons & Dragons rules knowledge. Preserve the session as played
and attribute relevant nonstandard rulings to the GM or table. Unmatched actors
remain permitted.
Treat each record as one turn-level event and keep its supporting transcript
evidence together. Use `turn` for a regular combat turn, `reaction` for an
off-turn reaction, `legendary_action` for a legendary action,
`lair_action` for a lair action, and `other` for another discrete combat
event that does not fit those categories.

View File

@@ -0,0 +1,45 @@
id: dnd.combat_turns
version: "v1"
default_profile: dnd-extraction
inputs:
- name: transcript
required: true
content_type: application/json
- name: players
required: false
content_type: text/plain
- name: party
required: false
content_type: text/plain
- name: glossary
required: false
content_type: text/plain
- name: npc_registry
required: false
content_type: application/json
messages:
- role: system
content_file: ./sharedassets/common-dnd-system.md
- role: user
content_file: ./sharedassets/common-dnd-identity.md
- role: user
content_file: ./sharedassets/common-dnd-references.md
cache_control:
type: ephemeral
- role: user
content_file: ./sharedassets/common-dnd-transcript-chunk.md
cache_control:
type: ephemeral
- role: user
content_file: ./sharedassets/common-dnd-extraction-evidence.md
- role: user
content_file: ./sharedassets/common-dnd-npc-registry.md
- role: user
content_file: ./instructions.md
cache_control:
type: ephemeral
output:
format: json
validation_mode: json_schema
schema_path: dnd_combat_turns_llm.v1.json
repair_attempts: 0

View File

@@ -0,0 +1,41 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "notarius.dnd.combat_turns.llm",
"type": "object",
"additionalProperties": false,
"required": ["combat_turns"],
"properties": {
"combat_turns": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": false,
"required": ["actor", "turn_kind", "source_refs"],
"properties": {
"actor": {
"type": "string"
},
"turn_kind": {
"type": "string"
},
"source_refs": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": false,
"required": ["start_unit_id", "end_unit_id"],
"properties": {
"start_unit_id": {
"type": "integer"
},
"end_unit_id": {
"type": "integer"
}
}
}
}
}
}
}
}
}

View File

@@ -0,0 +1,12 @@
Compact combat grounding is supplied below. It can guide attention and
disambiguation, but it is not evidence. Do not derive an event, subject,
outcome, or source range from either list. The current transcript alone must
directly establish every returned event.
Combat-turn grounding:
{{ input "combat_turns" }}
Named combat-opponent grounding:
{{ input "npc_occurrences" }}

View File

@@ -0,0 +1,25 @@
Extract Dungeons & Dragons enemy events from the supplied combat transcript.
An `engaged` event requires direct establishment that a subject is actively
opposing the party in combat. A `killed`, `fled`, `captured`, or
`incapacitated` event requires explicit establishment of that outcome. An
outcome may share evidence with an engagement, and a later engagement or
outcome for the same subject remains a separate observation. Emit at most one
`engaged` observation for the same subject in this combat scene.
For `killed`, direct death or killing is required. For `fled`, the subject
must explicitly escape, retreat, or leave combat to avoid continued engagement.
For `captured`, the subject must be explicitly taken prisoner or secured
under the party's control. For `incapacitated`, the subject must be explicitly
unable to continue acting without being established as killed or captured.
When the transcript identifies a named NPC, use its normalized registry
spelling. A hostile creature without a registry entry is allowed. For unnamed
individuals or groups, use only the narrowest transcript-grounded label, such
as `Orcs`, `One orc`, or `Remaining orcs`; never invent member names, IDs,
or quantities.
Exclude party members, allies, neutral observers, mentioned-but-absent enemies,
hazards, traps, environmental effects, uncertain allegiance, table talk,
planning, hypotheses, recaps outside this passage, and downstream inference.
Do not infer an engagement or outcome from initiative, turn absence, damage,
defeat, movement, or a scene ending.

View File

@@ -0,0 +1,53 @@
id: dnd.enemy_events
version: "v1"
default_profile: dnd-extraction
inputs:
- name: transcript
required: true
content_type: application/json
- name: players
required: false
content_type: text/plain
- name: party
required: false
content_type: text/plain
- name: glossary
required: false
content_type: text/plain
- name: npc_registry
required: true
content_type: application/json
- name: combat_turns
required: true
content_type: application/json
- name: npc_occurrences
required: true
content_type: application/json
messages:
- role: system
content_file: ./sharedassets/common-dnd-system.md
- role: user
content_file: ./sharedassets/common-dnd-identity.md
- role: user
content_file: ./sharedassets/common-dnd-references.md
cache_control:
type: ephemeral
- role: user
content_file: ./sharedassets/common-dnd-transcript-chunk.md
cache_control:
type: ephemeral
- role: user
content_file: ./sharedassets/common-dnd-extraction-evidence.md
- role: user
content_file: ./sharedassets/common-dnd-npc-registry.md
- role: user
content_file: ./combat-grounding.md
- role: user
content_file: ./instructions.md
cache_control:
type: ephemeral
output:
format: json
validation_mode: json_schema
schema_path: dnd_enemy_events_llm.v1.json
repair_attempts: 0

View File

@@ -0,0 +1,33 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "notarius.dnd.enemy_events.llm",
"type": "object",
"additionalProperties": false,
"required": ["events"],
"properties": {
"events": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": false,
"required": ["name", "kind", "source_refs"],
"properties": {
"name": {"type": "string"},
"kind": {"type": "string"},
"source_refs": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": false,
"required": ["start_unit_id", "end_unit_id"],
"properties": {
"start_unit_id": {"type": "integer"},
"end_unit_id": {"type": "integer"}
}
}
}
}
}
}
}
}

View File

@@ -0,0 +1,30 @@
Extract meaningful Dungeons & Dragons item and currency occurrences: discoveries and changes
in party possession established by the transcript. This is an occurrence history,
not an inventory or ledger: do not calculate balances, resolve item identity
across records, or infer ownership that the transcript does not establish.
For every occurrence, use the supplied canonical item `name`. Record a stated
quantity as an integer and leave it null when the transcript does not state
one. Preserve the stated currency denomination through the selected canonical
registry name.
Use `discovered` when the party learns of or encounters an item without
establishing possession. Use `acquired` when the party or a party member gains
possession. Use `lost` when party possession ends through a gift, sale, payment,
theft, abandonment, or destruction not caused by intended use. Use `consumed`
when intended use depletes an expendable item. Monetary spending, purchases, and
payments are always `lost`, not `consumed`. Classify currency as `consumed` only
when the transcript explicitly describes it being physically destroyed or
expended as a non-payment component. Use `transferred` only when possession
moves between two distinct named party members.
Return both `from` and `to` for every occurrence, using `null` when a holder does not
apply. For `discovered`, set both holders to `null`. For `acquired`, set `from`
to `null` and provide `to`; for `lost` and `consumed`, provide `from` and set
`to` to `null`; and for `transferred`, provide both holders. Use `party` only
for collective or unresolved party possession, never for either side of a
transfer. Do not emit a transfer for a gift, sale, or payment outside the party.
Ordinary non-depleting use is not an occurrence. Do not infer acquisition from a
discovery, or discovery from an acquisition: emit both only when each is
independently established.

View File

@@ -0,0 +1,6 @@
Use the supplied item registry only to ground each occurrence. Every record
must use one registry item's canonical `name`; do not invent, rename, merge,
or infer registry items. The registry is not transcript evidence: cite only the
current transcript chunk in `source_refs`.
{{ input "item_registry" }}

View File

@@ -0,0 +1,45 @@
id: dnd.item_occurrences
version: "v1"
default_profile: dnd-extraction
inputs:
- name: transcript
required: true
content_type: application/json
- name: players
required: false
content_type: text/plain
- name: party
required: false
content_type: text/plain
- name: glossary
required: false
content_type: text/plain
- name: item_registry
required: true
content_type: application/json
messages:
- role: system
content_file: ./sharedassets/common-dnd-system.md
- role: user
content_file: ./sharedassets/common-dnd-identity.md
- role: user
content_file: ./sharedassets/common-dnd-references.md
cache_control:
type: ephemeral
- role: user
content_file: ./sharedassets/common-dnd-transcript-chunk.md
cache_control:
type: ephemeral
- role: user
content_file: ./sharedassets/common-dnd-extraction-evidence.md
- role: user
content_file: ./item-registry.md
- role: user
content_file: ./instructions.md
cache_control:
type: ephemeral
output:
format: json
validation_mode: json_schema
schema_path: dnd_item_occurrences_llm.v1.json
repair_attempts: 0

View File

@@ -0,0 +1,36 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "notarius.dnd.item_occurrences.llm",
"type": "object",
"additionalProperties": false,
"required": ["occurrences"],
"properties": {
"occurrences": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": false,
"required": ["name", "kind", "quantity", "from", "to", "source_refs"],
"properties": {
"name": {"type": "string"},
"kind": {"type": "string"},
"quantity": {"type": ["integer", "null"]},
"from": {"type": ["string", "null"]},
"to": {"type": ["string", "null"]},
"source_refs": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": false,
"required": ["start_unit_id", "end_unit_id"],
"properties": {
"start_unit_id": {"type": "integer"},
"end_unit_id": {"type": "integer"}
}
}
}
}
}
}
}
}

View File

@@ -0,0 +1,12 @@
Extract only items established by the provided Dungeons & Dragons transcript.
Include named unique items, concrete reusable item types, and stable unique
designations. Record each currency denomination separately when it is
established, such as copper pieces, silver pieces, gold pieces, or platinum
pieces. Do not use capitalization as an eligibility test. Keep distinct names
and designations as separate candidates; do not merge aliases or invent
qualifiers.
Do not record vague categories such as "loot", "treasure", or "some gear";
generic weapons; inferred properties; quantities; or inferred uniqueness. Omit
uncertain or unsupported items.

View File

@@ -0,0 +1,40 @@
id: dnd.item_registry
version: "v1"
default_profile: dnd-extraction
inputs:
- name: transcript
required: true
content_type: application/json
- name: players
required: false
content_type: text/plain
- name: party
required: false
content_type: text/plain
- name: glossary
required: false
content_type: text/plain
messages:
- role: system
content_file: ./sharedassets/common-dnd-system.md
- role: user
content_file: ./sharedassets/common-dnd-identity.md
- role: user
content_file: ./sharedassets/common-dnd-references.md
cache_control:
type: ephemeral
- role: user
content_file: ./sharedassets/common-dnd-transcript-chunk.md
cache_control:
type: ephemeral
- role: user
content_file: ./sharedassets/common-dnd-extraction-evidence.md
- role: user
content_file: ./instructions.md
cache_control:
type: ephemeral
output:
format: json
validation_mode: json_schema
schema_path: dnd_item_registry_llm.v1.json
repair_attempts: 0

View File

@@ -0,0 +1,32 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "notarius.dnd.item_registry.llm",
"type": "object",
"additionalProperties": false,
"required": ["items"],
"properties": {
"items": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": false,
"required": ["name", "source_refs"],
"properties": {
"name": {"type": "string"},
"source_refs": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": false,
"required": ["start_unit_id", "end_unit_id"],
"properties": {
"start_unit_id": {"type": "integer"},
"end_unit_id": {"type": "integer"}
}
}
}
}
}
}
}
}

View File

@@ -0,0 +1,9 @@
Determine whether candidates identify the same item type or unique designation
using their contextual labels and cited transcript windows. Do not treat nearby
evidence, similar objects, or a shared owner as sufficient.
Keep currency denominations and materially different item types separate. Keep
uncertain aliases separate. Do not infer an item property or uniqueness.
When selecting a canonical display name, choose one supplied candidate name
that is the clearest established designation.

View File

@@ -0,0 +1,30 @@
id: dnd.item_registry.normalize
version: "v1"
default_profile: dnd-extraction
inputs:
- name: candidates
required: true
content_type: application/json
- name: transcript
required: true
content_type: application/json
messages:
- role: system
content_file: ./sharedassets/common-dnd-system.md
- role: user
content_file: ./sharedassets/protocol.md
- role: user
content_file: ./instructions.md
cache_control:
type: ephemeral
- role: user
content_file: ./sharedassets/candidates.md
- role: user
content_file: ./sharedassets/transcript-windows.md
cache_control:
type: ephemeral
output:
format: json
validation_mode: json_schema
schema_path: semantic_reconciliation_llm.v1.json
repair_attempts: 0

View File

@@ -0,0 +1,34 @@
Extract Dungeons & Dragons location occurrences from the supplied transcript.
Include an occurrence only when the transcript establishes one supplied
location, one occurrence kind, and a coherent passage supporting both.
Use exactly one kind per occurrence:
- visited: party members are physically present, arrive, remain, or depart;
- planned: the party explicitly proposes, intends, or agrees to future travel;
- recalled: the transcript explicitly recounts an earlier party visit; or
- mentioned: the location is explicitly referenced without stronger support,
including non-actionable speculation or a mere hypothetical reference.
A mere hypothetical or speculative reference is not planned unless the
transcript also establishes an actual proposal, intention, or agreement to
travel. When the hypothetical explicitly names a supplied location, it may be
mentioned.
A generic phrase in the current chunk may refer to a supplied named registry
location only when the chunk's context supports that coreference. It must not
create a registry location, and registry content or provenance must never
replace current-chunk evidence.
For every occurrence, return the exact selector from the location registry:
the canonical `name`, plus an empty `registry_refs` array for a unique name or
the complete ordered `registry_refs` array for a repeated name. Registry ranges
and context identify the location only; they are not occurrence evidence.
For overlapping support, visited outranks planned, recalled, and mentioned;
planned outranks recalled and mentioned; recalled outranks mentioned. A passage
may produce multiple records when it independently establishes separate facts,
such as recalling an earlier visit while planning a return. Omit inferred,
unstated, uncertain, or unsupported places and occurrences. Do not infer a
location or occurrence from surrounding events when the transcript does not
state it. Do not summarize location descriptions.

View File

@@ -0,0 +1,11 @@
A contextual location registry is provided below for identity grounding. It may
be empty. Every record supplies a canonical display name. A name that appears
once is selected with that name and an empty `registry_refs` array. A repeated
name is selected only by copying both its name and its complete, ordered
`registry_refs` array exactly as supplied.
Registry content is context, not occurrence evidence. Do not derive an
occurrence or `source_refs` range from the registry. Do not invent a location
or selector that is absent from it.
{{ input "location_registry" }}

View File

@@ -0,0 +1,45 @@
id: dnd.location_occurrences
version: "v1"
default_profile: dnd-extraction
inputs:
- name: transcript
required: true
content_type: application/json
- name: players
required: false
content_type: text/plain
- name: party
required: false
content_type: text/plain
- name: glossary
required: false
content_type: text/plain
- name: location_registry
required: true
content_type: application/json
messages:
- role: system
content_file: ./sharedassets/common-dnd-system.md
- role: user
content_file: ./sharedassets/common-dnd-identity.md
- role: user
content_file: ./sharedassets/common-dnd-references.md
cache_control:
type: ephemeral
- role: user
content_file: ./sharedassets/common-dnd-transcript-chunk.md
cache_control:
type: ephemeral
- role: user
content_file: ./sharedassets/common-dnd-extraction-evidence.md
- role: user
content_file: ./location-registry.md
- role: user
content_file: ./instructions.md
cache_control:
type: ephemeral
output:
format: json
validation_mode: json_schema
schema_path: dnd_location_occurrences_llm.v1.json
repair_attempts: 0

View File

@@ -0,0 +1,45 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "notarius.dnd.location_occurrences.llm",
"type": "object",
"additionalProperties": false,
"required": ["occurrences"],
"properties": {
"occurrences": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": false,
"required": ["name", "registry_refs", "kind", "source_refs"],
"properties": {
"name": {"type": "string"},
"registry_refs": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": false,
"required": ["start_unit_id", "end_unit_id"],
"properties": {
"start_unit_id": {"type": "integer", "minimum": 1},
"end_unit_id": {"type": "integer", "minimum": 1}
}
}
},
"kind": {"enum": ["visited", "planned", "recalled", "mentioned"]},
"source_refs": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": false,
"required": ["start_unit_id", "end_unit_id"],
"properties": {
"start_unit_id": {"type": "integer"},
"end_unit_id": {"type": "integer"}
}
}
}
}
}
}
}
}

View File

@@ -0,0 +1,13 @@
Extract only physical places established by the provided Dungeons & Dragons
transcript that have a stable proper name or unique in-world designation. This
includes named planes, regions, settlements, districts, buildings, rooms,
landmarks, routes, and geographic features.
Do not create a registry location for generic, temporary, relative, or merely
descriptive phrases, including "the room", "the bar", "the hallway",
"outside", and "upstairs". Do not use capitalization as an eligibility test.
Keep aliases and nested places when the transcript identifies them; do not merge
or invent qualifiers for similarly named places.
Exclude people, creatures, objects, organizations, abstract concepts, and
places merely inferred from an event. Omit uncertain or unsupported places.

View File

@@ -0,0 +1,40 @@
id: dnd.location_registry
version: "v1"
default_profile: dnd-extraction
inputs:
- name: transcript
required: true
content_type: application/json
- name: players
required: false
content_type: text/plain
- name: party
required: false
content_type: text/plain
- name: glossary
required: false
content_type: text/plain
messages:
- role: system
content_file: ./sharedassets/common-dnd-system.md
- role: user
content_file: ./sharedassets/common-dnd-identity.md
- role: user
content_file: ./sharedassets/common-dnd-references.md
cache_control:
type: ephemeral
- role: user
content_file: ./sharedassets/common-dnd-transcript-chunk.md
cache_control:
type: ephemeral
- role: user
content_file: ./sharedassets/common-dnd-extraction-evidence.md
- role: user
content_file: ./instructions.md
cache_control:
type: ephemeral
output:
format: json
validation_mode: json_schema
schema_path: dnd_location_registry_llm.v1.json
repair_attempts: 0

View File

@@ -0,0 +1,32 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "notarius.dnd.location_registry.llm",
"type": "object",
"additionalProperties": false,
"required": ["locations"],
"properties": {
"locations": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": false,
"required": ["name", "source_refs"],
"properties": {
"name": {"type": "string"},
"source_refs": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": false,
"required": ["start_unit_id", "end_unit_id"],
"properties": {
"start_unit_id": {"type": "integer"},
"end_unit_id": {"type": "integer"}
}
}
}
}
}
}
}
}

View File

@@ -0,0 +1,8 @@
Determine whether candidates identify the same physical place using their
contextual labels and cited transcript windows. Do not treat matching names,
nearby evidence, nested places, or generic labels as sufficient.
Keep parent and child places separate, as well as similarly named places and
uncertain aliases.
When selecting a canonical display name, prefer the clearest established name.

View File

@@ -0,0 +1,30 @@
id: dnd.location_registry.normalize
version: "v1"
default_profile: dnd-extraction
inputs:
- name: candidates
required: true
content_type: application/json
- name: transcript
required: true
content_type: application/json
messages:
- role: system
content_file: ./sharedassets/common-dnd-system.md
- role: user
content_file: ./sharedassets/protocol.md
- role: user
content_file: ./instructions.md
cache_control:
type: ephemeral
- role: user
content_file: ./sharedassets/candidates.md
- role: user
content_file: ./sharedassets/transcript-windows.md
cache_control:
type: ephemeral
output:
format: json
validation_mode: json_schema
schema_path: semantic_reconciliation_llm.v1.json
repair_attempts: 0

View File

@@ -0,0 +1,26 @@
Extract Dungeons & Dragons NPC occurrences from the supplied
transcript. Include an occurrence only when the transcript establishes one
supplied NPC, one occurrence kind, and a coherent passage supporting both.
Use the supplied canonical NPC `name`; never invent or substitute a similar
name. Cite current-transcript evidence for every occurrence.
Do not summarize, infer relationships, sentiment, factions, motives, aliases,
or persistent state. Do not identify player characters, anonymous groups, or
invented NPCs. Split records when an NPC's occurrence kind changes, when
combat alignment changes, or when an NPC is first mentioned and later becomes
present.
Use exactly one kind per occurrence:
- mentioned: the NPC is referred to but is not established as present or communicating;
- noncombat_presence: the NPC is present and relevant but does not meaningfully participate in dialogue or combat;
- dialogue: the NPC speaks, responds, or is directly engaged in a meaningful non-combat exchange;
- combat_ally: the NPC actively participates in combat on the party's side;
- combat_opponent: the NPC actively participates in combat against the party; or
- other: the transcript clearly establishes a direct NPC occurrence that fits none of the preceding kinds.
When activities overlap, active combat participation outranks dialogue,
presence, and mention; dialogue outranks noncombat presence and mention; and
noncombat presence outranks mention. Other is only for directly evidenced
activity outside those categories. Split an occurrence rather than assigning
both combat alignments.

View File

@@ -0,0 +1,45 @@
id: dnd.npc_occurrences
version: "v1"
default_profile: dnd-extraction
inputs:
- name: transcript
required: true
content_type: application/json
- name: players
required: false
content_type: text/plain
- name: party
required: false
content_type: text/plain
- name: glossary
required: false
content_type: text/plain
- name: npc_registry
required: true
content_type: application/json
messages:
- role: system
content_file: ./sharedassets/common-dnd-system.md
- role: user
content_file: ./sharedassets/common-dnd-identity.md
- role: user
content_file: ./sharedassets/common-dnd-references.md
cache_control:
type: ephemeral
- role: user
content_file: ./sharedassets/common-dnd-transcript-chunk.md
cache_control:
type: ephemeral
- role: user
content_file: ./sharedassets/common-dnd-extraction-evidence.md
- role: user
content_file: ./sharedassets/common-dnd-npc-registry.md
- role: user
content_file: ./instructions.md
cache_control:
type: ephemeral
output:
format: json
validation_mode: json_schema
schema_path: dnd_npc_occurrences_llm.v1.json
repair_attempts: 0

View File

@@ -0,0 +1,41 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "notarius.dnd.npc_occurrences.llm",
"type": "object",
"additionalProperties": false,
"required": ["occurrences"],
"properties": {
"occurrences": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": false,
"required": ["name", "kind", "source_refs"],
"properties": {
"name": {
"type": "string"
},
"kind": {
"type": "string"
},
"source_refs": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": false,
"required": ["start_unit_id", "end_unit_id"],
"properties": {
"start_unit_id": {
"type": "integer"
},
"end_unit_id": {
"type": "integer"
}
}
}
}
}
}
}
}
}

View File

@@ -0,0 +1,19 @@
Extract the individually identifiable Dungeons & Dragons non-player characters
established by the provided transcript.
Include an in-world non-PC only when the transcript factually establishes a
proper name or a stable, individually distinguishing title or alias. A factual
third-party mention establishes that identity even when the NPC is not
physically present, does not speak, and takes no direct action in this chunk.
Record only the NPC identity and the transcript evidence that establishes it;
do not infer or classify a separate occurrence.
Exclude human players, transcript speakers, and the GM as out-of-world people;
player characters identified by the player or party references; names used only
in hypothetical, speculative, or imagined examples; corrected transcription
mistakes; anonymous or generic roles; indistinguishable crowds or groups;
invented descriptive labels; and temporary summoned creatures or spell effects
without a persistent individual identity.
Preserve observed display spelling. Do not invent a label for an anonymous
creature, crowd, or generic role.

View File

@@ -0,0 +1,40 @@
id: dnd.npc_registry
version: "v1"
default_profile: dnd-extraction
inputs:
- name: transcript
required: true
content_type: application/json
- name: players
required: false
content_type: text/plain
- name: party
required: false
content_type: text/plain
- name: glossary
required: false
content_type: text/plain
messages:
- role: system
content_file: ./sharedassets/common-dnd-system.md
- role: user
content_file: ./sharedassets/common-dnd-identity.md
- role: user
content_file: ./sharedassets/common-dnd-references.md
cache_control:
type: ephemeral
- role: user
content_file: ./sharedassets/common-dnd-transcript-chunk.md
cache_control:
type: ephemeral
- role: user
content_file: ./sharedassets/common-dnd-extraction-evidence.md
- role: user
content_file: ./instructions.md
cache_control:
type: ephemeral
output:
format: json
validation_mode: json_schema
schema_path: dnd_npc_registry_llm.v1.json
repair_attempts: 0

View File

@@ -0,0 +1,41 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "notarius.dnd.npc_registry.llm",
"type": "object",
"additionalProperties": false,
"required": ["npcs"],
"properties": {
"npcs": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": false,
"required": [
"name",
"source_refs"
],
"properties": {
"name": {
"type": "string"
},
"source_refs": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": false,
"required": ["start_unit_id", "end_unit_id"],
"properties": {
"start_unit_id": {
"type": "integer"
},
"end_unit_id": {
"type": "integer"
}
}
}
}
}
}
}
}
}

View File

@@ -0,0 +1,11 @@
Determine whether candidates refer to the same individual using their
contextual labels and cited transcript windows. Preserve distinct individuals
even when their names are similar or their contextual descriptions are
identical.
When selecting a canonical display name, prefer a complete, stable proper name
over an abbreviation. Prefer an unadorned proper name over that name plus a
contextual class, role, title, or relationship descriptor unless the transcript
establishes the descriptor as part of the person's name. A longer display name
is not inherently more canonical; for example, do not prefer `Captain Aria`
over `Aria` solely because it includes the contextual title `Captain`.

View File

@@ -0,0 +1,30 @@
id: dnd.npc_registry.normalize
version: "v1"
default_profile: dnd-extraction
inputs:
- name: candidates
required: true
content_type: application/json
- name: transcript
required: true
content_type: application/json
messages:
- role: system
content_file: ./sharedassets/common-dnd-system.md
- role: user
content_file: ./sharedassets/protocol.md
- role: user
content_file: ./instructions.md
cache_control:
type: ephemeral
- role: user
content_file: ./sharedassets/candidates.md
- role: user
content_file: ./sharedassets/transcript-windows.md
cache_control:
type: ephemeral
output:
format: json
validation_mode: json_schema
schema_path: semantic_reconciliation_llm.v1.json
repair_attempts: 0

View File

@@ -0,0 +1,5 @@
id: dnd-extraction
backend: openrouter
model: openai/gpt-5.6-luna
timeout_seconds: 240
service_tier: flex

View File

@@ -0,0 +1,45 @@
Describe exactly one accepted Dungeons & Dragons scene from the supplied
transcript chunk. The complete chunk is the evidence boundary: do not split it
into multiple scenes or use facts that are not supported by it.
Return one kind, one concise title, and one concise summary. Choose exactly one
kind:
- combat: active combat materially organizes the scene, including
initiative-like exchanges or sustained hostile action. Planning a fight or
discussing a completed fight is not combat by itself.
- narrative: current-session in-world play that is not principally active
combat, a prior-session recap, or sustained out-of-character session
discussion. This includes exploration, travel, dialogue, investigation,
in-character planning, and aftermath.
- recap: the scene's organizing purpose is to recount events from a previous
session for the table. An in-world character recounting history during
current play remains narrative.
- meta: the scene's organizing purpose is sustained out-of-character
discussion about the game or session rather than advancing current in-world
play.
Narrative is the default for actual current-session gameplay that does not meet
another definition. When the accepted chunk is mixed:
1. use combat when active combat is a substantive central activity, even with
brief setup, rules clarification, or immediate aftermath;
2. otherwise use recap when recounting a previous session is the chunk's
primary table purpose;
3. otherwise use meta when sustained out-of-character session discussion is
primary and in-world progression is no more than incidental; and
4. use narrative for all remaining current-session in-world play.
Brief table talk, dice resolution, rules clarification, jokes, or
administrative comments do not make a gameplay scene meta. A short recollection
used to orient current action does not make a scene recap.
The title must be a short, distinguishing phrase rather than a sentence,
chapter number, or generic label such as "Scene." It may use names and places
established by the transcript or disambiguated by campaign references, but it
must not invent a proper noun.
The summary must briefly state the main activity and material transition or
outcome established within the accepted chunk. Do not add analysis, inferred
motives, hidden state, future consequences, relationship claims, or facts from
outside the chunk.

View File

@@ -1,13 +1,10 @@
id: dnd.spells id: dnd.scene_descriptions
version: "v1" version: "v1"
default_profile: gemini-2-flash default_profile: dnd-extraction
inputs: inputs:
- name: transcript - name: transcript
required: true required: true
content_type: application/json content_type: application/json
- name: spell_catalog
required: true
content_type: application/json
- name: players - name: players
required: false required: false
content_type: text/plain content_type: text/plain
@@ -21,21 +18,21 @@ messages:
- role: system - role: system
content_file: ./sharedassets/common-dnd-system.md content_file: ./sharedassets/common-dnd-system.md
- role: user - role: user
content_file: ./sharedassets/common-dnd-transcript.md content_file: ./sharedassets/common-dnd-identity.md
cache_control:
type: ephemeral
- role: user - role: user
content_file: ./sharedassets/common-dnd-references.md content_file: ./sharedassets/common-dnd-references.md
cache_control: cache_control:
type: ephemeral type: ephemeral
- role: user - role: user
content_file: ./catalog.md content_file: ./sharedassets/common-dnd-transcript-chunk.md
- role: user cache_control:
content_file: ./task.md type: ephemeral
- role: user - role: user
content_file: ./instructions.md content_file: ./instructions.md
cache_control:
type: ephemeral
output: output:
format: json format: json
validation_mode: json_schema validation_mode: json_schema
schema_path: dnd_spells_llm.v1.json schema_path: dnd_scene_descriptions_llm.v1.json
repair_attempts: 0 repair_attempts: 0

View File

@@ -0,0 +1,18 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "notarius.dnd.scene_descriptions.llm",
"type": "object",
"additionalProperties": false,
"required": ["kind", "title", "summary"],
"properties": {
"kind": {
"type": "string"
},
"title": {
"type": "string"
},
"summary": {
"type": "string"
}
}
}

View File

@@ -0,0 +1,19 @@
Divide the complete provided transcript into coherent Dungeons & Dragons scenes
for the `dnd/scenes` chunk module.
A scene is a coherent unit of play. Start a new scene when the transcript
establishes a meaningful change in location, objective, threat, activity,
encounter, or mode of play. Good reasons include a material move, beginning or
ending combat, a substantially different encounter phase, a shift between
combat, exploration, social interaction, planning, travel, rest, or downtime,
a change in the central NPC, faction, threat, or objective, or a sustained
table-level interruption that materially changes the activity.
Do not split a scene merely because a speaker or combat round changes, a
routine turn occurs, or the table briefly digresses. Prefer fewer coherent
scenes over speculative or fine-grained boundaries.
Cover the complete transcript from its first source unit to its last. Return
scenes in source-unit order with no gaps or overlaps. Use only positive integer
source-unit IDs from the transcript, and give every scene one inclusive
`start_unit_id` and one inclusive `end_unit_id`.

View File

@@ -1,6 +1,6 @@
id: dnd.scenes id: dnd.scenes
version: "v1" version: "v1"
default_profile: gemini-2-flash default_profile: dnd-extraction
inputs: inputs:
- name: transcript - name: transcript
required: true required: true
@@ -17,20 +17,18 @@ inputs:
messages: messages:
- role: system - role: system
content_file: ./sharedassets/common-dnd-system.md content_file: ./sharedassets/common-dnd-system.md
- role: user
content_file: ./sharedassets/common-dnd-transcript.md
cache_control:
type: ephemeral
- role: user - role: user
content_file: ./sharedassets/common-dnd-references.md content_file: ./sharedassets/common-dnd-references.md
cache_control: cache_control:
type: ephemeral type: ephemeral
- role: user
content_file: ./task.md
- role: user - role: user
content_file: ./instructions.md content_file: ./instructions.md
- role: user
content_file: ./sharedassets/common-dnd-transcript-full.md
cache_control:
type: ephemeral
output: output:
format: json format: json
validation_mode: json_schema validation_mode: json_schema
schema_path: dnd_scenes.v1.json schema_path: dnd_scenes_llm.v1.json
repair_attempts: 0 repair_attempts: 0

View File

@@ -0,0 +1,28 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "notarius.dnd.scenes.llm",
"type": "object",
"additionalProperties": false,
"required": ["scenes"],
"properties": {
"scenes": {
"type": "array",
"minItems": 1,
"items": {
"type": "object",
"additionalProperties": false,
"required": ["start_unit_id", "end_unit_id"],
"properties": {
"start_unit_id": {
"type": "integer",
"minimum": 1
},
"end_unit_id": {
"type": "integer",
"minimum": 1
}
}
}
}
}
}

View File

@@ -0,0 +1,6 @@
Transcript units are the only evidence for extracted events and factual claims.
Every reported factual claim must be supported by cited transcript units. Use
integer `start_unit_id` and `end_unit_id` values from the transcript.
When supporting evidence is non-contiguous, use multiple narrow ranges rather
than a broad range that bridges unrelated conversation.

View File

@@ -0,0 +1,7 @@
Use the most specific supported in-world character or creature identity for
each actor or participant. Do not identify a human player, transcript speaker,
or the GM as an out-of-world person when an in-world identity is supported.
Player, party, glossary, campaign, and NPC registry references may disambiguate
an identity, but a reference alone cannot establish that the identity
participated in the transcript.

View File

@@ -0,0 +1,10 @@
A normalized Dungeons & Dragons NPC registry is provided below as grounding
material. It may be empty. Use it only to prefer exact canonical participant
names when the transcript identifies a participant.
Registry content is context, not event evidence. Do not extract events,
participants, effects, or source references from the registry. Registry source
references describe registry provenance and may belong to another session; they
are never evidence for the current transcript.
{{ input "npc_registry" }}

View File

@@ -0,0 +1,5 @@
You process Dungeons & Dragons gameplay transcripts.
As input, you will receive one or more portions of a transcript. The transcript may contain transcription errors, repeated lines, incomplete sentences, and misheard proper nouns.
Return exactly one JSON object that conforms to the configured response schema, with no explanatory prose.

View File

@@ -0,0 +1,3 @@
One extraction chunk from a Dungeons & Dragons gameplay transcript is provided below. Report and infer only what is within this chunk. Its unit IDs retain their source-wide meaning.
{{ input "transcript" }}

View File

@@ -0,0 +1,3 @@
The complete ordered transcript of this Dungeons & Dragons gameplay session is provided below.
{{ input "transcript" }}

View File

@@ -0,0 +1,14 @@
Extract Dungeons & Dragons spell-cast artifacts from the provided transcript.
Include an actual casting event or an unambiguous declared casting attempt.
Exclude spell mentions, hypothetical plans, rules discussion, and catalog
matches that do not establish a casting event in the transcript.
For every extracted cast, the transcript evidence must collectively support the
in-world caster, the spell, and the fact that the cast or declared attempt
occurred.
Attribute every cast to its in-world caster. Map first-person player speech to
the associated player character, and attribute a spell narrated by the GM to
the in-world creature that casts it. If the caster cannot be resolved, use only
the most specific in-world identity supported by the transcript; do not invent
a name.

View File

@@ -0,0 +1,50 @@
id: dnd.spells
version: "v1"
default_profile: dnd-extraction
inputs:
- name: transcript
required: true
content_type: application/json
- name: spell_catalog
required: true
content_type: application/json
- name: players
required: false
content_type: text/plain
- name: party
required: false
content_type: text/plain
- name: glossary
required: false
content_type: text/plain
- name: npc_registry
required: false
content_type: application/json
messages:
- role: system
content_file: ./sharedassets/common-dnd-system.md
- role: user
content_file: ./sharedassets/common-dnd-identity.md
- role: user
content_file: ./sharedassets/common-dnd-references.md
cache_control:
type: ephemeral
- role: user
content_file: ./sharedassets/common-dnd-transcript-chunk.md
cache_control:
type: ephemeral
- role: user
content_file: ./sharedassets/common-dnd-extraction-evidence.md
- role: user
content_file: ./sharedassets/common-dnd-npc-registry.md
- role: user
content_file: ./spell-catalog.md
- role: user
content_file: ./instructions.md
cache_control:
type: ephemeral
output:
format: json
validation_mode: json_schema
schema_path: dnd_spells_llm.v1.json
repair_attempts: 0

View File

@@ -0,0 +1,6 @@
The spell catalog for this extraction is provided below as JSON. Each entry
lists a `canonical_name` and its recognized `aliases`. If the transcript uses
an alias, select that entry's `canonical_name`. Return spell names using the
canonical spelling exactly; never return an alias as a spell name.
{{ input "spell_catalog" }}

View File

@@ -13,42 +13,30 @@
"required": [ "required": [
"caster", "caster",
"spell", "spell",
"effect",
"narrative_description",
"source_refs" "source_refs"
], ],
"properties": { "properties": {
"caster": { "caster": {
"type": "string", "type": "string",
"minLength": 1 "description": "Canonical in-world character or creature that casts the spell, never the human player, transcript speaker, or GM when the in-world caster can be identified."
}, },
"spell": { "spell": {
"type": "string", "type": "string",
"minLength": 1 "description": "Canonical spell name from the provided spell-name catalog."
},
"effect": {
"type": "string",
"minLength": 1
},
"narrative_description": {
"type": "string",
"minLength": 1
}, },
"source_refs": { "source_refs": {
"type": "array", "type": "array",
"minItems": 1, "description": "Transcript ranges offered as evidence for the caster, spell name, and casting event in this spell-cast object.",
"items": { "items": {
"type": "object", "type": "object",
"additionalProperties": false, "additionalProperties": false,
"required": ["start_unit_id", "end_unit_id"], "required": ["start_unit_id", "end_unit_id"],
"properties": { "properties": {
"start_unit_id": { "start_unit_id": {
"type": "integer", "type": "integer"
"minimum": 1
}, },
"end_unit_id": { "end_unit_id": {
"type": "integer", "type": "integer"
"minimum": 1
} }
} }
} }

View File

@@ -0,0 +1,3 @@
Candidate material:
{{ input "candidates" }}

View File

@@ -0,0 +1,5 @@
Identify only high-confidence duplicate entities among the supplied candidates.
Preserve distinct entities even when their names are similar. Treat contextual descriptions and transcript evidence as supporting material, not as permission to merge ambiguous records.
When several records are duplicates, choose as canonical the candidate with the clearest stable identity. Prefer a complete proper name over an abbreviation, and prefer an unadorned proper name over one with incidental descriptors unless the evidence establishes those descriptors as part of the name. A longer name is not inherently more canonical.

View File

@@ -0,0 +1,27 @@
id: generic.semantic_reconciliation
version: "v1"
inputs:
- name: candidates
required: true
content_type: application/json
- name: transcript
required: true
content_type: application/json
messages:
- role: system
content_file: ./system.md
- role: user
content_file: ./protocol.md
- role: user
content_file: ./instructions.md
cache_control:
type: ephemeral
- role: user
content_file: ./candidates.md
- role: user
content_file: ./transcript-windows.md
output:
format: json
validation_mode: json_schema
schema_path: semantic_reconciliation_llm.v1.json
repair_attempts: 0

View File

@@ -0,0 +1,7 @@
Use only the positive integer `candidate_id` values supplied in the candidate material.
Return a duplicate group only when the evidence supports that every selected candidate describes the same underlying entity. Each group must contain at least two distinct candidate IDs, and its `canonical_candidate_id` must be one of those IDs. A candidate may appear in at most one group.
Omit uncertain matches and candidates that should remain distinct. Do not invent candidates or infer an ID from list position. An empty `duplicate_groups` array is valid.
The response must conform exactly to the selected JSON schema. Return IDs only: do not copy candidate names, evidence, transcript text, source identifiers, or source ranges into the response.

View File

@@ -0,0 +1,2 @@
You reconcile structured records that may describe the same underlying entity.
Follow the supplied protocol and return only the requested structured result.

View File

@@ -0,0 +1,3 @@
Transcript evidence windows:
{{ input "transcript" }}

View File

@@ -0,0 +1,32 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "notarius.generic.semantic_reconciliation.llm",
"title": "notarius_semantic_reconciliation_llm_v1",
"type": "object",
"additionalProperties": false,
"required": ["duplicate_groups"],
"properties": {
"duplicate_groups": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": false,
"required": ["candidate_ids", "canonical_candidate_id"],
"properties": {
"candidate_ids": {
"type": "array",
"minItems": 2,
"items": {
"type": "integer",
"minimum": 1
}
},
"canonical_candidate_id": {
"type": "integer",
"minimum": 1
}
}
}
}
}
}

15
assets/package.go Normal file
View File

@@ -0,0 +1,15 @@
// Package assets exposes embedded LLM-facing content.
package assets
import (
"embed"
"io/fs"
)
//go:embed dnd generic
var embedded embed.FS
// FS returns the embedded read-only asset filesystem.
func FS() fs.FS {
return embedded
}

View File

@@ -1,6 +1,6 @@
# ADR-0004: Package modules by domain, not by stage # ADR-0004: Package modules by domain, not by stage
**Status:** Accepted **Status:** Accepted — its asset-co-location rule is superseded by [ADR-0011](0011-centralize-llm-assets.md); its domain-first module packaging decision remains accepted.
**Date:** 2026-07-13 **Date:** 2026-07-13
## Context ## Context

View File

@@ -0,0 +1,50 @@
# ADR-0008: Bounded ordered pipeline steps and explicit artifact references
**Status:** Accepted
**Date:** 2026-07-21
## Context
Notarius currently models one pipeline-wide input, chunking plan, artifact
lanes, and output boundary. Some workflows need a deterministic handoff from
one set of normalized artifacts to a later set of artifacts, such as using
extracted NPC records while grounding later combat events. The workflow needs
an explicit topology without turning the pipeline into a general-purpose
workflow engine.
## Decision
Add an ordered collection of pipeline steps. Each step owns one or more
artifact lanes, and lanes within a step retain the existing independent
execution model. The pipeline continues to have one input, chunk plan, output,
and failure boundary. Steps are barriers: a later step may consume only
normalized artifacts from an earlier step.
Generated references use an explicit step-and-lane selector. Reference slots
declare the generated artifact kinds and media types they accept. The resolver
validates the topology, ordering, lane identity, artifact kind, schema, and
codec compatibility before execution. External references remain supported as
path sources, and the legacy top-level artifact map is interpreted as an
implicit `default` step.
Pipeline-level references may not select generated artifacts. General DAGs,
branches, loops, conditional execution, joins, and inferred dependencies are
not part of this model.
## Alternatives considered
- A general DAG would provide more flexibility but would also require a new
scheduler, lifecycle model, failure semantics, and provenance model.
- Separate pipeline runs connected through filesystem paths would lose the
static topology and typed compatibility checks.
- Inferring dependencies from module or lane names would make ordering and
configuration errors difficult to detect reliably.
## Consequences
The resolved pipeline has a deterministic, inspectable topology and can
include it in its identity digest. Configuration validation can reject invalid
generated bindings before any work begins. Existing single-step profiles keep
their behavior through the implicit `default` step. Execution handoff and
multi-step scheduling require follow-up work in the runner and checkpoint
layers.

View File

@@ -0,0 +1,98 @@
# ADR-0009: Prefer minimal evidence-grounded extraction artifacts
**Status:** Accepted
**Date:** 2026-07-22
## Context
Notarius is intended to extract structured facts from source material. Several
early D&D artifacts grew to include descriptive prose, inferred relationships,
immediate outcomes, summaries, and other enrichment alongside the facts that
identify an event or entity. Those fields make one model call responsible for
both extraction and synthesis.
In practice, the richer contracts have produced overlapping or weakly grounded
fields and have made structurally valid, semantically coherent output harder for
cost-effective smaller models. They also increase prompt size, validation and
normalization policy, durable schema surface, downstream coupling, and the
number of claims whose provenance must be evaluated.
The application needs a consistent rule for deciding what belongs in an
extractor before redesigning the current D&D spell, NPC, and combat-turn
contracts or adding new artifact families.
## Decision
An extraction module answers one narrowly stated question and returns the
smallest durable structured artifact that usefully answers it.
Every model-produced field in an extraction artifact must:
- be necessary to answer the extractor's stated question or serve a known
downstream consumer;
- represent a fact or bounded classification that can be supported directly by
cited source ranges;
- remain independently meaningful without model-generated explanatory prose;
and
- justify the additional prompt, schema, validation, normalization, and
compatibility surface it creates.
Source references are required provenance for extracted records. Auxiliary
references may disambiguate identities or canonical names, but they do not
establish source facts and are not copied into evidence.
Extraction artifacts do not include narrative summaries, general analysis,
speculative enrichment, inferred biography or relationships, or redundant
free-text descriptions by default. When such output has a demonstrated use, it
belongs in an explicitly named extraction, classification, enrichment, or
analysis module with its own contract and evidence policy.
Occurrence-level facts are not forced into entity-level attributes. A fact
that can change between encounters, such as an NPC's role in a scene, belongs
on an occurrence artifact rather than as one scalar property of a normalized
NPC registry entry.
Deterministic mapping and normalization may assign application-owned
identifiers, canonicalize known catalog values, order and deduplicate evidence,
and collapse records under an explicit identity rule. They must not manufacture
removed descriptive fields or synthesize missing claims to satisfy an older
contract.
This is a default design rule, not a prohibition on rich artifacts. A richer
field is appropriate when its consumer, evidence semantics, and ownership are
explicit.
## Alternatives considered
- Keep rich schemas and improve prompts or use larger models. This retains
potentially convenient prose but does not resolve overlapping field
responsibilities, weak provenance, higher cost, or unnecessary downstream
coupling.
- Make enrichment fields optional. This reduces rejection pressure but leaves
ambiguous artifact semantics and inconsistent records, and many strict
structured-output providers still require nullable placeholders.
- Keep minimal private LLM schemas while preserving rich durable artifacts.
Deterministic code would have to invent, default, or separately derive the
missing fields, hiding synthesis behind the extraction boundary.
- Use one broad session-analysis module. This reduces the number of lanes but
couples unrelated facts, schemas, retries, evaluation, and downstream
consumers into one model call.
## Consequences
Extraction prompts and response schemas become smaller, more focused, and more
suitable for lower-cost models. Artifacts carry fewer unsupported claims, and
their evidence and validation policies become easier to explain and evaluate.
Independent extractors can evolve, retry, and be consumed without requiring
unrelated enrichment.
Some descriptive convenience fields will disappear from primary artifacts.
Consumers that genuinely need them may require a separate module and explicit
pipeline step. Entity registries may no longer resolve aliases or relationships
unless a dedicated, evidence-grounded capability supplies them.
Removing durable fields is a schema compatibility change. Each affected
artifact requires an explicit version and reference policy; private prompt
changes alone are insufficient. Current-behavior integration and internal
documentation must change with implementation, while the roadmap owns the
proposed contract until then.

View File

@@ -0,0 +1,49 @@
# ADR-0010: Use workload-oriented LLM profile defaults
**Status:** Accepted
**Date:** 2026-08-03
## Context
LLM-backed D&D operations share an execution-policy choice, but repeating a
provider or model-named profile on every module binding ties pipeline structure
to a deployment decision. Different environments may require different model,
backend, timeout, or reasoning settings while retaining the same workload.
Notarius also needs a usable default for maintained D&D prompts without making
an operator profile mandatory. That default must remain owned by the D&D
family, while generic LLM infrastructure stays unaware of domain-specific
policy.
## Decision
Pipelines may name one workload-oriented default profile, inherited only by
selected LLM-backed bindings and validators. Binding-level profile IDs remain
intentional exceptions, and the run-wide CLI profile override has highest
precedence.
The D&D family owns an embedded fallback profile named `dnd-extraction`.
Operators may provide a complete profile with the same ID through a PromptKit
filesystem source. PromptKit selects the higher-precedence matching definition;
Notarius does not merge profile documents. Production, development, and local
deployments can therefore use different execution policy behind one unchanged
pipeline ID.
## Alternatives considered
- Repeat a model-named profile on every binding. This makes routine deployment
policy changes noisy and obscures the shared workload intent.
- Require every deployment to install a profile file. This adds configuration
friction and leaves maintained D&D prompts without an application-owned
fallback.
- Put D&D profile policy in generic LLM infrastructure. This breaks domain
ownership and makes generic code depend on one workload.
## Consequences
Pipeline configuration expresses workload intent rather than a specific
provider or model. Operators can replace the complete execution policy without
editing bindings, while binding-level and run-wide exceptions remain available.
Profile changes affect resolved pipeline and checkpoint identity, so they may
intentionally cause work to be recomputed. The D&D fallback becomes a
maintained application execution-policy asset.

View File

@@ -0,0 +1,69 @@
# ADR-0011: Centralize LLM-facing assets in a content-only package
**Status:** Accepted
**Date:** 2026-08-05
## Context
LLM prompts, private response schemas, generic schemas, and fallback profiles
are authored and reviewed as content, but package-local embedding scattered that
content across implementation trees. Finding all of the assets that contribute
to a prompt family required navigating code ownership boundaries rather than a
single discoverable content boundary.
The repository must retain module ownership of prompt semantics, schema
identities, registration, and prompt-cache behavior. Durable artifact schemas
and non-LLM domain data have different compatibility and ownership rules, so
they must not move merely because they are embedded files.
## Decision
LLM-facing content is embedded by the root `assets` package. It is a data-only
dependency leaf: its single `FS() fs.FS` API returns the read-only embedded
filesystem, and the package contains no business logic or internal or PromptKit
dependencies. The accepted import path is
`gitea.maximumdirect.net/eric/notarius/assets`; it makes repository-owned
content available to its consumers, not a public extension contract.
Consumers scope that filesystem to the subtree they own before reading or
registering content. Modules continue to own their manifests, prompt ordering,
private response-schema identity, and registration. Centralizing physical files
does not centralize domain semantics or transfer those responsibilities to the
root package.
The root package contains prompt content, private LLM response schemas, generic
LLM schemas, shared fragments, and fallback profiles. Durable artifact schemas
and non-LLM domain data remain with their current owners. A module fingerprint
is derived from its manifest-selected module and shared files, rather than from
an entire asset tree. The relocation is accepted to cause a one-time checkpoint
invalidation.
This decision supersedes only the physical asset-co-location portion of
ADR-0004's decision that places domain-specific prompt fragments and schemas
within the domain tree. ADR-0004's domain-first packaging and registrar
ownership decisions remain accepted.
## Alternatives Considered
- Keep package-local assets. This preserves physical co-location with code but
makes prompt-author discovery and cross-family review unnecessarily costly.
- Use `internal/llmassets`. This would hide content from legitimate owners
outside the `internal` subtree and would make the root asset boundary depend
on implementation-layer placement.
- Build a behavioral central registry. This would mix content discovery with
prompt selection and registration behavior, moving module semantics into a
shared registry.
- Use runtime filesystem overlays. This would add runtime configuration and
failure modes where compile-time embedded content is sufficient.
## Consequences
Prompt authors can find in-scope LLM content in one top-level tree while module
packages continue to define its meaning and registration. Consumers have an
explicit, narrow dependency on only the content they need. The root package is
intentionally importable but must remain a stable, content-only leaf rather
than becoming a general extension API.
The initial relocation invalidates existing checkpoints once. Later checkpoint
identity changes remain limited to the manifest-selected prompt and shared
content, so unrelated files do not trigger recomputation.

View File

@@ -0,0 +1,64 @@
# ADR-0012: Resolve opaque entity identifiers deterministically
**Status:** Accepted
**Date:** 2026-08-08
## Context
Entity IDs in durable Notarius artifacts are application-owned, deterministic
identifiers. They are useful to artifact consumers, but their hash-based form
does not help a model distinguish entities and would make the model reproduce
an opaque implementation detail. A plain name is likewise insufficient where
multiple supplied records share that name.
The LLM boundary must preserve the typed artifact and durable-schema ownership
of [ADR-0003](0003-typed-interfaces-with-two-zone-data-model.md) and the distinction
between disambiguating references and source evidence in
[ADR-0009](0009-minimal-evidence-grounded-extraction-artifacts.md).
## Decision
Callers present a model with semantic selections: a canonical name when it is
unique in the request, or a contextual descriptor containing the name and
source coordinates when that context is needed to distinguish supplied
records. The model returns only those supplied selections. The caller resolves
each accepted selection against the request-local supplied records and attaches
the opaque application ID deterministically.
Source coordinates are permitted in a selection solely as identity context.
They neither establish an occurrence fact nor replace that occurrence's
current-transcript evidence. A selector must resolve exactly; unknown,
ambiguous, partial, reordered, or otherwise unsafe selections are not mapped.
Where an operation requires a complete grounded artifact, that failure rejects
the complete artifact rather than accepting a partially mapped result.
An explicitly scoped request-local short label is permitted only when a
contextual descriptor would be impractical and the caller can deterministically
map the label within that one request. Such a label is not a durable ID, must
not escape the request boundary, and requires a concrete justification in its
own module contract.
## Alternatives considered
- Ask the model to return durable IDs. This exposes opaque implementation
state, does not improve semantic disambiguation, and makes model output
depend on hash formatting.
- Select by name alone. This cannot safely distinguish same-name records.
- Make request-local labels durable identifiers. This would turn prompt
presentation into a public identity contract and create avoidable migration
pressure.
- Let the model invent identifiers or resolve ambiguity. This makes identity
assignment non-deterministic and weakens validation.
## Consequences
Durable integration contracts retain their exact ID/name pairs while models
operate on readable contextual selections. Calling modules must own selector
construction, exact resolution, ambiguity handling, and conversion into their
durable artifact type; PromptKit and its adapter remain transport-only.
Some ambiguous or invalid proposals are deliberately omitted, retried, or
rejected according to the caller's existing failure policy. Internal candidate
keys may support deterministic request-local mapping, but they are not
model-visible selectors or durable data. This adds local validation work while
keeping identity assignment auditable and stable.

View File

@@ -0,0 +1,91 @@
# ADR-0013: Use request-local candidate handles for semantic reconciliation
**Status:** Accepted
**Date:** 2026-08-09
## Context
Several typed normalize stage modules need semantic reconciliation after
deterministic preprocessing: a model can judge whether source-backed candidates
refer to the same underlying entity, while application code remains responsible
for constructing the normalized artifact. Requiring the model to reproduce a
candidate's full contextual selector makes the response larger and introduces
avoidable formatting, ordering, and transcription failure modes.
Reconciliation must preserve the exact typed artifact boundary established by
[ADR-0003](0003-typed-interfaces-with-two-zone-data-model.md), the domain-neutral
framework and concrete-domain dependency direction established by
[ADR-0004](0004-package-modules-by-domain.md), and the distinction in
[ADR-0009](0009-minimal-evidence-grounded-extraction-artifacts.md) between source
evidence and auxiliary identity context. It also needs a concrete, narrowly
scoped application of the request-local-label exception allowed by
[ADR-0012](0012-resolve-opaque-entity-identifiers-deterministically.md).
## Decision
Semantic reconciliation will be a domain-neutral framework mechanism used by
typed normalize stage modules. A consuming artifact family will retain
ownership of its typed records, identity rules, consolidation policy, durable
IDs, and domain warnings; the framework mechanism will not infer those rules
from arbitrary data.
For each reconciliation request, deterministic code will assign every eligible
model-visible candidate a contiguous, one-based integer handle. The model may
receive the candidate's contextual label, source references, and bounded source
context needed to judge identity, but its structured response will identify
candidates only by those supplied handles. A handle is local to one request,
does not represent entity identity, and must never enter a durable artifact or
be used to derive a durable ID.
The model will propose duplicate groups and select one supplied member of each
group as canonical. Deterministic code will resolve the handles through the
retained request mapping, validate the complete proposal, discard unsafe
groups, and apply only validated groups through typed domain-owned policy. The
model will not synthesize replacement records or directly mutate an artifact.
Every reconciliation prompt will combine a mandatory framework-owned protocol
and safety policy with an explicitly selected semantic policy. The semantic
policy may be the conservative generic policy or a domain-owned policy, but it
cannot replace the shared response protocol or deterministic safety boundary.
## Alternatives considered
- Return durable application IDs. Opaque IDs do not help semantic judgment,
expose application identity mechanics, and make model output reproduce data
that deterministic code already owns.
- Return names alone or copied contextual selectors. Names can be ambiguous,
while reproducing labels and source ranges adds response complexity and
creates mismatches without adding semantic information. Request-local
handles preserve exact selection without either failure mode.
- Ask the model to return synthesized canonical replacement records. This
would transfer typed artifact construction, provenance consolidation, and
durable identity policy to a probabilistic boundary.
- Reconcile reflection-discovered fields or arbitrary JSON. This would weaken
the typed artifact contract and move domain semantics into generic code.
- Hide reconciliation inside extraction or another stage. This would obscure
stage ownership and create cross-stage behavior outside the fixed pipeline;
reconciliation remains explicit normalize-stage behavior.
- Let each domain replace the complete prompt protocol. This would duplicate
safety mechanics and allow domain policy to bypass the common response and
validation contract.
## Consequences
Model responses become smaller and easier to validate, while deterministic
application code retains authority over identity, provenance, ordering, and
typed artifact construction. The framework requires a request-local mapping,
bounded context preparation, a private integer response contract, proposal
assessment, and shared prompt assets. Each consuming artifact family still
requires a typed adapter for its irreducibly domain-specific rules.
Request-local handles are deliberately unsuitable for persistence, logging as
entity identity, checkpoint contracts, or cross-request correlation. Changes
to shared protocol and policy assets must participate in the normal prompt,
schema, and checkpoint fingerprint mechanisms.
Acceptance of this decision does not imply that the shared mechanism or its
consumer migrations are implemented. The
[feature roadmap](../roadmap/semantic-reconciliation.md) owns target behavior
and status, and the
[implementation plan](../roadmap/implementation.md) owns delivery sequence
until the work is complete.

View File

@@ -1,229 +1,175 @@
# CLI Reference # CLI Reference
This is the canonical reference for the implemented Notarius command-line This is the canonical reference for the implemented Notarius command-line
interface. interface. For the shortest successful run, see the [README](../README.md).
Configuration fields, discovery rules, and selectable module keys are defined
in [Configuration](config.md); runtime state and recovery procedures are
defined in [Operations](operations.md).
For the minimal end-to-end invocation, see the [README](../README.md). ## Command Summary
## Commands ~~~
```text
notarius help notarius help
notarius run <pipeline-id> --input path/to/source.json [--config path/to/config.yml] [--only lane-a,lane-b] [--chunk_cache auto|bypass|refresh] [--output-dir path] [--resume] [--debug [--debug-dir path]] [--llm-profile id] [--session-id id] [--reference selector=path] [--without-reference selector] notarius run <pipeline-id> --input path/to/source.json [--json] [flags]
notarius config validate [--config path/to/config.yml] [--pipeline pipeline-id] [--only lane-a,lane-b] notarius config validate [--config path/to/config.yml] [--pipeline pipeline-id] [--only lane-a,lane-b]
notarius pipelines list [--config path/to/config.yml] [--json] notarius pipelines list [--config path/to/config.yml] [--json]
``` ~~~
Running `notarius` with no arguments, `notarius help`, `notarius --help`, or Running Notarius without arguments, or with **help**, **--help**, or **-h**,
`notarius -h` prints usage and exits successfully. writes the command summary to standard output and exits with status 0.
## `run` ## run
`notarius run <pipeline-id>` executes a configured pipeline against one input ~~~
file. notarius run <pipeline-id> --input path/to/source.json [--json] [flags]
~~~
Flags: The **run** command executes the named pipeline for one input file. The
pipeline ID and **--input** are required.
- `--input path`: required source input file. | Flag | Meaning |
- `--config path`: config file path. If omitted, Notarius uses the discovery | --- | --- |
rules in [Configuration](config.md#discovery). | **--config path** | Use this configuration file. When omitted, configuration discovery applies; see [Configuration](config.md). |
- `--only lane-a,lane-b`: run only the named artifact lanes. Values are | **--input path** | Source input file to process. Required. |
comma-separated and must be non-empty. | **--output-dir path** | Override the configured output root for this run. |
- `--resume`: request checkpoint reuse for this invocation. Checkpoint recording | **--json** | Write the successful run-result receipt as JSON to standard output. |
must be enabled in configuration. See | **--chunk_cache auto\|bypass\|refresh** | Override chunk-plan cache handling for this run. |
[Operations](operations.md#checkpoint-cache) for prerequisites and reuse | **--resume** | Reuse compatible recorded checkpoints when checkpoint recording is enabled. |
behavior. | **--recompute-step step-id** | With **--resume**, recompute the selected ordered step and its dependent lanes. It cannot be combined with **--only**. |
- `--chunk_cache auto|bypass|refresh`: select chunk-plan reuse for this | **--debug** | Retain a debug bundle for this run. |
invocation. `auto` reuses a valid plan by canonical source digest, `bypass` | **--debug-dir path** | Override the debug-bundle root. Requires **--debug**. |
performs no plan-cache I/O, and `refresh` regenerates and replaces a valid | **--only lane-a,lane-b** | Run only the selected comma-separated artifact lanes when that selection is valid for the configured pipeline. |
plan only after chunk validation succeeds. See | **--llm-profile id** | Highest-precedence configured profile for selected LLM-backed bindings and validators; it replaces binding and [pipeline](config.md#pipelines) defaults. |
[Configuration](config.md#state-surfaces) for the persistent setting, precedence, | **--session-id id** | Override the generated prompt session identifier with a non-empty value for LLM-backed module calls. |
and cache-root selection. | **--reasoning-effort value** | Replace the selected PromptKit profile's reasoning effort for every LLM-backed call in this run. The value must be non-empty and the flag may be specified only once. |
- `--output-dir path`: output root. Defaults to `./notarius-output`. | **--clear-reasoning-effort** | Clear reasoning effort inherited from the selected PromptKit profile for every LLM-backed call in this run. |
- `--debug`: allocate and retain one debug bundle for this invocation. | **--reference selector=path** | Add or replace a file reference binding. Repeatable. |
- `--debug-dir path`: debug-bundle root override. This flag requires `--debug`. | **--without-reference selector** | Remove a configured optional reference binding. Repeatable. |
- `--llm-profile id`: override every effective LLM-capable pipeline module
binding with one Scriptorium profile ID. Validator-specific profiles are not
overridden.
- `--session-id id`: pass a stable prompt session identifier through LLM-backed
module calls.
- `--reference selector=path`: bind a reference path to a chunk, extractor,
merger, or normalizer reference slot. Repeatable.
- `--without-reference selector`: remove a configured optional reference binding.
Repeatable. It accepts the same selector forms as `--reference`, without
`=path`.
On success, the command prints the completed pipeline ID, normalized output and **--chunk_cache** accepts only **auto**, **bypass**, or **refresh**.
rejected output counts, and the output directory. A debug-enabled run also **--debug-dir**, **--output-dir**, **--session-id**, and
prints `debug=<bundle-path>`. If the run completes with warnings, the warning **--reasoning-effort**, and **--recompute-step** reject explicit empty values.
count is printed to stderr. **--reasoning-effort** and **--clear-reasoning-effort** are mutually exclusive.
When neither is present, reasoning effort comes from the selected PromptKit
profile. These controls apply to the shared run client, including retries and
LLM-backed validators, and do not modify configuration or profile files.
Persistent reasoning settings remain a PromptKit profile concern.
**--recompute-step** requires **--resume**; checkpoint requirements and reuse
behavior are documented in [Operations](operations.md).
Reference flags are resolved against selected chunk, extractor, merger, and Every run uses one effective prompt session. Without **--session-id**, Notarius
normalizer targets before the run starts. Flat slot names are accepted only generates a stable `notarius:v1:` identifier from the trimmed resolved input
when exactly one selected target declares that slot. For configured reference module key and the input file's exact raw bytes. The same module and bytes
bindings, precedence, path resolution, and validation, see therefore produce the same identifier, regardless of pipeline, references,
[Configuration](config.md#pipelines). profile, retries, or run settings. An explicit non-empty value replaces that
default. Session identifiers are visible to providers; they are non-secret
correlation identifiers, not credential storage. See
[Operations](operations.md#operational-limits) for privacy and workflow
guidance.
`--reference` binds or replaces one slot for one selected target. Selectors are: ### Reference selectors
- `slot=path`: valid when exactly one selected target declares `slot`; Use **--reference** only for a reference slot declared by the selected
- `chunk.slot=path`: target the chunker; configured target. The accepted selector forms are:
- `merge.slot=path`: valid when exactly one selected merger declares `slot`;
- `lane.slot=path`: valid when exactly one selected extractor, merger, or
normalizer in that lane declares `slot`;
- `lane.extract.slot=path`: target a lane extractor;
- `lane.merge.slot=path`: target a lane merger;
- `lane.normalize.slot=path`: target a lane normalizer.
Use `slot=path` when the selected targets declare the slot unambiguously: | Form | Target |
| --- | --- |
| slot=path | The unique selected target that declares slot. |
| chunk.slot=path | The chunker. |
| merge.slot=path | The unique selected merger that declares slot. |
| lane.slot=path | The unique extractor, merger, or normalizer in lane that declares slot. |
| lane.extract.slot=path | The extractor in lane. |
| lane.merge.slot=path | The merger in lane. |
| lane.normalize.slot=path | The normalizer in lane. |
```sh **--without-reference** uses the same selector forms without =path. Slot
go run ./cmd/notarius run dnd-session \ names, requiredness, and configured bindings are part of the
--config examples/dnd-spells.config.yml \ [configuration contract](config.md).
--input examples/seriatim-minimal-transcript.json \
--reference roster=./campaign-roster.txt
```
Use an explicit selector when multiple selected targets declare the same slot or ### Run output
when you want to target a specific target:
```sh Without **--json**, standard output contains the completed pipeline ID, counts
go run ./cmd/notarius run dnd-session \ of normalized and rejected outputs, and the output directory. A debug-enabled
--config examples/dnd-spells.config.yml \ run also prints its debug-bundle path to standard output. A successful run with
--input examples/seriatim-minimal-transcript.json \ warnings reports the warning count to standard error. The published JSON bundle
--reference spells.extract.glossary=./campaign-glossary.txt is defined by the [JSON output contract](integrations/json-output.md).
```
The same grammar can target chunk, merge, and normalize slots when the configured With **--json**, successful standard output is exactly one
modules declare them: `notarius.run-result.v1` JSON document followed by a newline, with no
human-oriented status or debug-path line. Its fields and compatibility policy
are defined by the [run-result contract](integrations/run-result.md). A caller
must check for exit status 0 before decoding this output; a failed write can
leave incomplete standard-output bytes that are not a result document.
```sh Example:
go run ./cmd/notarius run dnd-session \
--config path/to/config.yml \
--input examples/seriatim-minimal-transcript.json \
--reference chunk.scene_guide=./campaign-scenes.txt \
--reference spells.merge.merge_notes=./merge-notes.txt \
--reference spells.normalize.normalization_notes=./normalization-notes.txt
```
Use `--without-reference` to remove a configured optional binding for a run: ~~~
OPENROUTER_API_KEY=your-api-key \
go run ./cmd/notarius run dnd-session \
--config examples/dnd-minimal.config.yml \
--input examples/seriatim-minimal-transcript.json
~~~
```sh ## config validate
go run ./cmd/notarius run dnd-session \
--config examples/dnd-spells.config.yml \
--input examples/seriatim-minimal-transcript.json \
--without-reference glossary
```
Use `--session-id` when an external orchestrator needs all prompt calls from one ~~~
run to share an identifier: notarius config validate [--config path/to/config.yml] [--pipeline pipeline-id] [--only lane-a,lane-b]
~~~
```sh This command loads and validates a configuration. With **--pipeline**, it also
go run ./cmd/notarius run dnd-session \ resolves that pipeline against the production module catalog. **--only** selects
--config examples/dnd-spells.config.yml \ lanes during that resolution and requires **--pipeline**.
--input examples/seriatim-minimal-transcript.json \
--session-id campaign-17-session-04
```
When `cache.checkpoints.enabled` is `true`, runs record checkpoints whether or Success is written to standard output as either config "<path>" is valid or
not `--resume` is present. Add the resume flag to load and reuse compatible config "<path>" is valid for pipeline "<pipeline-id>".
recorded work; using it while checkpoint recording is disabled is an error:
```sh
go run ./cmd/notarius run dnd-session \
--config examples/dnd-spells.config.yml \
--input examples/seriatim-minimal-transcript.json \
--resume
```
Use `--debug` to retain the redacted summary and trace bundle for one run. The
bundle is allocated before pipeline resolution; once allocated, its path is
also printed to stderr if the command fails. Debug-write failures cause exit
code `1`.
```sh
go run ./cmd/notarius run dnd-session \
--config examples/dnd-spells.config.yml \
--input examples/seriatim-minimal-transcript.json \
--debug --debug-dir ./notarius-debug
```
Use `refresh` when intentionally replacing the cached plan for the same source:
```sh
go run ./cmd/notarius run dnd-session \
--config examples/dnd-spells.config.yml \
--input examples/seriatim-minimal-transcript.json \
--chunk_cache refresh
```
Use `bypass` for a one-off run that must not inspect or create plan-cache state:
```sh
go run ./cmd/notarius run dnd-session \
--config examples/dnd-spells.config.yml \
--input examples/seriatim-minimal-transcript.json \
--chunk_cache bypass
```
`--diagnostics-dir` has been removed. For checkpoint behavior, durable output,
debug-bundle lifecycle, and failure inspection, see [Operations](operations.md).
## `config validate`
`notarius config validate` loads and validates configuration.
Flags:
- `--config path`: config file path. If omitted, Notarius uses the discovery
rules in [Configuration](config.md#discovery).
- `--pipeline pipeline-id`: additionally resolve one configured pipeline against
the production module catalog.
- `--only lane-a,lane-b`: validate resolution for selected artifact lanes. This
flag requires `--pipeline`.
Examples: Examples:
```sh ~~~
go run ./cmd/notarius config validate \ go run ./cmd/notarius config validate \
--config examples/dnd-spells.config.yml --config examples/dnd-minimal.config.yml \
--pipeline dnd-session
go run ./cmd/notarius config validate \ OPENROUTER_API_KEY=validation-placeholder \
--config examples/dnd-spells.config.yml \ go run ./cmd/notarius config validate \
--pipeline dnd-session \ --config examples/dnd-complete.config.yml \
--only spells --pipeline dnd-session
``` ~~~
## `pipelines list` The placeholder in the second command is sufficient only for offline
validation; it cannot run a provider-backed pipeline.
`notarius pipelines list` prints configured pipeline IDs in sorted order. ## pipelines list
Flags: ~~~
notarius pipelines list [--config path/to/config.yml] [--json]
~~~
- `--config path`: config file path. If omitted, Notarius uses the discovery This command lists configured pipeline IDs in sorted order. By default, it
rules in [Configuration](config.md#discovery). writes one ID per line to standard output. **--json** writes an object shaped as
- `--json`: print `{"pipelines":[...]}` instead of one ID per line. {"pipelines":[...]} instead.
Examples: ~~~
```sh
go run ./cmd/notarius pipelines list \ go run ./cmd/notarius pipelines list \
--config examples/dnd-spells.config.yml --config examples/dnd-minimal.config.yml
~~~
go run ./cmd/notarius pipelines list \ ## Output Streams And Exit Statuses
--config examples/dnd-spells.config.yml \
--json
```
## Exit Codes Successful commands write their primary result to standard output. Warnings and
errors are written to standard error.
- `0`: command succeeded. For **run --json**, warnings remain on standard error and standard output is a
- `1`: command syntax was valid, but loading config, resolving modules, running machine-readable success result only. Syntax and runtime diagnostics remain on
the pipeline, calling the provider, writing output, or writing a requested standard error. Parse the result only after the process exits with status 0.
debug bundle failed.
- `2`: command syntax was invalid, a command was unknown, a required argument
was missing, or a flag value was malformed.
For YAML structure, defaults, Scriptorium profile sources, environment | Status | Meaning |
overrides, and selectable module and validator keys, see | --- | --- |
[Configuration](config.md). | 0 | The command completed successfully, including root help. |
| 1 | Command syntax was valid but configuration loading or validation, pipeline resolution or execution, provider use, output, or requested debug handling failed. |
| 2 | The command or flag syntax was invalid, including unknown commands, missing required arguments, invalid flag values, or invalid flag combinations. |
The root help spellings are the supported help path. Invoking **--help** on
**run**, **config validate**, or **pipelines list** is handled by the flag
parser as a usage error: it writes an error to standard error and exits with
status 2.

View File

@@ -1,494 +1,490 @@
# Configuration # Configuration
This is the canonical reference for implemented Notarius configuration. This is the canonical reference for Notarius configuration. Configuration files
are YAML and must declare version 4. They select pipelines and their modules;
the [CLI reference](cli.md) owns invocation syntax, and
[Operations](operations.md) owns run-state procedures.
Notarius reads YAML config files with `version: 3`. File configuration is ## Configuration Discovery And Precedence
applied over built-in defaults, then environment overrides are applied. Explicit
CLI overrides are applied last where the command supports them.
## Discovery Commands that load configuration choose a file in this order:
Commands that load configuration use this order: 1. a non-empty **--config** CLI value;
2. a non-empty **NOTARIUS_CONFIG** environment value;
3. the installed default file at **/usr/local/etc/notarius/config.yml**, when
it exists.
1. an explicit path supplied through the CLI, when provided; The command fails if none of these paths provides a configuration file.
2. `NOTARIUS_CONFIG`, when set to a non-empty path;
3. `/usr/local/etc/notarius/config.yml`.
If none is available, the command fails with a config file not found error. For configuration values, precedence is:
The explicit-path option is defined in the [CLI reference](cli.md).
1. built-in defaults;
2. the selected YAML file;
3. supported operational environment variables; and
4. the CLI run overrides that apply to a command.
Environment variables do not provide a second configuration schema. They only
override the fields listed below.
## Maintained Examples ## Maintained Examples
- [Minimal D&D spell configuration](../examples/dnd-spells.config.yml) - [Minimal D&D configuration](../examples/dnd-minimal.config.yml) is a
- [Production-oriented D&D spell configuration](../examples/dnd-spells-production.config.yml) single-lane Seriatim-to-spell pipeline.
- [Complete D&D configuration](../examples/dnd-complete.config.yml) uses
ordered steps, all implemented D&D lanes, generated references, state
settings, bounded LLM concurrency, and the maintained
[operator profile](../examples/profiles/dnd-extraction.yml).
Both are complete version 3 files. The fragments below illustrate individual Use these complete files as starting points rather than combining the
fields and are not alternate complete configurations. illustrative fragments in this reference.
## Top-Level Fields ## File Shape And Defaults
- `version`: required. The only supported value is `3`. Unknown fields, duplicate mapping keys, empty identifiers, and identifiers that
- `scriptorium`: optional Scriptorium profile source settings. become duplicates after trimming whitespace are rejected. Every top-level field
- `pipelines`: optional map of pipeline IDs to pipeline definitions. other than **version** is optional.
- `concurrency`: optional global concurrency settings.
- `output`: optional durable output placement.
- `cache`: optional chunk-plan and checkpoint cache placement.
- `debug`: optional debug-bundle placement. It does not enable debug capture.
Unknown YAML fields are rejected. The removed top-level `llm_profiles` field is | Field | Type | Default | Rules |
rejected; execution profiles now come from Scriptorium. | --- | --- | --- | --- |
| **version** | integer | none | Required; must be 4. |
| **promptkit** | object | none | Profile source and optional local-backend configuration. |
| **pipelines** | map | empty | Maps pipeline IDs to pipeline definitions. |
| **concurrency** | object | see below | Global LLM and extraction limits. |
| **output** | object | see below | Published output settings. |
| **cache** | object | see below | Chunk-plan and checkpoint settings. |
| **debug** | object | see below | Debug-bundle root only; it does not enable capture. |
## Defaults Built-in defaults are:
Built-in defaults: | Field | Default |
| --- | --- |
| **concurrency.total_llm** | 16 |
| **concurrency.stage_workers.extract** | Effective **total_llm** |
| **output.directory** | **./notarius-output** |
| **cache.chunk_plans.mode** | **auto** |
| **cache.chunk_plans.directory** | Empty, selecting the per-user chunk-plan root |
| **cache.checkpoints.enabled** | false |
| **cache.checkpoints.directory** | Empty, selecting the per-user checkpoint root |
| **debug.directory** | **./notarius-debug** |
- `concurrency.total_llm`: `1` An empty cache directory in YAML deliberately selects the corresponding
- `concurrency.stage_workers.extract`: effective `concurrency.total_llm` per-user root. An explicit empty output or debug directory is invalid.
- `output.directory`: `./notarius-output`
- `cache.chunk_plans.mode`: `auto`
- `cache.chunk_plans.directory`: unset, selecting
`<os.UserCacheDir>/notarius/chunk-plans`
- `cache.checkpoints.enabled`: `false`
- `cache.checkpoints.directory`: unset, selecting
`<os.UserCacheDir>/notarius/checkpoints`
- `debug.directory`: `./notarius-debug`
No pipelines are built in. A run requires a configured pipeline. ## PromptKit Profiles
If `scriptorium` is omitted, Notarius uses Scriptorium's built-in profile The optional **promptkit** object selects one source of profile definitions and
catalog. Prompt definitions may also name default profile IDs. The current D&D may register one conventional local OpenAI-compatible backend:
scene and spell prompts default to the built-in `mistral-small-3` profile when a
module binding does not set `llm_profile`. That built-in profile reads its
credential from `OPENROUTER_API_KEY`.
## Scriptorium Profiles ~~~yaml
version: 4
`scriptorium` fields: promptkit:
profile_dir: ./profiles
# profile_file: ./profiles.yml
local_backend:
endpoint: http://localhost:8000/v1
concurrency_limit: 2
~~~
- `profile_dir`: optional directory containing Scriptorium profile YAML files. | Field | Type | Rules |
- `profile_file`: optional Scriptorium profile YAML file.
`profile_dir` and `profile_file` are mutually exclusive. Custom profiles
overlay Scriptorium built-in profiles by profile ID.
Scriptorium profile files use Scriptorium's profile schema. A minimal profile
looks like:
```yaml
id: local-fast
endpoint: http://127.0.0.1:8080/v1
model: your-model
api_key_env: SCRIPTORIUM_API_KEY
timeout_seconds: 180
```
Notarius does not accept raw API keys in Notarius config. For file-backed
Scriptorium profiles, store the environment variable name in `api_key_env` and
set that variable in the run environment. Scriptorium rejects raw `api_key`
fields in profile YAML.
## Environment Overrides
These environment variables are applied after the config file:
- `NOTARIUS_CONFIG`: config discovery path.
- `NOTARIUS_TOTAL_LLM_CONCURRENCY`: integer global LLM concurrency.
- `NOTARIUS_STAGE_WORKERS_EXTRACT`: integer extract worker limit.
- `NOTARIUS_OUTPUT_DIR`: durable output root.
- `NOTARIUS_CACHE_CHUNK_PLANS_MODE`: chunk-plan cache mode.
- `NOTARIUS_CACHE_CHUNK_PLANS_DIR`: chunk-plan cache root.
- `NOTARIUS_CACHE_CHECKPOINTS_DIR`: checkpoint cache root.
- `NOTARIUS_DEBUG_DIR`: debug-bundle root.
Integer environment values must parse as base-10 integers. Directory overrides
must be non-empty after trimming. Cache-directory fields in a file may be
empty, which deliberately selects the corresponding per-user default.
The removed `NOTARIUS_LLM_DEFAULT_*` variables are not read. Configure provider
endpoint, model, and credential environment variable names through Scriptorium
profiles.
## Concurrency
`concurrency` fields:
- `total_llm`: positive integer ceiling on concurrent provider calls.
- `stage_workers`: optional map of framework worker limits. The only supported
key is `extract`.
`stage_workers.extract` defaults to the effective `total_llm` value after file
and environment precedence. It must be between `1` and `total_llm`, inclusive.
Unknown or empty stage-worker keys are rejected. The environment override
`NOTARIUS_STAGE_WORKERS_EXTRACT` takes precedence over the file value, as does
`NOTARIUS_TOTAL_LLM_CONCURRENCY` for the global ceiling.
The worker value is present in effective and redacted configuration. It bounds
the fixed run-wide extract pool and its bounded dispatch queue. Extract jobs are
submitted by source chunk and then resolved lane; `total_llm` independently
bounds actual provider calls made by extracts, retries, and validators.
## Pipelines
A pipeline selects implementations for the fixed workflow defined by
[Architecture](policy/architecture.md#system-shape).
Pipeline fields:
- `input`: required module binding.
- `chunk`: optional module binding. Default module is `generic`.
- `artifacts`: required for pipeline resolution. It maps artifact lane IDs to
lane definitions.
- `output`: optional module binding. Default module is `json`.
- `references`: optional map of reference slot names to reference paths. These
bindings are defaults for eligible pipeline targets that declare the matching
slot.
Artifact lane fields:
- `extract`: required module binding.
- `merge`: optional module binding. Default module is `appendorder`.
- `normalize`: optional module binding. Default module is `noop`.
- `validators`: deprecated lane-level validator list. Non-empty lists are
rejected; use `extract.validators`, `merge.validators`, or
`normalize.validators`.
- `references`: optional compatibility alias for extractor reference bindings.
Lane bindings override pipeline-level bindings for the same slot.
Commands that resolve a pipeline fail for unknown or incompatible module keys.
See [CLI Reference](cli.md) for command syntax.
Reference bindings are validated against reference slots declared by eligible
chunk, extract, merge, and normalize targets during pipeline resolution. Required slots
must be bound after config defaults, target-local references, lane-level
compatibility bindings, and command-line reference overrides are applied.
Config-relative paths are resolved relative to the config file; command-line
reference paths are resolved relative to the current working directory. Bound
files must be UTF-8 text. Reference media types are inferred from file
extensions and checked when a module restricts accepted types; unknown
extensions use `application/octet-stream`. See [CLI Reference](cli.md#run) for
command-line selectors and [Operations](operations.md) for recorded provenance
and sensitive-data handling.
Pipeline-level `references` are defaults. They are valid when at least one
eligible target in the full configured pipeline declares the slot, including
chunk, extractor, merger, and normalizer targets. During a run, they apply only
to the selected targets that declare the slot:
```yaml
pipelines:
dnd-session:
input: seriatim
references:
players: ./campaign/players.txt
party: ./campaign/party-roster.txt
glossary: ./campaign/glossary.txt
artifacts:
spells:
extract: dnd/spells
```
Extractor binding `references` are the canonical lane-local location. The
legacy lane-level `references` field remains supported as an alias; when both
bind the same slot, `extract.references` wins:
```yaml
pipelines:
dnd-session:
input: seriatim
references:
glossary: ./campaign/glossary.txt
artifacts:
spells:
references:
roster: ./campaign/legacy-roster.txt
extract:
module: dnd/spells
references:
party: ./campaign/session-party.txt
```
`chunk.references`, `merge.references`, and `normalize.references` are accepted
in object-form bindings. They override pipeline-level defaults for slots
declared by that target module. Extractor-local references apply only to the
extractor, merger-local references apply only to the merger, and
normalizer-local references apply only to the normalizer.
Target-local reference fields use the same map shape at:
- `pipelines.<id>.chunk.references`
- `pipelines.<id>.artifacts.<lane>.extract.references`
- `pipelines.<id>.artifacts.<lane>.merge.references`
- `pipelines.<id>.artifacts.<lane>.normalize.references`
Each binding is valid only when that target module declares the slot.
## Module Bindings
Every module binding may use shorthand:
```yaml
input: seriatim
```
or object form:
```yaml
chunk:
module: dnd/scenes
llm_profile: local-fast
```
Binding fields:
- `module`: module key.
- `llm_profile`: optional Scriptorium profile ID. Empty or omitted lets the
Scriptorium prompt default select the profile.
- `retries`: non-negative retry count for extra runtime attempts after the
first attempt. Default: `0`. Supported on `chunk`, `extract`, `merge`, and
`normalize` bindings.
- `options`: optional module-specific settings.
- `references`: optional reference bindings. Supported only for `chunk`,
`extract`, `merge`, and `normalize` bindings. `input` and `output` bindings
reject this field during validation.
- `validators`: optional stage-local validator chain override. Supported only
for `chunk`, `extract`, `merge`, and `normalize` bindings. Omit the field to
use the production default chain; set `validators: []` to force an empty
chain; set a non-empty list to use exactly those validators in configured
order.
During resolution, each selected module's registered option validator runs.
Production input, chunk, and output bindings reject unknown or invalid options
with the affected binding context.
Validator bindings use the same shorthand or object module-binding form, but
only these fields are supported:
- `module`: validator key.
- `llm_profile`: optional Scriptorium profile ID for LLM-backed validators.
- `options`: optional validator-specific settings.
Validator bindings reject `references`, `retries`, and nested `validators`.
During resolution, deterministic validators reject explicit `llm_profile`
values.
Configured LLM-backed validators with explicit `llm_profile` values are
validated against the configured Scriptorium profile source. Deterministic
production validators do not call the LLM and must not set `llm_profile`.
## Implemented Production Modules
| Slot | Key | Notes |
| --- | --- | --- | | --- | --- | --- |
| input | `seriatim` | Reads Seriatim transcript JSON. | | **profile_dir** | string | Non-empty directory containing profile files. |
| chunk | `generic` | Splits source units into ordered chunks. | | **profile_file** | string | Non-empty profile file. |
| chunk | `dnd/scenes` | Uses an LLM to split transcript source units into D&D scenes. | | **local_backend** | object | Optional registration for the conventional PromptKit backend ID **local**. |
| extract | `dnd/spells` | Extracts typed D&D spell-list artifacts. | | **local_backend.endpoint** | string | Required when **local_backend** is present; absolute HTTP or HTTPS URL with a host. |
| merge | `appendorder` | Combines typed artifacts in chunk order. | | **local_backend.concurrency_limit** | integer | Optional non-negative limit; defaults to 0. |
| normalize | `noop` | Passes merged typed artifacts through unchanged. |
| normalize | `dnd/spells` | Deterministically canonicalizes and de-duplicates typed D&D spell-list artifacts. |
| output | `json` | Produces JSON output files for normalized `application/json` lanes. |
## Implemented Production Validators Set at most one of **profile_dir** and **profile_file**. Relative values use
the process working directory, not the configuration file's directory. The
complete example's `./examples/profiles/dnd-extraction.yml` value is therefore
valid when Notarius is launched from the repository root; use an absolute path
for services and containers.
| Key | Execution | Notes | An operator source is optional. For a requested ID, PromptKit checks the
configured operator source first, then Notarius's embedded fallback profiles,
then its own built-in catalog. A matching profile is complete: it replaces a
lower-precedence definition rather than merging with it. The maintained
[`dnd-extraction` operator profile](../examples/profiles/dnd-extraction.yml)
is a secret-free deployment artifact; production, development, and local
deployments can each provide a complete definition with that same workload ID.
Use workload-oriented IDs for new profiles instead of model names.
[Operations](operations.md#promptkit-profile-deployment) owns the deployment
workflow and credential-handling guidance.
When **local_backend** is present, its endpoint is trimmed and must use HTTP or
HTTPS case-insensitively, be absolute, and have a non-empty host. URL paths are
allowed. User information, queries, and fragments are rejected. A zero
**concurrency_limit** leaves the local backend unrestricted inside PromptKit;
a positive value limits simultaneous local generations. The application-wide
**concurrency.total_llm** limit still applies in both cases. Neither local
backend field has an environment override. Omitting **local_backend** registers
nothing and preserves existing built-in and endpoint-only profile behavior.
A file-backed PromptKit profile selects the registration by its case-sensitive
backend ID:
~~~yaml
id: local-summary
backend: local
model: example-model
~~~
Keep credentials out of the local-backend object. A PromptKit profile may name
its credential environment variable through `api_key_env`; set that variable
only in the run environment. PromptKit owns the
[pinned profile-file format](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.5.0/docs/formats.md).
The [PromptKit upstream boundary](integrations/pkg-promptkit.md) identifies the
supported package API, and [Operations](operations.md#operational-limits)
describes the effective concurrency layers.
`notarius config validate --pipeline <id>` resolves the selected pipeline and
inspects every explicit effective profile without contacting a provider or
requiring credential values. It rejects absent, malformed, or incompatible
profiles before a run prepares modules. Credential availability is checked only
when a generation is prepared.
## Migrating Version 3 Configuration
Version 3 files are not decoded or rewritten. Change **version: 3** to
**version: 4** and rename the top-level **scriptorium:** section to
**promptkit:**. Version 4 decoding is strict, so a remaining **scriptorium**
field is rejected as unknown.
## Operational Environment Variables
These variables are applied after YAML values:
| Variable | Overrides | Rules |
| --- | --- | --- | | --- | --- | --- |
| `generic/always_accept` | deterministic | Accepts returned module output. | | **NOTARIUS_TOTAL_LLM_CONCURRENCY** | **concurrency.total_llm** | Integer. |
| `generic/always_reject` | deterministic | Rejects returned module output with reason `always_reject`. | | **NOTARIUS_STAGE_WORKERS_EXTRACT** | **concurrency.stage_workers.extract** | Integer. |
| `generic/valid_json` | deterministic | Rejects payloads that are not syntactically valid JSON. | | **NOTARIUS_OUTPUT_DIR** | **output.directory** | Non-empty path. |
| `generic/valid_json_schema` | deterministic | Rejects invalid JSON or JSON that does not conform to the module response schema. | | **NOTARIUS_CACHE_CHUNK_PLANS_MODE** | **cache.chunk_plans.mode** | **auto**, **bypass**, or **refresh**. |
| `extract/dnd/spells/shape` | deterministic | Rejects malformed D&D spell-list artifacts. | | **NOTARIUS_CACHE_CHUNK_PLANS_DIR** | **cache.chunk_plans.directory** | Non-empty path. |
| `extract/dnd/spells/catalog` | deterministic | Rejects spell-list artifacts containing names outside the effective SRD and overlay catalog. | | **NOTARIUS_CACHE_CHECKPOINTS_DIR** | **cache.checkpoints.directory** | Non-empty path. |
| `extract/dnd/spells/source_refs` | deterministic | Rejects missing or invalid D&D spell source references. | | **NOTARIUS_DEBUG_DIR** | **debug.directory** | Non-empty path. |
| `extract/dnd/spells/source_relatedness` | deterministic | Emits warnings when a spell name is not found near its cited source text. |
The production default chain for `dnd/spells` is used for both its extract and Integer values are trimmed then parsed as base-10 integers. Directory and
normalize stages: output values reject NUL characters. **NOTARIUS_CONFIG** participates only in
configuration discovery.
```yaml ## Concurrency, Output, Cache, And Debug
validators:
- generic/valid_json
- generic/valid_json_schema
- extract/dnd/spells/shape
- extract/dnd/spells/catalog
- extract/dnd/spells/source_refs
- extract/dnd/spells/source_relatedness
```
No other production module currently has a default validator chain. Empty ~~~yaml
chains approve output by default. concurrency:
total_llm: 2
The `generic` chunker accepts: stage_workers:
extract: 2
- `max_units`: positive integer, default `50`;
- `overlap_units`: non-negative integer, default `0`, and must be less than
`max_units`.
The `dnd/scenes` chunker requires transcript source capabilities, calls the
configured structured LLM runtime, and does not accept module options. It
declares optional `players`, `party`, and `glossary` references for scene
disambiguation, and accepts `roster` as a deprecated compatibility alias for
`party`.
The `dnd/spells` extractor declares optional campaign reference slots:
- `players`
- `party`
- `glossary`
- `roster` as a deprecated compatibility alias for `party`
These campaign slots accept UTF-8 plain text, Markdown, YAML, or JSON reference
files. The extractor also declares an optional `spell_catalog` slot that accepts
one UTF-8 `application/json` overlay bundle no larger than 1 MiB. The slot does
not allow multiple files. Its format is defined in the
[spell-catalog overlay contract](integrations/dnd-spell-catalog-overlays.md).
The extractor uses campaign references only as supporting disambiguation
material; spell casts still must be present in the source transcript.
The `dnd/spells` normalizer declares the same optional `spell_catalog` slot.
When an overlay is used, bind it independently under
`artifacts.<lane>.normalize.references.spell_catalog`; normalize-stage
references are local to that stage and are not inherited from extraction. The
normalizer uses the embedded SRD catalog when no normalize-stage overlay is
bound.
## State Surfaces
The `output`, `cache`, and `debug` top-level fields select independent physical
roots. Their layout, permissions, lifecycle, and sensitive-data handling are
defined in [Operations](operations.md).
```yaml
output: output:
directory: ./notarius-output directory: ./notarius-output
cache: cache:
chunk_plans: chunk_plans:
directory: ""
mode: auto mode: auto
directory: ./notarius-cache/chunk-plans
checkpoints: checkpoints:
enabled: false enabled: true
directory: "" directory: ./notarius-cache/checkpoints
debug: debug:
directory: ./notarius-debug directory: ./notarius-debug
``` ~~~
`output.directory` is the durable output root. Its precedence is **concurrency.total_llm** must be greater than zero. The only supported
`--output-dir`, `NOTARIUS_OUTPUT_DIR`, the file value, then the default. **concurrency.stage_workers** key is **extract**; its value must be from 1
through **total_llm**. When omitted, it is recalculated from the effective
**total_llm** after YAML and environment precedence.
`cache.chunk_plans.mode` accepts `auto`, `bypass`, or `refresh`. Its precedence **cache.chunk_plans.mode** accepts **auto**, **bypass**, or **refresh**.
is `--chunk_cache`, `NOTARIUS_CACHE_CHUNK_PLANS_MODE`, the file value, then **cache.checkpoints.enabled** is a boolean. The CLI can override the output
`auto`. `auto` reuses a valid source-addressed plan and regenerates missing or directory and chunk-plan mode for one run; see [CLI reference](cli.md#run).
invalid records; `bypass` performs no plan-cache I/O; `refresh` regenerates and
publishes a plan after chunk validation.
`cache.chunk_plans.directory` and `cache.checkpoints.directory` each name an ## Pipelines
exact cache-family root. Their precedence is the corresponding environment
variable, the file value, then the family-specific per-user default. There is
no CLI cache-root override. The defaults are
`<os.UserCacheDir>/notarius/chunk-plans` and
`<os.UserCacheDir>/notarius/checkpoints`; on Unix, `os.UserCacheDir` ordinarily
uses an absolute `$XDG_CACHE_HOME` or falls back to `$HOME/.cache`. A relative
`XDG_CACHE_HOME` is an error.
`cache.checkpoints.enabled` defaults to `false`. When `true`, every run records Each **pipelines** entry has a unique, non-empty ID and the following shape:
checkpoint transitions and reusable approved results. When `false`, Notarius
does not resolve or create the checkpoint root, and `--resume` is rejected.
The `--resume` flag authorizes loading compatible checkpoints; it does not
control recording.
`debug.directory` chooses a root but never enables debug capture. Its precedence ~~~yaml
is `--debug-dir`, `NOTARIUS_DEBUG_DIR`, the file value, then the default. pipelines:
Only `--debug` requests a bundle; `--debug-dir` is valid only with `--debug`. dnd-session:
llm_profile: dnd-extraction
input: seriatim
chunk: generic
output: json
artifacts:
spells:
extract: dnd/spells
merge: appendorder
normalize: dnd/spells
~~~
Every supplied file, environment, and CLI value is validated even when a | Field | Type | Default | Rules |
higher-precedence value wins. | --- | --- | --- | --- |
| **llm_profile** | string | none | Optional non-empty default PromptKit profile ID for selected LLM-backed bindings and validators. An explicitly present blank value is invalid. |
| **input** | module binding | none | Required. |
| **chunk** | module binding | **generic** | Optional. |
| **output** | module binding | **json** | Optional. |
| **artifacts** | map | none | Compact single-step lane map. |
| **steps** | list | none | Ordered lane definitions. Mutually exclusive with **artifacts**. |
| **references** | map | none | External reference defaults for eligible targets. |
## Version 2 To Version 3 Migration Use either **artifacts** or **steps**. The compact **artifacts** form is an
implicit single step. An explicit **steps** list must be non-empty; every step
needs a unique non-empty **id**, an **artifacts** map, and may have
**references**. A lane ID must not appear more than once in a pipeline,
including across explicit steps.
Version 2 files are rejected. Move each setting to the surface it controls and For each selected LLM-backed binding or validator, profile selection occurs
remove obsolete enablement and retention controls. This complete before/after after module, validator, and `--only` lane selection. It uses the
example preserves an existing chunk-plan cache and checkpoint directory while run-level **--llm-profile** value first, then the binding's **llm_profile**,
choosing an output and debug root explicitly. then the pipeline's **llm_profile**, and finally the PromptKit default.
Deterministic bindings do not receive these defaults or run overrides.
```yaml A lane has these fields:
# Version 2 (no longer accepted)
version: 2
workspace:
directory: /srv/notarius/state
resume:
enabled: true
debug:
enabled: true
chunk_cache:
directory: /srv/notarius/chunk-plans
mode: auto
diagnostics:
retention: always
diagnostics:
work_dir: /srv/notarius/inspection
```
```yaml | Field | Type | Default | Rules |
# Version 3 | --- | --- | --- | --- |
version: 3 | **extract** | module binding | none | Required. |
| **merge** | module binding | **appendorder** | Optional. |
| **normalize** | module binding | **noop** | Optional. |
| **references** | map | none | Supported compatibility alias for **extract.references**. |
| **validators** | list | none | Non-empty lane-level lists are rejected. Set validator overrides on a binding instead. |
The lane-level **references** alias remains accepted. When the alias and
**extract.references** bind the same slot, **extract.references** wins. Use
the binding-local form in new configurations.
## Module Bindings And Validators
Use a module key directly when no other binding fields are needed:
~~~yaml
input: seriatim
~~~
Use an object for fields:
~~~yaml
extract:
module: dnd/spells
llm_profile: dnd-extraction
retries: 2
references:
spell_catalog: ./dnd-spell-catalog.json
~~~
| Binding field | Type | Default | Rules |
| --- | --- | --- | --- |
| **module** | string | none | Required for an object binding. Must be a registered compatible key. |
| **llm_profile** | string | none | Optional non-empty PromptKit profile ID for an LLM-backed binding. It overrides the pipeline default unless the run supplies **--llm-profile**. |
| **retries** | integer | 0 | Non-negative additional attempts for chunk, extract, merge, and normalize bindings. |
| **options** | object | none | Must satisfy the selected module. |
| **references** | map | none | Valid only on chunk, extract, merge, and normalize bindings. |
| **validators** | list | production chain | Valid only on chunk, extract, merge, and normalize bindings. |
Omitting **validators** uses the registered chain. **validators: []** selects
an empty chain; a non-empty list replaces the chain in the listed order.
Validator bindings accept only **module**, **llm_profile**, and **options**.
They reject **references**, **retries**, and nested **validators**. Deterministic
validators reject an explicit **llm_profile**. Deterministic module bindings
also reject an explicit **llm_profile**.
The **json** output module accepts optional **include_chunk_map** and
**evidence_context** settings:
~~~yaml
output: output:
directory: /srv/notarius/output module: json
cache: options:
chunk_plans: include_chunk_map: true
directory: /srv/notarius/chunk-plans evidence_context:
mode: auto enabled: true
checkpoints: window_units: 3
enabled: true lanes:
directory: /srv/notarius/state/checkpoints - npc-registry
debug: - spells
directory: /srv/notarius/debug ~~~
```
Run the migrated configuration with `--resume` when checkpoint reuse is wanted, **include_chunk_map** is a boolean and defaults to false. It adds the accepted
and with `--debug` when a debug bundle is wanted. Enabled checkpoint recording chunk map when one exists; its wire format is defined in the
occurs with or without `--resume`. [chunk-map contract](integrations/chunk-map.md).
The removed fields are `workspace.directory`, `workspace.resume.enabled`, Omitting **evidence_context** disables evidence publication. When present, it
`workspace.debug.enabled`, `workspace.chunk_cache.mode`, is an object with these strict fields:
`workspace.chunk_cache.directory`, `workspace.diagnostics.enabled`,
`workspace.diagnostics.retention`, `diagnostics.work_dir`, and | Field | Type | Rules |
`diagnostics.retention`. The removed environment variables are `NOTARIUS_WORKSPACE_DIR`, | --- | --- | --- |
`NOTARIUS_WORKSPACE_DIAGNOSTICS_ENABLED`, | **enabled** | boolean | Required. `false` permits no other evidence fields. |
`NOTARIUS_WORKSPACE_DIAGNOSTICS_RETENTION`, | **lanes** | array of strings | Required and non-empty when enabled. Each value is trimmed and must be unique; every value must name a configured pipeline lane. |
`NOTARIUS_WORKSPACE_RESUME_ENABLED`, `NOTARIUS_WORKSPACE_DEBUG_ENABLED`, | **window_units** | non-negative integer | Optional when enabled; defaults to 3. Zero retains only directly cited units. |
`NOTARIUS_WORKSPACE_CHUNK_CACHE_MODE`,
`NOTARIUS_WORKSPACE_CHUNK_CACHE_DIR`, `NOTARIUS_WORK_DIR`, and Unknown outer or nested option fields are rejected, as are incompatible YAML
`NOTARIUS_DIAGNOSTICS_RETENTION`. The chunk-cache variables are replaced by types. The allowlist remains valid when a run uses lane filtering: a configured
`NOTARIUS_CACHE_CHUNK_PLANS_MODE` and lane that is not active for that invocation simply contributes no evidence.
`NOTARIUS_CACHE_CHUNK_PLANS_DIR`; the former shared directory has no direct Evidence publication is opt-in because it can persist source text and metadata.
replacement. When enabled, it publishes the selected source-unit excerpt defined by the
[Published Evidence Context contract](integrations/evidence-context.md).
## References And Ordered Handoffs
Reference maps bind named slots that the selected target declares. A scalar is
an external path. Pipeline-level maps accept only external paths; step-local
and binding-local maps may also select a normalized artifact from an earlier
step:
~~~yaml
steps:
- id: describe-session
artifacts:
npc-registry:
extract: dnd/npc-registry
normalize: dnd/npc-registry
- id: extract-events
references:
npc_registry:
artifact:
step: describe-session
lane: npc-registry
artifacts:
spells:
extract: dnd/spells
normalize: dnd/spells
~~~
An artifact selector contains only **step** and **lane**. The producer must be
an earlier step and the selected artifact must be compatible with the consumer
slot. A generated binding supplies one accepted normalized artifact; it does
not name a file. A configured generated dependency remains required even when
that consumer slot is otherwise optional.
Pipeline references are defaults. A matching step-local or binding-local
external path overrides a pipeline default. Required slots must be bound after
these configuration values and any CLI reference overrides are applied.
Reference paths in YAML are resolved relative to the configuration file.
### D&D Reference Slots
The following slot names are accepted by the implemented D&D modules when the
selected target declares them:
| Slot | Source and use |
| --- | --- |
| **party** | Optional text campaign context. This is the canonical party-roster spelling. |
| **roster** | Accepted compatibility alias for **party**. |
| **players** | Optional text player context. |
| **glossary** | Optional text campaign glossary. |
| **spell_catalog** | Optional JSON spell-catalog overlay for spell extraction and normalization. See [spell-catalog overlays](integrations/dnd-spell-catalog-overlays.md). |
| **location_registry** | Required normalized location registry for location-occurrence extraction and normalization. |
| **item_registry** | Required normalized item registry for item-occurrence extraction and normalization. |
| **npc_registry** | Normalized NPC registry. Optional for spells and combat turns; required for NPC occurrences and enemy-event extraction and normalization. |
| **scene_descriptions** | Required normalized scene-description artifact for combat-turn and enemy-event extraction. |
| **combat_turns** | Required normalized combat-turn artifact for enemy-event extraction. |
| **npc_occurrences** | Required normalized NPC-occurrence artifact for enemy-event extraction. |
Registry-backed occurrence and enemy-event artifact slots have the following
exact binding contracts. Durable semantics and wire shapes remain in their
[NPC occurrence](integrations/dnd-npc-occurrence-artifacts.md),
[location occurrence](integrations/dnd-location-occurrence-artifacts.md),
[item occurrence](integrations/dnd-item-occurrence-artifacts.md), and
[enemy-event](integrations/dnd-enemy-event-artifacts.md) contracts.
| Slot | Accepted artifact kind | Media type | Maximum size | Required stage |
| --- | --- | --- | --- | --- |
| `npc_registry` | `dnd/npc-registry` | `application/json` | 1,048,576 bytes | extract and normalize |
| `scene_descriptions` | `dnd/scene-description-list` | `application/json` | 1,048,576 bytes | extract only |
| `combat_turns` | `dnd/combat-turn-list` | `application/json` | 1,048,576 bytes | extract only |
| `npc_occurrences` | `dnd/npc-occurrence-list` | `application/json` | 1,048,576 bytes | extract only |
| `location_registry` | `dnd/location-registry` | `application/json` | 1,048,576 bytes | location-occurrence extract and normalize |
| `item_registry` | `dnd/item-registry` | `application/json` | 1,048,576 bytes | item-occurrence extract and normalize |
Scene descriptions accept **party**, **players**, and **glossary**, but not
**roster**. NPC occurrences require **npc_registry** for both extraction and
normalization. Combat turns require **scene_descriptions** for extraction; the
normalized combat-turn module may use optional **npc_registry**. Location occurrences
require **location_registry** for extraction and normalization. Item occurrences require
**item_registry** for extraction and normalization. Enemy-event extraction requires all
four of its JSON artifact slots; its normalizer requires **npc_registry**.
The [complete example](../examples/dnd-complete.config.yml) shows the ordered
generated bindings.
## Production Module Keys
| Kind | Keys |
| --- | --- |
| Input | **seriatim** |
| Chunk | **generic**, **dnd/scenes** |
| Extract | **dnd/spells**, **dnd/npc-registry**, **dnd/combat-turns**, **dnd/item-occurrences**, **dnd/item-registry**, **dnd/npc-occurrences**, **dnd/scene-descriptions**, **dnd/enemy-events**, **dnd/location-registry**, **dnd/location-occurrences** |
| Merge | **appendorder** |
| Normalize | **noop**, **dnd/spells**, **dnd/npc-registry**, **dnd/combat-turns**, **dnd/item-occurrences**, **dnd/item-registry**, **dnd/npc-occurrences**, **dnd/scene-descriptions**, **dnd/enemy-events**, **dnd/location-registry**, **dnd/location-occurrences** |
| Output | **json** |
`dnd/scenes` and every D&D extractor are `llm_backed`. The
`dnd/npc-registry`, `dnd/location-registry`, and `dnd/item-registry`
normalizers are also `llm_backed` for bounded duplicate proposals; every other
D&D normalizer is `deterministic`. LLM-backed bindings use the effective
[PromptKit profile](#promptkit-profiles). The complete example binds each
registry in an earlier step before its occurrence consumer.
The D&D artifact contracts define each emitted schema:
[spells](integrations/dnd-spell-artifacts.md),
[NPC registry](integrations/dnd-npc-registry-artifacts.md),
[NPC occurrences](integrations/dnd-npc-occurrence-artifacts.md),
[combat turns](integrations/dnd-combat-turn-artifacts.md),
[item registry](integrations/dnd-item-registry-artifacts.md),
[item occurrences](integrations/dnd-item-occurrence-artifacts.md),
[scene descriptions](integrations/dnd-scene-description-artifacts.md),
[enemy events](integrations/dnd-enemy-event-artifacts.md),
[location registry](integrations/dnd-location-registry-artifacts.md), and
[location occurrences](integrations/dnd-location-occurrence-artifacts.md).
## Production Validator Keys And Default Chains
Available validator keys are:
| Family | Keys |
| --- | --- |
| Generic | **generic/always_accept**, **generic/always_reject**, **generic/valid_json**, **generic/valid_json_schema** |
| Spells | **extract/dnd/spells/shape**, **extract/dnd/spells/catalog**, **extract/dnd/spells/source_refs**, **extract/dnd/spells/source_relatedness** |
| NPC registry | **extract/dnd/npc-registry/shape**, **extract/dnd/npc-registry/source_refs**, **extract/dnd/npc-registry/source_relatedness**, **normalize/dnd/npc-registry/identity** |
| Combat turns | **extract/dnd/combat-turns/shape**, **extract/dnd/combat-turns/source_refs**, **extract/dnd/combat-turns/source_relatedness**, **normalize/dnd/combat-turns/invariants** |
| Item occurrences | **extract/dnd/item-occurrences/shape**, **extract/dnd/item-occurrences/registry**, **extract/dnd/item-occurrences/source_refs**, **extract/dnd/item-occurrences/source_relatedness**, **normalize/dnd/item-occurrences/invariants** |
| Item registry | **extract/dnd/item-registry/shape**, **extract/dnd/item-registry/source_refs**, **extract/dnd/item-registry/source_relatedness**, **normalize/dnd/item-registry/identity** |
| NPC occurrences | **extract/dnd/npc-occurrences/shape**, **extract/dnd/npc-occurrences/registry**, **extract/dnd/npc-occurrences/source_refs**, **extract/dnd/npc-occurrences/source_relatedness**, **normalize/dnd/npc-occurrences/invariants** |
| Scene descriptions | **extract/dnd/scene-descriptions/shape**, **extract/dnd/scene-descriptions/source_refs**, **extract/dnd/scene-descriptions/source_relatedness**, **normalize/dnd/scene-descriptions/invariants** |
| Enemy events | **extract/dnd/enemy-events/shape**, **extract/dnd/enemy-events/engagements**, **extract/dnd/enemy-events/source_refs**, **extract/dnd/enemy-events/source_relatedness**, **normalize/dnd/enemy-events/invariants** |
| Location registry | **extract/dnd/location-registry/shape**, **extract/dnd/location-registry/source_refs**, **extract/dnd/location-registry/source_relatedness**, **normalize/dnd/location-registry/identity** |
| Location occurrences | **extract/dnd/location-occurrences/shape**, **extract/dnd/location-occurrences/registry**, **extract/dnd/location-occurrences/source_refs**, **extract/dnd/location-occurrences/source_relatedness**, **normalize/dnd/location-occurrences/invariants** |
When no override is configured, production D&D bindings use the following
ordered chains. Each row lists extract then normalize; spell chains are the
same at both stages.
| Lane | Extract | Normalize |
| --- | --- | --- |
| Spells | generic/valid_json, extract/dnd/spells/shape, extract/dnd/spells/catalog, extract/dnd/spells/source_refs, generic/valid_json_schema, extract/dnd/spells/source_relatedness | Same as extract |
| NPC registry | generic/valid_json, extract/dnd/npc-registry/shape, extract/dnd/npc-registry/source_refs, generic/valid_json_schema, extract/dnd/npc-registry/source_relatedness | generic/valid_json, extract/dnd/npc-registry/shape, normalize/dnd/npc-registry/identity, extract/dnd/npc-registry/source_refs, generic/valid_json_schema, extract/dnd/npc-registry/source_relatedness |
| Combat turns | generic/valid_json, extract/dnd/combat-turns/shape, extract/dnd/combat-turns/source_refs, generic/valid_json_schema, extract/dnd/combat-turns/source_relatedness | generic/valid_json, extract/dnd/combat-turns/shape, normalize/dnd/combat-turns/invariants, extract/dnd/combat-turns/source_refs, generic/valid_json_schema, extract/dnd/combat-turns/source_relatedness |
| Item occurrences | generic/valid_json, extract/dnd/item-occurrences/shape, extract/dnd/item-occurrences/registry, extract/dnd/item-occurrences/source_refs, generic/valid_json_schema, extract/dnd/item-occurrences/source_relatedness | generic/valid_json, extract/dnd/item-occurrences/shape, extract/dnd/item-occurrences/registry, normalize/dnd/item-occurrences/invariants, extract/dnd/item-occurrences/source_refs, generic/valid_json_schema, extract/dnd/item-occurrences/source_relatedness |
| Item registry | generic/valid_json, extract/dnd/item-registry/shape, extract/dnd/item-registry/source_refs, generic/valid_json_schema, extract/dnd/item-registry/source_relatedness | generic/valid_json, extract/dnd/item-registry/shape, normalize/dnd/item-registry/identity, extract/dnd/item-registry/source_refs, generic/valid_json_schema, extract/dnd/item-registry/source_relatedness |
| NPC occurrences | generic/valid_json, extract/dnd/npc-occurrences/shape, extract/dnd/npc-occurrences/registry, extract/dnd/npc-occurrences/source_refs, generic/valid_json_schema, extract/dnd/npc-occurrences/source_relatedness | generic/valid_json, extract/dnd/npc-occurrences/shape, extract/dnd/npc-occurrences/registry, normalize/dnd/npc-occurrences/invariants, extract/dnd/npc-occurrences/source_refs, generic/valid_json_schema, extract/dnd/npc-occurrences/source_relatedness |
| Scene descriptions | generic/valid_json, extract/dnd/scene-descriptions/shape, extract/dnd/scene-descriptions/source_refs, generic/valid_json_schema, extract/dnd/scene-descriptions/source_relatedness | generic/valid_json, extract/dnd/scene-descriptions/shape, normalize/dnd/scene-descriptions/invariants, extract/dnd/scene-descriptions/source_refs, generic/valid_json_schema, extract/dnd/scene-descriptions/source_relatedness |
| Enemy events | generic/valid_json, extract/dnd/enemy-events/shape, extract/dnd/enemy-events/engagements, extract/dnd/enemy-events/source_refs, generic/valid_json_schema, extract/dnd/enemy-events/source_relatedness | generic/valid_json, extract/dnd/enemy-events/shape, normalize/dnd/enemy-events/invariants, extract/dnd/enemy-events/source_refs, generic/valid_json_schema, extract/dnd/enemy-events/source_relatedness |
| Location registry | generic/valid_json, extract/dnd/location-registry/shape, extract/dnd/location-registry/source_refs, generic/valid_json_schema, extract/dnd/location-registry/source_relatedness | generic/valid_json, extract/dnd/location-registry/shape, normalize/dnd/location-registry/identity, extract/dnd/location-registry/source_refs, generic/valid_json_schema, extract/dnd/location-registry/source_relatedness |
| Location occurrences | generic/valid_json, extract/dnd/location-occurrences/shape, extract/dnd/location-occurrences/registry, extract/dnd/location-occurrences/source_refs, generic/valid_json_schema, extract/dnd/location-occurrences/source_relatedness | generic/valid_json, extract/dnd/location-occurrences/shape, extract/dnd/location-occurrences/registry, normalize/dnd/location-occurrences/invariants, extract/dnd/location-occurrences/source_refs, generic/valid_json_schema, extract/dnd/location-occurrences/source_relatedness |
Chains are only registered for the D&D extract and normalize modules shown
above; select an explicit override when a different compatible chain is
required.
## Validation ## Validation
Configuration validation checks: Validate a file and one pipeline before running it:
- supported config version and known YAML fields; ~~~sh
- mutually exclusive `scriptorium.profile_dir` and `scriptorium.profile_file`; go run ./cmd/notarius config validate \
- non-empty, non-duplicated IDs after trimming; --config examples/dnd-minimal.config.yml \
- positive global LLM concurrency; --pipeline dnd-session
- supported stage-worker keys and an effective extract worker count in the ~~~
inclusive range `1..concurrency.total_llm`;
- non-empty output and debug directories;
- a supported chunk-cache mode, Boolean checkpoint enablement, and state-surface
directories without NUL bytes;
- stale removed fields such as `llm_profiles`.
Pipeline resolution additionally checks: Configuration validation rejects invalid YAML, unsupported fields, invalid
defaults or environment overrides, incompatible module keys, unknown options,
- the pipeline ID exists; invalid reference bindings, missing required reference slots, invalid validator
- at least one artifact lane is declared and selected; overrides, and incompatible generated artifact handoffs. Use
- lanes selected through the CLI exist in the resolved pipeline; [pipelines list](cli.md#pipelines-list) to inspect configured IDs.
- required module keys are present;
- module keys are registered for the expected slot;
- module capability requirements are satisfied;
- non-empty validator overrides reference registered validator keys;
- deterministic validators do not set `llm_profile`;
- LLM-backed validators with explicit `llm_profile` values reference configured
Scriptorium profile IDs;
- bound reference slots are declared by selected chunk, extractor, merger, or
normalizer targets;
- required reference slots are bound for selected targets.

View File

@@ -0,0 +1,77 @@
# Using Notarius As A Subprocess
Use this workflow when an orchestrator runs Notarius and consumes its published
artifacts. The [CLI reference](../cli.md) owns invocation syntax and exit
statuses, while the [run-result receipt](../integrations/run-result.md) and
[Published JSON Output contract](../integrations/json-output.md) own the
durable result formats.
## Run And Check The Process
Optionally preflight a selected configuration and pipeline before work starts:
```sh
notarius config validate --config /path/to/notarius.yml --pipeline pipeline-id
```
Invoke the run with explicit paths and machine-readable output. Capture
standard output and standard error separately; do not combine them before
processing the result.
```sh
notarius run pipeline-id \
--config /path/to/notarius.yml \
--input /path/to/source.json \
--output-dir /path/to/output-root \
--json
```
Use absolute paths for supplied input, configuration, output-root, and
reference files. Notarius generates a stable prompt session for the resolved
input module and exact input bytes. Pass **--session-id** only when intentionally
grouping different invocations under a different session. Supply credentials
through Notarius's documented configuration and environment mechanisms, never
as command-line arguments or generated secret-bearing configuration. In
particular, a session identifier is provider-visible and is not a credential
mechanism.
Wait for the process before interpreting standard output. Only an exit status
of 0 permits decoding the receipt. On a nonzero exit, retain standard error for
diagnosis and ignore all standard-output bytes: a failed receipt write may have
left a partial document.
## Discover Required Artifacts
Decode the successful receipt and accept the schema versions supported by the
caller. Use its `output_directory` as the bundle root. For the production JSON
output, resolve `index_file` under that root with a confinement check and reject
an absolute path or a result that escapes the root.
Read the resulting `index.json` and locate each artifact by `lane_id`, not by a
guessed filename. Before decoding a selected payload, verify its descriptor's
media type and schema identity against the relevant published artifact
contract. The JSON bundle contract links to the available lane contracts.
If `index.json` has an `evidence_context` descriptor, treat it as a
pipeline-wide artifact rather than a lane entry. Verify its six descriptor
fields before decoding the linked file according to the [Published Evidence
Context contract](../integrations/evidence-context.md). Decode its top-level
source-unit array as a reading excerpt. Obtain authoritative citations and lane
provenance from the normalized lane artifacts; the excerpt has neither and its
nearby units do not widen a lane artifact's cited source reference.
A zero exit status may still report rejected outputs, warnings, or absent
lanes. The caller decides which lane IDs are required for its own work and
which are optional; it should make that decision explicitly rather than infer
failure from the receipt counts alone.
## Preserve Provenance And Handle Data Carefully
Keep the receipt with the published `manifest.json`, and retain
`rejected.json` and `warnings.json` when review or later provenance requires
them. Treat the input, output bundle, cache, debug bundle, and captured process
logs as potentially sensitive data. Apply the caller's access controls and
retention policy, and avoid copying secrets into arguments, logs, or
provenance records. An evidence-context artifact contains source-unit text and
metadata and can cover most of an input; preserve and share it only when that
source content is authorized for the recipient.

View File

@@ -17,11 +17,13 @@ implemented component map.
| Application shape, package boundaries, contracts, dependency direction, runtime guarantees, or safety properties | [Architecture](policy/architecture.md) and relevant [ADRs](adr/) | Architecture defines the intended system and its invariants; ADRs preserve significant decision rationale. | | Application shape, package boundaries, contracts, dependency direction, runtime guarantees, or safety properties | [Architecture](policy/architecture.md) and relevant [ADRs](adr/) | Architecture defines the intended system and its invariants; ADRs preserve significant decision rationale. |
| Any documentation addition or revision | [Documentation Policy](policy/documentation.md) | It defines canonical homes, audiences, current-behavior rules, and maintenance requirements. | | Any documentation addition or revision | [Documentation Policy](policy/documentation.md) | It defines canonical homes, audiences, current-behavior rules, and maintenance requirements. |
| Adding, changing, reviewing, or deleting tests | [Testing Policy](policy/testing.md) | It defines risk-based sufficiency, durable test boundaries, test-double guidance, and criteria for retaining tests. | | Adding, changing, reviewing, or deleting tests | [Testing Policy](policy/testing.md) | It defines risk-based sufficiency, durable test boundaries, test-double guidance, and criteria for retaining tests. |
| CLI composition or command behavior | [CLI Internals](internal/cli.md) and [CLI Reference](cli.md) | The internal guide owns composition and command flow; the reference owns public syntax. |
| Building a subprocess caller or changing its result protocol | [Subprocess Consumer Guide](consumers/subprocess.md), [Run Result Receipt](integrations/run-result.md), and [CLI Internals](internal/cli.md) | These separate caller workflow, durable receipt contract, and CLI implementation behavior. |
| Configuration loading, resolution, or user-visible configuration behavior | [Configuration Internals](internal/configuration.md) and [Configuration](config.md) | The internal guide owns loading and resolution mechanics; the reference owns the configuration contract. |
| Pipeline resolution or execution | [Pipeline Internals](internal/pipeline.md) | It documents profiles, references, validation, retries, checkpoints, and runner behavior. | | Pipeline resolution or execution | [Pipeline Internals](internal/pipeline.md) | It documents profiles, references, validation, retries, checkpoints, and runner behavior. |
| Production modules or validators | [Module Internals](internal/modules.md) | It documents implemented module contracts, capabilities, assets, and registration. | | Production modules or validators | [Module Internals](internal/modules.md), [D&D Module Internals](internal/dnd.md), and [D&D integration contracts](integrations/) | The generic guide owns extension mechanics, the D&D guide owns shared family conventions, and the contracts own durable output shapes. |
| LLM clients, prompts, schemas, profiles, or scheduling | [LLM Runtime](internal/llm.md) | It documents the transport boundary and Scriptorium integration. | | LLM clients, prompts, schemas, profiles, or scheduling | [LLM Runtime](internal/llm.md) | It documents the transport boundary and PromptKit integration. |
| Output, cache, resume, or debug artifacts | [Run State Internals](internal/state.md), [Operations](operations.md), and [Configuration](config.md) | These separate implementation details, operator behavior, and configuration contracts. | | Output, cache, resume, or debug artifacts | [Run State Internals](internal/state.md), [Operations](operations.md), and [Configuration](config.md) | These separate implementation details, operator behavior, and configuration contracts. |
| CLI or user-visible configuration behavior | [CLI Reference](cli.md) and [Configuration](config.md) | These are the canonical user and operator references. |
| External input formats, artifact schemas, or durable output files | [Integration Contracts](integrations/) | Integration documents define external and durable data contracts. | | External input formats, artifact schemas, or durable output files | [Integration Contracts](integrations/) | Integration documents define external and durable data contracts. |
| Proposed or unimplemented behavior | [Roadmap](roadmap/) | Future work belongs only in roadmap documentation until implemented. | | Proposed or unimplemented behavior | [Roadmap](roadmap/) | Future work belongs only in roadmap documentation until implemented. |

View File

@@ -0,0 +1,81 @@
# Accepted Chunk Map
This document defines the optional durable `chunk-map.json` artifact in a
[published JSON bundle](json-output.md). It describes the accepted,
materialized chunk plan used by one run. It is not a lane payload and is never
an input to a later pipeline step.
## Contract Identity
| Property | Value |
| --- | --- |
| Artifact kind | `source/chunk-map` |
| Logical file | `chunk-map.json` |
| Media type | `application/json` |
| Schema ID | `notarius.source.chunk_map` |
| Schema name | `notarius_source_chunk_map_v1` |
| Schema version | `v1` |
The optional `chunk_map` descriptor in `index.json` identifies this artifact.
Export is controlled by the JSON output binding described in
[Configuration](../config.md#module-bindings-and-validators).
## Wire Shape
Every payload has these required fields:
| Field | Meaning |
| --- | --- |
| `source_id` | Accepted source-document identity. |
| `source_digest` | Lower-case `sha256:` digest of that source document. |
| `plan_digest` | Lower-case `sha256:` digest of the logical chunk plan. |
| `requested_chunker` | Chunk module selected by the resolved pipeline. |
| `producer` | Original accepted-plan producer. `input_module` and `chunk_module` are required; `llm_profile` is optional. |
| `plan_annotations` | Plan-level annotation namespace map; `{}` when none are present. |
| `chunks` | Non-empty execution-order chunk collection. |
Each `chunks` entry contains non-empty `id`, zero-based `index`, `source_ref`,
positive `unit_count`, and an explicit `annotations` map. `source_ref` contains
the same `source_id` as the top-level value plus positive inclusive
`start_unit_id` and `end_unit_id` values. Endpoints identify source units; their
numeric values do not by themselves establish source-document order.
Annotation namespaces are non-empty trimmed strings. Their values are arbitrary
valid JSON and are retained without interpreting a module-specific namespace.
## Ordering And Validation
`chunks` are in execution order. Their indexes are contiguous, start at zero,
and equal their array positions; chunk IDs are unique. The emitted map is built
only after the selected plan has been accepted and materialized against the
source document, so its ranges, unit counts, annotations, and digests describe
that exact plan.
The codec rejects malformed JSON, trailing content, unknown fixed-object
fields, invalid identities or digests, invalid annotations, duplicate chunk
IDs, non-contiguous indexes, and a `plan_digest` that does not match the
reconstructed logical plan. The checked-in
[schema](../../internal/framework/chunkmap/assets/schemas/source_chunk_map.v1.json)
defines the strict JSON shape.
## Valid Example
The compact
[source chunk-map fixture](../../internal/framework/chunkmap/testdata/source_chunk_map.v1.json)
is decoded by the production codec and demonstrates an accepted map with
annotations, producer identity, and ordered chunks.
## Publication And Compatibility
The map is present only when a chunk plan was accepted and its export is
enabled. It remains publishable if a later lane is rejected, but is absent when
chunk-plan validation rejects the plan. `requested_chunker` identifies the
current pipeline selection, while `producer` identifies the component that
originally produced the accepted plan; they may differ when an accepted plan is
reused.
The map contains structure rather than source content: it excludes transcript
bytes, source-unit metadata, chunk text, private model output, reference
content, debug data, and filesystem paths. Treat the exported map with the
same care as other published output. Publication location and retention are
defined in [Operations](../operations.md#output-bundles).

View File

@@ -0,0 +1,71 @@
# D&D Combat-Turn Artifact
This contract defines the durable combat-action occurrence list produced by
`dnd/combat-turns`. It records source-grounded turns and actions; it is not a
complete initiative tracker, combat summary, or state model.
## Identity and compatibility
| Property | Value |
| --- | --- |
| Artifact kind | `dnd/combat-turn-list` |
| Schema ID | `notarius.dnd.combat_turns` |
| Schema name | `notarius_dnd_combat_turns_v1` |
| Schema version | `v1` |
| Media type | `application/json` |
`v1` is a strict JSON object with required `combat_turns`; the array may be
empty. Turn and source-reference objects reject unknown fields. An incompatible
shape change requires a new schema version.
## Wire shape
Each combat turn has these required fields:
| Field | Contract |
| --- | --- |
| `actor` | Non-empty acting character or creature name. |
| `turn_kind` | `turn`, `reaction`, `legendary_action`, `lair_action`, or `other`. |
| `source_refs` | One or more transcript evidence ranges. |
Each source reference has exactly `source_id`, `start_unit_id`, and
`end_unit_id`. It identifies an inclusive current-transcript range; unit IDs
are positive and the start may not follow the end.
```json
{
"combat_turns": [
{
"actor": "Mira Thorn",
"turn_kind": "turn",
"source_refs": [
{"source_id": "session-7", "start_unit_id": 31, "end_unit_id": 32}
]
}
]
}
```
## Eligibility, evidence, and normalized form
The extractor requires an approved [scene-description artifact](dnd-scene-description-artifacts.md).
It emits combat turns only for a chunk with an exact matching scene classified
`combat`; an exact non-combat scene produces an accepted empty list. The scene
record controls eligibility only: its title, summary, and reference do not
become turn evidence. No exact matching scene also produces an empty list and
the `scene_classification_unavailable` warning.
An optional normalized [NPC registry artifact](dnd-npc-registry-artifacts.md) can ground an
actor name. Its registry references are provenance, never combat evidence.
Normalization trims and, where possible, canonicalizes actor names; orders and
deduplicates exact source references; orders valid-evidence turns by source
chronology; and collapses only duplicates with the same actor identity, turn
kind, and complete valid evidence. It does not infer turns, initiative, or
actions from registry or scene data.
The [NPC-occurrence artifact](dnd-npc-occurrence-artifacts.md) records
broader NPC occurrences. The [enemy-event artifact](dnd-enemy-event-artifacts.md)
uses combat turns as grounding only; turns do not establish an enemy event or
its outcome. The [JSON output contract](json-output.md) defines publication,
and [D&D module internals](../internal/dnd.md) describes routing and validation
mechanics.

View File

@@ -0,0 +1,114 @@
# D&D Enemy-Event Artifact
This contract defines the durable, source-grounded enemy-event occurrence list.
It records enemies directly established as opposing the party and explicitly
observed combat outcomes. It is an ordered observation artifact from which a
consumer may derive a ledger; it is not a ledger, encounter roster, or terminal
state model.
## Identity and compatibility
| Property | Value |
| --- | --- |
| Artifact kind | `dnd/enemy-event-list` |
| Schema ID | `notarius.dnd.enemy_events` |
| Schema name | `notarius_dnd_enemy_events_v1` |
| Schema version | `v1` |
| Media type | `application/json` |
`v1` is a strict JSON object with required `events`; the array may be empty.
Event and source-reference objects reject unknown fields. An incompatible shape
change requires a new schema version.
## Wire shape
Every event has these required fields:
| Field | Contract |
| --- | --- |
| `name` | Non-empty display name or directly grounded collective subject label. |
| `kind` | `engaged`, `killed`, `fled`, `captured`, or `incapacitated`. |
| `source_refs` | One or more current-transcript evidence ranges. |
Each source reference has exactly `source_id`, `start_unit_id`, and
`end_unit_id`. It identifies an inclusive current-transcript range; unit IDs
are positive and the start may not follow the end.
```json
{
"events": [
{
"name": "Ashfang",
"kind": "engaged",
"source_refs": [
{"source_id": "session-7", "start_unit_id": 41, "end_unit_id": 42}
]
},
{
"name": "Ashfang",
"kind": "fled",
"source_refs": [
{"source_id": "session-7", "start_unit_id": 57, "end_unit_id": 58}
]
}
]
}
```
## Event semantics and evidence
| Kind | Required evidence |
| --- | --- |
| `engaged` | The subject is directly established as actively opposing the party in combat. At most one engagement is emitted for one subject in one combat scene. |
| `killed` | The transcript explicitly establishes that the subject died or was killed. Damage, defeat, disappearance, or combat ending is insufficient. |
| `fled` | The subject explicitly escapes, retreats, or otherwise leaves combat to avoid continued engagement. Movement or absence from later turns is insufficient. |
| `captured` | The subject is explicitly taken prisoner or secured under the party's control. A grapple or temporary restraint alone is insufficient. |
| `incapacitated` | The subject is explicitly rendered unable to continue acting without being established as killed or captured. A missed turn is insufficient. |
The current transcript is the only event evidence. Campaign context and
normalized NPC, scene-description, combat-turn, and NPC-occurrence artifacts
can ground names or control combat eligibility, but none may supply event
evidence. An outcome may share evidence with an engagement, in which case both
events are retained.
Extraction is limited to chunks with an exact combat-scene classification. An
exact non-combat classification produces an accepted empty list. Missing or
mismatched classification also produces an accepted empty list and a
`scene_classification_unavailable` warning.
## Subjects, normalization, and order
A subject matching the normalized NPC registry uses that registry's canonical
display name. Unmatched hostile creatures, summoned entities, and directly
grounded groups remain valid subjects. An unnamed homogeneous group uses the
narrowest transcript-grounded label, such as `Orcs`, `One orc`, or `Remaining
orcs`; the artifact never invents synthetic member identities or quantities.
Party members, allies, neutral observers, mentioned-but-absent enemies, hazards,
traps, and environmental effects are excluded.
Normalization collapses surrounding and repeated internal whitespace in subject
display values, canonicalizes recognized registry names, canonicalizes and
deduplicates exact source ranges, then orders events by valid evidence
chronology, normalized subject identity, display name, kind, and reference
sequence. The deterministic kind tie order is `engaged`,
`incapacitated`, `captured`, `fled`, then `killed`. Only entries with the same
normalized name, kind, and complete canonical evidence sequence are collapsed.
Different kinds, evidence, repeated engagement in separate scenes, and later
outcomes remain separate. A later engagement for the same named subject is
preserved after an earlier outcome because the artifact does not assert an
irreversible state transition.
## Non-goals
The artifact has no NPC or scene ID, quantity, confidence, description,
rationale, summary, current state, or inferred terminal outcome. It does not
emit `active` or `unresolved`; consumers may derive an unresolved ledger view
only when an engagement has no later explicit outcome. It never infers an
outcome from turn absence, scene termination, initiative order, hit-point
guesses, or other artifacts.
The [JSON output contract](json-output.md) defines publication. Configuration
keys, required generated-reference slots, and validator-chain selection are
defined in the [configuration reference](../config.md). Implementation and
prompt-grounding mechanics are described in the
[D&D module internals](../internal/dnd.md).

View File

@@ -0,0 +1,72 @@
# D&D Item-Occurrence Artifact
`dnd/item-occurrences` currently produces this source-grounded item and currency
occurrence list. It records discoveries and possession changes, not an
inventory, balance, or ledger.
## Identity and compatibility
| Property | Value |
| --- | --- |
| Artifact kind | `dnd/item-occurrence-list` |
| Schema ID | `notarius.dnd.item_occurrences` |
| Schema name | `notarius_dnd_item_occurrences_v1` |
| Schema version | `v1` |
| Media type | `application/json` |
`v1` accepts one strict JSON object with required `occurrences`; the array may
be empty. Each occurrence has required `item_id`, `name`, `kind`, and
`source_refs`, and occurrence and source-reference objects reject unknown
fields. `quantity`, `from`, and `to` appear only when their kind permits them.
An incompatible shape change requires a new schema version.
## Registry grounding
Both extraction and normalization require an `item_registry` reference bound to
an earlier normalized `dnd/item-registry` artifact. The registry is immutable
for an operation and contributes names-only grounding after the shared evidence
message. Notarius resolves the model's selected name into the unchanged exact
durable ID/name pair. It is never occurrence evidence.
Each occurrence must use one exact registry ID/name pair. An extraction response
with an unknown or ambiguous selected name is rejected as invalid model output;
the configured pipeline may retry it and never accepts a partial artifact.
Normalization and validation remain defense in depth for artifacts entering
through other boundaries: normalization canonicalizes a recognized name by ID,
preserves unknown values for the registry validator, and the registry validator
rejects unknown or mismatched pairs.
## Wire shape
Each source reference has exactly `source_id`, `start_unit_id`, and
`end_unit_id`. It identifies an inclusive range in the current transcript;
unit IDs are positive and the start may not follow the end.
```json
{
"occurrences": [
{
"item_id": "item:sha256:…",
"name": "Silver Pieces",
"kind": "acquired",
"quantity": 20,
"to": "party",
"source_refs": [
{"source_id": "session-7", "start_unit_id": 2, "end_unit_id": 2}
]
}
]
}
```
The five kinds remain `discovered`, `acquired`, `lost`, `consumed`, and
`transferred`. Holder, quantity, currency, ordering, and exact-duplicate rules
are unchanged: discovered has no holder; acquired requires `to`; lost and
consumed require `from`; transferred requires distinct non-`party` holders.
The only current downstream compatibility requirement is its registry handoff;
the normalized occurrence list is otherwise published for callers. See
[Configuration](../config.md#d-d-reference-slots) for the binding and
[JSON output](json-output.md) for publication.
See [item registry](dnd-item-registry-artifacts.md) for the grounding artifact
and [D&D module internals](../internal/dnd.md) for implementation details.

View File

@@ -0,0 +1,95 @@
# D&D Item Registry Artifact
This contract defines the durable, source-grounded item registry produced by
`dnd/item-registry`. It records transcript-established item types and unique
designations for one source document; it is not an inventory, holder record,
quantity ledger, or item-occurrence artifact.
## Identity and compatibility
| Property | Value |
| --- | --- |
| Artifact kind | `dnd/item-registry` |
| Schema ID | `notarius.dnd.item_registry` |
| Schema name | `notarius_dnd_item_registry_v1` |
| Schema version | `v1` |
| Media type | `application/json` |
| Identity policy | `dnd.item_registry.identity.v1` |
`v1` accepts one strict JSON object with required `items`; the array may be
empty. Item and source-reference objects reject unknown fields. An incompatible
artifact shape or identity-policy change uses a new version or policy.
## Wire shape and identity
Each item has these required fields:
| Field | Contract |
| --- | --- |
| `id` | `item:sha256:` followed by 64 lowercase hexadecimal characters. |
| `name` | Non-empty transcript-established item type or unique designation. |
| `source_refs` | One or more transcript evidence ranges that establish the item. |
A source reference has exactly `source_id`, `start_unit_id`, and `end_unit_id`.
The source ID identifies the transcript, unit IDs are positive inclusive unit
identifiers, and the start may not follow the end.
```json
{
"items": [
{
"id": "item:sha256:31e73b6280ef98e4d8070e07fd4de9b2c3e842cc03af1a09ca631cb95b73e3b3",
"name": "Star Compass",
"source_refs": [
{"source_id": "session-7", "start_unit_id": 4, "end_unit_id": 5}
]
}
]
}
```
The ID is deterministic for an item name or type, rather than for one physical
instance. Notarius normalizes the display name for comparison with Unicode
NFKC, supported apostrophe normalization, collapsed whitespace, and case
folding. It hashes compact JSON for this array:
```text
["dnd.item_registry.identity.v1", comparison_name]
```
The canonical ID is the lowercase SHA-256 digest of those bytes with the
`item:sha256:` prefix. Equal comparison names represent one item identity;
normalization unions their transcript evidence when it safely consolidates a
candidate group.
## Scope, reconciliation, and evidence
The registry includes named unique items, concrete reusable item types, stable
unique designations, and separately established currency denominations. It
excludes vague loot or treasure, generic weapons, quantities, inferred
properties, and inferred uniqueness. Capitalization alone does not establish
eligibility.
Normalization first applies deterministic display, evidence, and ID rules. It
then may use a bounded LLM-assisted proposal to reconcile semantically duplicate
records. The proposal may choose only a supplied candidate display name;
invalid, uncertain, overlapping, or unsafe proposals retain the deterministic
result with retry or fallback diagnostics. A proposal that mixes a recognized
currency denomination with a non-currency item, or combines recognized
denominations, is unsafe and retains every deterministic record. Currency
denominations, materially different item types, and merely nearby objects
remain distinct. Source references establish registry provenance, not evidence
for later artifacts.
## Consumers and publication
`dnd/item-occurrences` requires one approved item registry through its
`item_registry` reference slot for both extraction and normalization. Its
consumer receives names-only grounding; Notarius resolves the selected name
into the unchanged exact durable ID/name pair. The registrys source references
are never occurrence evidence. Unknown or ambiguous selections are rejected by
the occurrence contract. See the
[item-occurrence artifact](dnd-item-occurrence-artifacts.md) for that strict
wire contract, [Configuration](../config.md#d-d-reference-slots) for binding
rules and validator selection, and the [JSON output contract](json-output.md)
for publication.

View File

@@ -0,0 +1,86 @@
# D&D Location-Occurrence Artifact
This contract defines the durable occurrence list produced by
`dnd/location-occurrences`. It records source-grounded ways the party relates
to locations in a required normalized location registry; it does not extend
that registry or infer a place absent from it.
## Identity and compatibility
| Property | Value |
| --- | --- |
| Artifact kind | `dnd/location-occurrence-list` |
| Schema ID | `notarius.dnd.location_occurrences` |
| Schema name | `notarius_dnd_location_occurrences_v1` |
| Schema version | `v1` |
| Media type | `application/json` |
`v1` accepts one strict JSON object with required `occurrences`; the array may
be empty. Occurrence and source-reference objects reject unknown fields. An
incompatible shape change requires a new schema version.
## Wire shape
Each occurrence has these required fields:
| Field | Contract |
| --- | --- |
| `location_id` | Exact ID from the required normalized [location registry](dnd-location-registry-artifacts.md). |
| `name` | Exact canonical display name for `location_id` in that registry. |
| `kind` | One of `visited`, `planned`, `recalled`, or `mentioned`. |
| `source_refs` | One or more current-transcript evidence ranges for this occurrence. |
A source reference has exactly `source_id`, `start_unit_id`, and `end_unit_id`.
It identifies an inclusive range in the current transcript; unit IDs are
positive and the start may not follow the end.
```json
{
"occurrences": [
{
"location_id": "location:sha256:fb05475da0fc7debf994b517e1906ffe7209887a6a1ec306356d84de820b1a24",
"name": "Moon Gate",
"kind": "visited",
"source_refs": [
{"source_id": "session-7", "start_unit_id": 12, "end_unit_id": 13}
]
}
]
}
```
## Occurrence categories
| Kind | Meaning |
| --- | --- |
| `visited` | The transcript establishes physical party presence, including arrival, continuing presence, or departure. |
| `planned` | The party explicitly proposes, intends, or agrees to future travel; speculation alone is not enough. |
| `recalled` | The transcript explicitly recounts prior party presence before the current live events. |
| `mentioned` | The location is explicit but no stronger category applies, including lore, directions, third-party activity, non-actionable speculation, a mere hypothetical reference, or out-of-character discussion. |
For overlapping evidence, precedence is `visited`, then `planned`, then
`recalled`, then `mentioned`. For example, “What if we went to Moon Gate?” is
eligible as `mentioned` when its narrow evidence explicitly references that
registry location, but it is not `planned` without an actual proposal,
intention, or agreement to travel. Inferred, unstated, uncertain, and
unsupported places or occurrences are omitted. Normalization
canonicalizes the registry name, orders and deduplicates source references, and
orders occurrences by source chronology, location ID, name, kind, and reference
sequence. It collapses only exact duplicates with the same ID, kind, and
complete canonical evidence sequence.
## Required grounding and evidence
Both extraction and normalization require exactly one `location_registry` reference of
kind `dnd/location-registry`, media type `application/json`, and at most 1 MiB. The
registry provides identity grounding only. The model selects a supplied
contextual name-and-registry-reference descriptor, and Notarius resolves it
into the exact durable ID/name pair. Unknown, partial, or ambiguous selections
are rejected rather than guessed or reassigned. The current transcript is the
only evidence source for an occurrence; registry evidence and provenance never
become occurrence evidence.
See [Configuration](../config.md#d-d-reference-slots) for the selectable slot
and generated-handoff compatibility, [D&D module internals](../internal/dnd.md)
for implementation behavior, and the [JSON output contract](json-output.md)
for publication.

View File

@@ -0,0 +1,93 @@
# D&D Location Registry Artifact
This contract defines the durable, source-grounded location registry produced
by `dnd/location-registry`. It records transcript-established physical places for one
source document; it is not a map, location hierarchy, campaign-wide world
registry, or location description.
## Identity and compatibility
| Property | Value |
| --- | --- |
| Artifact kind | `dnd/location-registry` |
| Schema ID | `notarius.dnd.location_registry` |
| Schema name | `notarius_dnd_location_registry_v1` |
| Schema version | `v1` |
| Media type | `application/json` |
| Identity policy | `dnd.location_registry.identity.v1` |
`v1` accepts one strict JSON object with required `locations`; the array may be
empty. Location and source-reference objects reject unknown fields. An
incompatible artifact shape or identity-policy change uses a new version or
policy.
## Wire shape and identity
Each location has these required fields:
| Field | Contract |
| --- | --- |
| `id` | `location:sha256:` followed by 64 lowercase hexadecimal characters. |
| `name` | Non-empty transcript-established display name. |
| `source_refs` | One or more transcript evidence ranges that identify the place. |
A source reference has exactly `source_id`, `start_unit_id`, and `end_unit_id`.
The source ID identifies the transcript, unit IDs are positive inclusive unit
identifiers, and the start may not follow the end.
```json
{
"locations": [
{
"id": "location:sha256:fb05475da0fc7debf994b517e1906ffe7209887a6a1ec306356d84de820b1a24",
"name": "Moon Gate",
"source_refs": [
{"source_id": "session-7", "start_unit_id": 4, "end_unit_id": 5}
]
}
]
}
```
The ID is deterministic and scoped to the source document. Notarius normalizes
the display name for comparison with Unicode NFKC, supported apostrophe
normalization, collapsed whitespace, and case folding. It hashes compact JSON
for this array, using the earliest canonical source reference as the anchor:
```text
["dnd.location_registry.identity.v1", comparison_name, source_id, start_unit_id, end_unit_id]
```
The canonical ID is the lowercase SHA-256 digest of those bytes with the
`location:sha256:` prefix. Equal display names are allowed when their evidence
anchors differ, so a generic name does not force distinct places to collapse.
## Scope, reconciliation, and evidence
Locations are physical or spatial places established by the transcript with a
stable proper name or unique in-world designation, such as named planes,
regions, settlements, districts, buildings, rooms, landmarks, routes, and
geographic features. Generic, temporary, relative, and descriptive phrases
such as “the room,” “the bar,” “the hallway,” “outside,” and “upstairs” are not
registry locations. Capitalization alone does not establish eligibility.
Notarius does not infer an unstated place or add hierarchy, coordinates,
descriptions, participants, or ownership.
Normalization first applies deterministic display, evidence, and ID rules. It
then may use a bounded LLM-assisted proposal to reconcile semantically duplicate
records. The proposal is validated and applied conservatively; invalid or
unusable proposals retain the deterministic result with retry or fallback
diagnostics. The registry's source references establish registry provenance,
not evidence for later artifacts.
## Consumers and publication
`dnd/location-occurrences` requires one approved location registry through its
`location_registry` reference slot. Its prompt receives contextual selectors
containing a canonical name and registry references; Notarius resolves a
selection into the unchanged exact durable ID/name pair. Registry references
must not be treated as occurrence evidence. See the
[location-occurrence artifact](dnd-location-occurrence-artifacts.md)
for that contract, [Configuration](../config.md#references-and-ordered-handoffs)
for binding rules, and the [JSON output contract](json-output.md) for
publication.

View File

@@ -0,0 +1,90 @@
# D&D NPC Occurrence Artifact
This contract defines the durable occurrence list produced by
`dnd/npc-occurrences`. It records discrete, source-grounded occurrences with
NPCs already present in a normalized registry; it does not extend that registry
or summarize the session.
## Identity and compatibility
| Property | Value |
| --- | --- |
| Artifact kind | `dnd/npc-occurrence-list` |
| Schema ID | `notarius.dnd.npc_occurrences` |
| Schema name | `notarius_dnd_npc_occurrences_v1` |
| Schema version | `v1` |
| Media type | `application/json` |
`v1` is a strict JSON object with required `occurrences`; the array may be
empty. Occurrence and source-reference objects reject unknown fields. An
incompatible shape change requires a new schema version.
## Wire shape
Each occurrence has these required fields:
| Field | Contract |
| --- | --- |
| `npc_id` | Exact durable ID from the required NPC registry. |
| `name` | Non-empty canonical display name from the required NPC registry. |
| `kind` | One of the occurrence categories below. |
| `source_refs` | One or more transcript evidence ranges. |
Each source reference has exactly `source_id`, `start_unit_id`, and
`end_unit_id`. It identifies an inclusive range in the current transcript;
unit IDs are positive and the start may not follow the end. Extraction evidence
for an occurrence is confined to its accepted chunk.
```json
{
"occurrences": [
{
"npc_id": "npc:sha256:example",
"name": "Mira Thorn",
"kind": "dialogue",
"source_refs": [
{"source_id": "session-7", "start_unit_id": 12, "end_unit_id": 13}
]
}
]
}
```
## Occurrence categories
| Kind | Meaning |
| --- | --- |
| `mentioned` | The NPC is referred to but is not established as present or communicating. |
| `noncombat_presence` | The NPC is present and relevant without meaningful dialogue or combat participation. |
| `dialogue` | The NPC speaks, responds, or meaningfully participates in a non-combat exchange. |
| `combat_ally` | The NPC actively participates in combat on the party's side. |
| `combat_opponent` | The NPC actively participates in combat against the party. |
| `other` | A clearly evidenced direct occurrence not covered by another category. |
The categories do not represent motives, relationships, state, or events that
the cited transcript does not establish. An `other` entry is not a substitute
for uncertain classification.
## Identity, evidence, and order
The required normalized [NPC registry artifact](dnd-npc-registry-artifacts.md)
supplies names-only contextual grounding to the model. Notarius resolves the
selected name and writes the exact `{npc_id, name}` pair. An unknown or
ambiguous selection rejects the complete model result; normalization does not
repair names by similarity. Registry references are provenance only and never
replace an occurrence's own evidence.
The registry may include an identity established by a factual third-party
mention; that provenance alone does not create a `mentioned` occurrence. Each
occurrence remains a separately cited fact in the current transcript.
Normalization validates the exact pair, orders and
deduplicates exact source references, then orders occurrences by valid source
chronology, NPC comparison identity, display name, kind, and reference sequence.
Only entries with the same NPC ID, canonical name, kind, and complete valid evidence
sequence are collapsed; distinct categories or evidence remain separate.
See the [combat-turn artifact](dnd-combat-turn-artifacts.md) for combat-action
occurrences. The [enemy-event artifact](dnd-enemy-event-artifacts.md) consumes
only `combat_opponent` occurrences as grounding; they never establish an enemy
event or outcome. The [JSON output contract](json-output.md) defines
publication. Pipeline mechanics are described in
[D&D module internals](../internal/dnd.md).

View File

@@ -0,0 +1,92 @@
# D&D NPC Registry Artifact
This contract defines the durable NPC registry produced by `dnd/npc-registry`. It is a
minimal, source-grounded identity registry for other D&D artifacts, not a
character sheet or a relationship summary.
## Identity and compatibility
| Property | Value |
| --- | --- |
| Artifact kind | `dnd/npc-registry` |
| Schema ID | `notarius.dnd.npc_registry` |
| Schema name | `notarius_dnd_npc_registry_v1` |
| Schema version | `v1` |
| Media type | `application/json` |
| Identity policy | `dnd.npc_registry.identity.v1` |
`v1` accepts one strict JSON object with required `npcs`; the array may be
empty. NPC and source-reference objects reject unknown fields. An incompatible
artifact shape or identity-policy change uses a new version or policy.
## Wire shape and identity
Each NPC has these required fields:
| Field | Contract |
| --- | --- |
| `id` | `npc:sha256:` followed by 64 lowercase hexadecimal characters. |
| `name` | Non-empty canonical display name. |
| `source_refs` | One or more transcript evidence ranges for the identity. |
A source reference has exactly `source_id`, `start_unit_id`, and `end_unit_id`.
The source ID identifies the transcript, unit IDs are positive inclusive unit
identifiers, and the start may not follow the end.
```json
{
"npcs": [
{
"id": "npc:sha256:35ba5f679aee69e07ae3bd65c44278f29539d5dc9bb5225db1c0060555b23221",
"name": "Mira Thorn",
"source_refs": [
{"source_id": "session-7", "start_unit_id": 4, "end_unit_id": 5}
]
}
]
}
```
The ID is deterministic: normalize the name to Unicode NFKC, normalize the
supported apostrophe forms, collapse whitespace, case-fold it, then serialize
`["dnd.npc_registry.identity.v1", comparison_name]` as compact JSON. SHA-256
those UTF-8 bytes and prefix the lowercase hexadecimal digest with
`npc:sha256:`. Each canonical identity and ID appears at most once.
Normalization collapses records with the same canonical identity, retains their
earliest position, and merges
their canonicalized evidence; it does not add aliases, roles, descriptions, or
relationship fields.
When evidence supports a semantically duplicate group, the canonical display
name is one of that group's supplied candidates. A complete, stable proper name
is preferred over an abbreviation. An unadorned proper name is preferred over
the same name plus a contextual class, role, title, or relationship descriptor
unless the transcript establishes that descriptor as part of the person's
name. A longer candidate is not preferred solely because it includes such a
descriptor.
## Scope and consumers
Only individually identifiable NPC names with transcript evidence belong in
this artifact. A factual third-party mention can establish an identity even if
the NPC is not present, speaking, or acting in the cited passage. Names used
only in hypothetical, speculative, or imagined examples are excluded, as are
groups, generic roles, invented labels, and descriptive enrichment. Its source
references prove registry provenance; they do not become evidence for a spell,
occurrence, combat, or enemy-event occurrence.
Registry evidence establishes an identity, not an [NPC occurrence](dnd-npc-occurrence-artifacts.md).
That later artifact independently records any current-transcript occurrence
with its own cited evidence and category.
This registry can ground actor or caster names in the [spell](dnd-spell-artifacts.md)
and [combat-turn](dnd-combat-turn-artifacts.md) artifacts. It is required to
resolve the canonical `name` in an [NPC occurrence](dnd-npc-occurrence-artifacts.md).
Occurrence consumers receive names-only grounding; Notarius resolves the
selected canonical name and writes the unchanged exact durable ID/name pair.
Spells, combat turns, and the [enemy-event artifact](dnd-enemy-event-artifacts.md)
also receive names-only grounding for actor or subject display. None of these
projections supply later-artifact evidence. [Configuration](../config.md#d-d-reference-slots)
owns the `npc_registry` binding rules.
The [JSON output contract](json-output.md) defines publication, and
[D&D module internals](../internal/dnd.md) owns pipeline mechanics.

View File

@@ -0,0 +1,70 @@
# D&D Scene-Description Artifact
This contract defines the durable output of `dnd/scene-descriptions`. Each
record classifies one accepted transcript chunk and gives it a minimal
source-grounded title and summary.
## Identity and compatibility
| Property | Value |
| --- | --- |
| Artifact kind | `dnd/scene-description-list` |
| Schema ID | `notarius.dnd.scene_descriptions` |
| Schema name | `notarius_dnd_scene_descriptions_v1` |
| Schema version | `v1` |
| Media type | `application/json` |
`v1` is a strict JSON object with required non-empty `scenes`. Scene and
source-reference objects reject unknown fields. An incompatible shape change
requires a new schema version.
## Wire shape
Each scene has exactly these required fields:
| Field | Contract |
| --- | --- |
| `id` | Non-empty accepted chunk ID, assigned by Notarius. |
| `source_ref` | The assigned inclusive source range for that chunk. |
| `kind` | `combat`, `narrative`, `recap`, or `meta`. |
| `title` | Non-empty, trimmed, source-grounded title. |
| `summary` | Non-empty, trimmed, source-grounded summary. |
`source_ref` has exactly `source_id`, `start_unit_id`, and `end_unit_id`.
Its source ID identifies the input transcript; its positive unit IDs identify
the chunk's inclusive range, with the start no later than the end.
```json
{
"scenes": [
{
"id": "chunk-000001",
"source_ref": {"source_id": "session-7", "start_unit_id": 1, "end_unit_id": 3},
"kind": "narrative",
"title": "Arrival at the watchtower",
"summary": "The party reaches the ruined watchtower and begins to investigate it."
}
]
}
```
## Meaning and normalized form
`combat` identifies a chunk where active combat is the central activity.
`narrative` is current in-world play that is not principally combat, recap, or
meta discussion. `recap` is primarily a recounting of an earlier session, and
`meta` is primarily out-of-character discussion. The artifact does not add
participants, confidence, events, or information absent from the chunk.
Normalization trims title and summary, orders scenes by source position and
then ID, and removes exact duplicate records. A reused ID with different
durable fields, or the same source range with different kind, title, or
summary, is invalid. It does not merge adjacent ranges, alter prose, or infer
missing scenes.
The [combat-turn artifact](dnd-combat-turn-artifacts.md) and
[enemy-event artifact](dnd-enemy-event-artifacts.md) use an exact matching
`combat` scene only as eligibility control; scene title, summary, and source
reference never become their evidence. Publication is defined by the
[JSON output contract](json-output.md); implementation details live in
[D&D module internals](../internal/dnd.md).

View File

@@ -1,147 +1,73 @@
# D&D Spell Artifact # D&D Spell Artifact
This document is the durable serialized artifact contract for the production This contract defines the durable output of the `dnd/spells` extractor and
D&D spell extractor. Selectable extractor keys are cataloged in normalizer. It records source-grounded spell-casting occurrences; it is not a
[Configuration](../config.md#implemented-production-modules). spellbook, a rules lookup result, or a record of hypothetical casts.
## Identity ## Identity and compatibility
- Artifact kind: `dnd/spell-list` | Property | Value |
- Prompt ID: `dnd.spells`
- Response schema key: `dnd_spells`
- Response schema ID: `notarius.dnd.spells`
- Response schema name: `notarius_dnd_spells_v1`
- Response schema version: `v1`
- Media type: `application/json`
The durable JSON Schema is owned by the D&D spell artifact codec. The
extractor's private LLM response schema is a separate transport contract: its
source-reference objects omit `source_id`, which the extractor assigns while
mapping the response to the canonical artifact. The LLM DTO and transport
schema are not part of this durable contract.
The output contains canonical spell casts derived from transcript evidence.
Source IDs are assigned from the input identity; source-unit ranges identify
the evidence location.
## Output Shape
The extractor payload is a JSON object with one required top-level array. Its
structure is:
```text
{"spell_casts": [<spell-cast object>, ...]}
```
`spell_casts` must be present. It may be empty when no spell casts are found.
When multiple chunk results are combined, spell casts remain in chunk order.
When the payload is written as durable output, its logical path is derived from
the configured artifact lane ID as defined by the
[JSON output contract](json-output.md#output-payload-files).
## Spell-Cast Fields
Each spell cast contains exactly these required fields:
- `caster`: in-world character or creature casting the spell;
- `spell`: spell name;
- `effect`: concise spell effect in the scene;
- `narrative_description`: short description of the spell cast in context;
- `source_refs`: transcript source references with extractor-assigned source
IDs and evidence unit ranges. It must contain at least one entry.
All four string fields must be non-empty. `caster` is the in-world caster, not
the transcript speaker. The `spell` value must resolve through the effective
SRD-plus-overlay catalog as either a canonical name or alias. Catalog
validation accepts aliases but does not rewrite them; unknown fields are
rejected.
## Source References
Each source reference contains exactly three required fields: `source_id`,
`start_unit_id`, and `end_unit_id`. The source ID must match the input identity.
The unit IDs must be positive integers present in the input, and the start unit
must not appear after the end unit. Unknown fields are rejected.
Reference slot keys and accepted file types are defined in
[Configuration](../config.md#implemented-production-modules). References are
supporting disambiguation material, not source evidence, and are not
addressable through `source_refs`.
## Normalization Behavior
When the `dnd/spells` normalizer is selected, each recognized spell name is
rewritten to the effective catalog's canonical display name. Lookup uses the
catalog's case-insensitive, whitespace-normalizing, apostrophe-normalizing, and
alias rules. Unknown names are preserved exactly for the normalize validators;
the normalizer does not guess or apply fuzzy matching.
Each cast's `source_refs` is copied, sorted by exact `source_id`,
`start_unit_id`, and `end_unit_id`, and stripped of exact structural
duplicates. Adjacent or overlapping ranges are not merged, and the normalizer
does not synthesize references or change their boundaries.
After those per-cast changes, duplicate identity requires the same canonical
spell name, the same caster after case folding and whitespace normalization,
and the same complete, non-empty set of source references valid for the source
document. Only the first occurrence is retained, in stable order. Its caster,
effect, narrative description, and canonical references are preserved without
prose merging or source union. Unknown names, empty or invalid evidence, and
casts with different evidence remain separate.
Mutation and duplicate decisions are returned through the normal warnings
surface. Warning scopes use the merged input index, such as `spell_casts[0]`,
so they remain meaningful even when a later duplicate is removed. The
normalizer uses these reason codes:
| Reason code | Meaning |
| --- | --- | | --- | --- |
| `spell_name_canonicalized` | A catalog lookup replaced an input name with its canonical display name. | | Artifact kind | `dnd/spell-list` |
| `spell_name_unresolved` | A name was not found in the effective catalog and was retained unchanged. | | Schema ID | `notarius.dnd.spells` |
| `source_references_normalized` | Reference order changed or exact duplicate references were removed. | | Schema name | `notarius_dnd_spells_v1` |
| `duplicate_spell_cast_collapsed` | A later cast matched the retained cast's complete duplicate identity. | | Schema version | `v1` |
| Media type | `application/json` |
Only warnings from an accepted normalize attempt are promoted to `v1` is a single strict JSON object. It requires `spell_casts`; the array may
`warnings.json`. If an unresolved name reaches the default normalize validator be empty. Each spell-cast object and source-reference object rejects unknown
chain, the catalog validator rejects the candidate with `unknown_spell`; the fields. An incompatible shape change requires a new schema version.
`spell_name_unresolved` warning remains in the attempt's debug artifact. An
explicit validator override that accepts the candidate promotes the unresolved
warning normally.
## Manifest Metadata ## Wire shape
The extractor adds prompt and response-schema provenance under the artifact lane Each `spell_casts` entry has these required fields:
manifest metadata:
| Field | Contract |
| --- | --- |
| `caster` | Non-empty in-world character or creature name. |
| `spell` | Non-empty spell name. |
| `source_refs` | One or more transcript evidence ranges. |
Every source reference has exactly `source_id`, `start_unit_id`, and
`end_unit_id`. The source ID identifies the input transcript; the unit IDs are
positive inclusive unit identifiers, and the start may not follow the end in
that source. References are evidence for the cast, not campaign-reference or
NPC-registry provenance.
```json ```json
{ {
"metadata": { "spell_casts": [
"extractor": { {
"prompt_id": "dnd.spells", "caster": "Mira Thorn",
"prompt_version": "v1", "spell": "Fireball",
"prompt_sha256": "sha256:...", "source_refs": [
"response_schema_key": "dnd_spells", {"source_id": "session-7", "start_unit_id": 12, "end_unit_id": 13}
"response_schema_id": "notarius.dnd.spells", ]
"response_schema_name": "notarius_dnd_spells_v1",
"response_schema_version": "v1",
"response_schema_sha256": "sha256:...",
"catalog_base_id": "dnd-5e-2014-srd-spells",
"catalog_digest": "sha256:...",
"catalog_overlay_ids": ["campaign.example"]
},
"normalizer": {
"catalog_base_id": "dnd-5e-2014-srd-spells",
"catalog_digest": "sha256:...",
"catalog_overlay_ids": ["campaign.example"]
} }
} ]
} }
``` ```
`catalog_digest` identifies the effective semantic catalog, while ## Evidence and normalized form
`catalog_overlay_ids` is sorted and empty for a base-only configuration. Raw
prompt, schema, catalog, alias, and local overlay-file content are not An entry represents an actual cast or an unambiguous declared attempt. A spell
included in manifest metadata. The `normalizer` metadata uses the same catalog mention, rules discussion, plan, or catalog match alone is not an occurrence.
identity fields when that module is selected. Overlay origin, media type, byte The configured catalog checks the name; it does not establish evidence.
size, and raw digest are recorded separately in the manifest's reference
provenance; see the [JSON output contract](json-output.md#manifestjson). When normalization is selected, recognized spell names use the effective
catalog's canonical display name. Source references are put in canonical source
order and exact duplicate references are removed. A later entry is collapsed
only when it has the same canonical spell, the same case- and
whitespace-insensitive caster identity, and the same complete valid reference
sequence. Remaining entries retain their merged order.
The optional normalized [NPC registry artifact](dnd-npc-registry-artifacts.md) can ground a
caster name. Its own references remain registry provenance and are never copied
into `source_refs`.
## Related contracts
The [spell-catalog overlay contract](dnd-spell-catalog-overlays.md) defines
the configured catalog additions. The [JSON output contract](json-output.md)
defines where this logical artifact is published; [D&D module internals](../internal/dnd.md)
describes extraction and validation mechanics.

View File

@@ -1,19 +1,27 @@
# D&D Spell-Catalog Overlay Contract # D&D Spell-Catalog Overlays
This document defines the JSON format accepted by the D&D spell catalog This document defines the optional JSON overlay consumed by the D&D spell
resolver. An overlay supplies campaign-specific spell names and aliases for extractor. An overlay contributes campaign spell names and aliases for
recognition. It does not supply spell rules, levels, classes, effects, or recognition. It does not define spell rules, effects, levels, classes, or
source evidence. transcript evidence. Bind the optional `spell_catalog` reference as described
in [Configuration](../config.md#references-and-ordered-handoffs).
The `dnd/spells` extractor accepts one optional UTF-8 `application/json` overlay ## Contract Identity
bundle through its `spell_catalog` reference slot. The framework materializes
that file relative to the configuration or command-line binding, enforces the
1 MiB slot limit, and records its origin and raw digest separately from the
effective catalog digest.
## Shape | Property | Value |
| --- | --- |
| Consumer | D&D spell extraction and normalization |
| Reference slot | `spell_catalog` |
| Media type | `application/json` |
| Required schema version | `notarius.dnd.spell-catalog-overlay.v1` |
| Base catalog | Embedded D&D 5e 2014 SRD catalog |
An overlay bundle has this shape: At most one overlay document may be bound. The maintained example is
[dnd-spell-catalog.json](../../examples/dnd-spell-catalog.json).
## Wire Shape
This is a minimal valid overlay:
```json ```json
{ {
@@ -22,49 +30,49 @@ An overlay bundle has this shape:
{ {
"id": "campaign.example", "id": "campaign.example",
"ruleset": "dnd-5e-2014", "ruleset": "dnd-5e-2014",
"source": { "source": {"title": "Example campaign spells"},
"title": "Example campaign spells", "spells": [{"name": "Aegis of Emberfall"}]
"version": "1",
"url": "",
"license": ""
},
"spells": [
{
"name": "Aegis of Emberfall",
"aliases": ["Emberfall Aegis"]
}
]
} }
] ]
} }
``` ```
The top-level `schema_version` and `catalogs` fields are required. The schema | Field | Required | Meaning and constraints |
version must be exactly `notarius.dnd.spell-catalog-overlay.v1`, and at least | --- | --- | --- |
one catalog is required. Catalogs require a unique, non-empty, trimmed `id`, | `schema_version` | Yes | Exactly `notarius.dnd.spell-catalog-overlay.v1`. |
the exact `dnd-5e-2014` `ruleset`, a `source`, and a non-empty `spells` array. | `catalogs` | Yes | Non-empty array of catalog objects with unique IDs. |
| `catalogs[].id` | Yes | Non-empty trimmed string. |
| `catalogs[].ruleset` | Yes | Exactly `dnd-5e-2014`. |
| `catalogs[].source.title` | Yes | Non-empty trimmed string. |
| `catalogs[].source.version` | No | String when present. |
| `catalogs[].source.url` | No | String when present. |
| `catalogs[].source.license` | No | String when present. |
| `catalogs[].spells` | Yes | Non-empty array of spell objects. |
| `catalogs[].spells[].name` | Yes | Non-empty trimmed string. |
| `catalogs[].spells[].aliases` | No | Array of non-empty trimmed strings when present. |
`source.title` is required and must be non-empty and trimmed. `source.version`, Unknown fields are rejected at every object level. The document must contain
`source.url`, and `source.license` are optional strings and may be empty. one JSON value; `null` is not accepted for optional strings or aliases.
Each spell requires a non-empty, trimmed `name`. `aliases` may be omitted or
may be an array of trimmed, non-empty strings; JSON `null` is not an alias
array. Overlay objects contain no other supported spell fields.
Decoding is strict: unknown fields, malformed JSON, trailing JSON values, and ## Composition And Compatibility
non-string optional source fields are rejected.
## Composition Notarius starts with the embedded base catalog, then applies overlay catalogs
in ascending catalog-ID order. A new canonical spell name adds a recognition
entry. If an overlay names an existing canonical spell, it augments that spell
with aliases while retaining the established display spelling.
The resolver always starts with the embedded D&D 5e 2014 SRD catalog. Overlay Repeated aliases for the same spell are accepted. A canonical-name, canonical-
catalogs are sorted by `id` before composition, so the input order does not to-alias, or alias-to-alias collision between different spells is rejected,
affect the result. A new canonical name adds a recognition entry. A canonical including a collision with the embedded catalog. Matching uses the catalogs
name matching an existing canonical name augments that spell and keeps the case, whitespace, and apostrophe normalization, so authors should avoid names
established canonical display spelling. Repeated aliases for the same spell or aliases that normalize to another spell.
are idempotent.
Canonical-name display conflicts and canonical/alias or alias/alias collisions Spell extraction receives the effective catalog as deterministic canonical-name
between different spells are errors, including collisions with the embedded and alias pairs. An alias in the transcript selects its associated canonical
catalog. Canonical names and aliases use the catalog's case, whitespace, and name; the extractor is instructed to return that canonical spelling. The
common-apostrophe normalization rules. The effective catalog returns canonical projection contains no catalog source metadata or provenance, and aliases
names in sorted order and produces a semantic SHA-256 digest that is stable remain recognition context rather than transcript evidence.
under JSON formatting, object-key, catalog, spell, and alias reordering.
The overlay is a recognition aid only. The durable spell-artifact schema and
source-evidence rules are defined by the
[D&D spell artifact contract](dnd-spell-artifacts.md).

View File

@@ -0,0 +1,107 @@
# Published Evidence Context
This contract defines the optional `source/evidence-context` artifact emitted
by the production JSON output. It is a selected source-unit excerpt for
convenient reading alongside normalized lane artifacts; it is not a second
citation or provenance model. Its configuration is owned by
[Configuration](../config.md#module-bindings-and-validators), and its
logical-file discovery is owned by [Published JSON Output](json-output.md).
## Identity And Discovery
When enabled, the JSON bundle contains `evidence-context.json` and an
`index.json` `evidence_context` descriptor with the same six fields as other
pipeline-wide artifact descriptors.
| Property | Value |
| --- | --- |
| Artifact kind | `source/evidence-context` |
| Media type | `application/json` |
| Schema ID | `notarius.source.evidence_context` |
| Schema name | `notarius_source_evidence_context_v1` |
| Schema version | `v1` |
| Logical file | `evidence-context.json` |
Consumers must discover the file from the descriptor, verify all six descriptor
fields, and decode only a supported schema version. The descriptor is optional:
its absence means evidence publication was not enabled for that bundle.
## Payload
The v1 payload is a top-level JSON array of generic source units. There is no
wrapper, source-level metadata, context grouping, lane identifier, or evidence
reference in the payload. An enabled configuration with no contributing
accepted evidence publishes `[]`.
```json
[
{
"id": 10,
"kind": "transcript_segment",
"text": "Aria casts Cure Wounds.",
"ref": {
"source_id": "session-alpha",
"start_unit_id": 10,
"end_unit_id": 10
}
},
{
"id": 20,
"kind": "transcript_segment",
"text": "The party regroups.",
"ref": {
"source_id": "session-alpha",
"start_unit_id": 20,
"end_unit_id": 20
}
}
]
```
Each source unit has required `id`, `kind`, `text`, and self `ref` fields.
`ref` contains `source_id`, `start_unit_id`, and `end_unit_id`, and both unit
endpoints identify that unit's `id`. A unit may also contain source-owned
`metadata`, an open-ended JSON object. Fixed unit and reference fields are
strict: consumers must reject unknown fixed fields, malformed units, invalid
self-references, units whose `source_id` differs from other units in the same
excerpt, and a payload that is not the array described here.
The excerpt preserves each selected unit exactly as represented by the
validated generic source document. It does not add evidence-context-specific
annotations or reshape source-owned metadata.
## Selection And Citations
The framework obtains direct source references only through typed evidence
projections of accepted normalized artifacts in the configured lane allowlist.
It validates each reference against the current source document, expands its
range by `window_units` source-unit positions on each side, clamps at document
boundaries, and takes the union of all expanded ranges. The output contains
each selected source unit once in source-document position order, regardless
of numeric unit IDs. Repeated references, overlapping windows, and citations
from multiple lanes do not duplicate a unit. Rejected, failed, absent,
inactive, and unselected lanes contribute nothing.
Normalized lane artifacts remain authoritative for citations and for which lane
cited a range. The excerpt has no lane attribution and must not be used to
reconstruct it. Its included nearby units provide reading context only; they
do not widen any citation in a lane artifact.
The excerpt contains at most every generic source unit once. It can therefore
equal the complete generic source document when coverage is broad or the
window is large. No byte-, token-, or compression-size guarantee is made, and
the framework does not truncate the excerpt to meet an arbitrary size limit.
## Consumer Responsibilities And Data Handling
The artifact is additive to the JSON bundle and is not a lane payload,
normalized-output count, checkpoint, or generated reference. Consumers that
do not need it must tolerate an absent descriptor. Consumers that do use it
should validate the descriptor and payload before use, retain the artifact with
its schema identity when needed for a run record, and read citations from the
corresponding normalized lane artifacts.
The excerpt contains source-unit text and source-owned metadata and is durable
output. Treat it as sensitive source content, apply appropriate access controls
and retention, and do not assume its selected form is materially smaller or
less sensitive than the original input.

View File

@@ -1,189 +1,142 @@
# JSON Output # Published JSON Output
This document is the durable JSON output file-format contract produced by the This document defines the logical JSON bundle emitted by the production JSON
production JSON encoder and written by the CLI. Selectable output-encoder keys output encoder. The bundles physical destination, atomic publication, and
are cataloged in retention are operational concerns; see [Operations](../operations.md#output-bundles).
[Configuration](../config.md#implemented-production-modules). Output configuration, including chunk-map and evidence-context publication, belongs in
[Configuration](../config.md#module-bindings-and-validators).
The output module produces the logical bundle described here. The CLI's ## Bundle Layout
physical placement and lifecycle for that bundle are defined in
[Operations](../operations.md#output-directory).
## Files All paths below are logical, relative, slash-separated bundle paths. The
encoder always emits the first four JSON files below and adds lane or
pipeline-wide artifact files when their corresponding artifacts are available:
The encoder writes: A subprocess caller first obtains the physical bundle root from the
[run-result receipt](run-result.md), then resolves `index.json` beneath that
root for the logical discovery described here.
- `index.json` | Path | Purpose |
- `manifest.json` | --- | --- |
- `lanes/<lane-id>.json`, one file per normalized serialized artifact | `index.json` | Entry point that names the other published files and lane payloads. |
- `rejected.json` | `manifest.json` | Run provenance and result summaries. |
- `warnings.json` | `rejected.json` | Rejected pipeline outputs. |
| `warnings.json` | Accepted-output and run warnings. |
| `lanes/<safe-lane-id>.json` | One normalized artifact payload for each lane. |
| `chunk-map.json` | Optional accepted chunk map, when its export is enabled and available. |
| `evidence-context.json` | Optional selected source-unit excerpt, when evidence publication is enabled. |
Files are pretty-printed JSON with a trailing newline when the payload is JSON. JSON files are pretty-printed with a trailing newline. Lane payloads are
Logical file paths are relative, slash-separated, and may not contain `..`. accepted only when their media type is `application/json`.
## `index.json` ## `index.json`
Shape: `index.json` is the bundles discovery document. An approved run with no
normalized lanes has this valid minimal index:
```json ```json
{ {
"manifest_file": "manifest.json", "manifest_file": "manifest.json",
"output_files": [ "output_files": [],
{
"lane_id": "spells",
"media_type": "application/json",
"file": "lanes/spells.json",
"module_key": "noop",
"schema_id": "notarius.dnd.spells",
"schema_name": "notarius_dnd_spells_v1",
"schema_version": "v1"
}
],
"rejected_file": "rejected.json", "rejected_file": "rejected.json",
"warnings_file": "warnings.json" "warnings_file": "warnings.json"
} }
``` ```
`output_files` is sorted by lane ID. Output file names are produced by | Field | Required | Meaning |
sanitizing the lane ID: | --- | --- | --- |
| `manifest_file` | Yes | Always `manifest.json`. |
| `output_files` | Yes | Lane descriptors sorted by `lane_id`. |
| `rejected_file` | Yes | Always `rejected.json`. |
| `warnings_file` | Yes | Always `warnings.json`. |
| `chunk_map` | No | Descriptor for the pipeline-wide `chunk-map.json`; never a lane descriptor. |
| `evidence_context` | No | Descriptor for the pipeline-wide `evidence-context.json`; never a lane descriptor. |
- characters outside `A-Z`, `a-z`, `0-9`, `.`, `_`, and `-` become `_`; Each lane descriptor has required `lane_id` and `file`. It may also include
- repeated `..` sequences are replaced; `media_type`, `module_key`, `schema_id`, `schema_name`, and `schema_version`
- leading and trailing `.`, `_`, and `-` are trimmed; when supplied by the normalized artifact. Each pipeline-wide artifact
- empty sanitized names are rejected; descriptor (`chunk_map` or `evidence_context`) contains `artifact_kind`,
- two lanes that sanitize to the same output file are rejected. `file`, `media_type`, `schema_id`, `schema_name`, and `schema_version`. Their
payloads are defined by the [Accepted Chunk Map contract](chunk-map.md) and
[Published Evidence Context](evidence-context.md), respectively.
`manifest_file`, `rejected_file`, and `warnings_file` contain the fixed paths The lane path is derived from its lane ID. Characters outside letters, digits,
shown above. Each `output_files` entry requires `lane_id` and `file`. It also periods, underscores, and hyphens become underscores; `..` sequences are
contains the normalized payload `media_type`, normalizer `module_key`, and neutralized; leading and trailing periods and underscores are removed. A lane
response `schema_id`, `schema_name`, and `schema_version` when those values are that produces an empty name, or two lanes that produce the same path, makes
available. output encoding fail.
## Lane Payloads
Each `lanes/<safe-lane-id>.json` file is the codec-owned normalized JSON for
that lane. Consumers should use the index descriptors schema identity rather
than infer a lane schema from its name. The current D&D payload contracts are
[spells](dnd-spell-artifacts.md), [NPC registry](dnd-npc-registry-artifacts.md),
[NPC occurrences](dnd-npc-occurrence-artifacts.md),
[combat turns](dnd-combat-turn-artifacts.md),
[item registry](dnd-item-registry-artifacts.md),
[item occurrences](dnd-item-occurrence-artifacts.md),
[scene descriptions](dnd-scene-description-artifacts.md),
[enemy events](dnd-enemy-event-artifacts.md),
[location registry](dnd-location-registry-artifacts.md), and
[location occurrences](dnd-location-occurrence-artifacts.md).
## `manifest.json` ## `manifest.json`
`manifest.json` contains a run manifest. This abridged example shows its core `manifest.json` is published provenance, not a copy of lane payloads or a
structure: checkpoint store. Fields without a value may be omitted. Its top-level fields
group into the following externally observable summaries:
```json | Group | Fields |
{ | --- | --- |
"run_id": "run-123", | Run identity and result | `run_id`, `pipeline_id`, `pipeline_digest`, `schema_version`, `validation_status`, `started_at`, `completed_at` |
"pipeline_id": "dnd-session", | Resolved components | `input_module`, `chunker`, `extractors`, `merger`, `normalizer`, `output_encoder`, `artifact_lanes`, `validator_chains`, `module_metadata` |
"artifact_lanes": [ | Source and references | `source_digests`, `references` |
{ | Published result summaries | `normalized_outputs`, `rejected_outputs` |
"id": "spells", | Execution summaries | `chunk_plan`, `checkpoint_decisions`, `llm_profiles`, `metadata` |
"extractor": "dnd/spells",
"merger": "appendorder",
"normalizer": "noop"
}
],
"validation_status": "approved",
"started_at": "2026-01-01T00:00:00Z",
"completed_at": "2026-01-01T00:00:01Z"
}
```
Fields with empty values may be omitted by JSON encoding. `references` records provenance such as the target, slot, origin, digest,
media type, size, and generated-artifact identity. It does not contain
reference content. `normalized_outputs` and `rejected_outputs` likewise
summarize results without embedding lane payload bytes. A chunk-plan summary is
provenance for the plan used by this run; cache records, debug artifacts, and
other operational state are not published as bundle files.
The manifest fields are: When present, `metadata.session_id` is the effective non-secret routing
correlation identifier used for the run. It can be visible to providers and is
not a substitute for a cache or checkpoint identity. Its generation and
override behavior are defined by the [CLI reference](../cli.md#run).
- `run_id`, `pipeline_id`, and `pipeline_digest`: run and resolved-pipeline Each `llm_profiles` entry identifies effective, non-secret LLM execution
identity; provenance:
- `input_module`, `chunker`, `extractors`, `merger`, `normalizer`, and
`output_encoder`: resolved module keys;
- `chunk_plan`: payload-free provenance for the effective chunk plan. `mode`
is the effective cache mode; `action` is `reused`, `generated`,
`refreshed`, or `bypassed` when a plan was materialized. `requested_module`
is the current pipeline chunker, while `producer_input_module`,
`producer_module`, `producer_llm_profile`, `producer_references`,
`producer_metadata`, `source_digest`, `plan_digest`, `plan_schema_version`,
and `created_at` describe the stored or generated producer when available.
A cached plan can therefore identify a producer different from the requested
module. This object never embeds ranges, units, annotations, prompts,
responses, or reference content;
- `module_metadata` and `artifact_lanes`: module and per-lane provenance,
including prompt and response-schema provenance when provided;
- `validator_chains`: resolved validation points and validators;
- `source_digests` and `references`: source and reference provenance;
- `normalized_outputs` and `rejected_outputs`: payload-free result summaries;
- `llm_profiles`: selected profile IDs and provider or model names when
available;
- `metadata`: the effective prompt `session_id`;
- `validation_status`: `approved` or `rejected`;
- `started_at` and `completed_at`: UTC run timestamps.
`source_digests` contains source document digests only. Bound references are | Field | Required | Meaning |
recorded separately under `references`, which contains provenance only: target | --- | --- | --- |
stage, lane ID when present, slot name, origin type and URI, digest, media | `id` | Yes | Selected PromptKit profile identifier. |
type, byte size, and binding source. Reference content is not written to | `provider` | No | Notarius adapter provider identifier. |
durable output. | `model` | No | Effective provider model identifier. |
| `backend_id` | No | Effective PromptKit backend registration identifier. Endpoint-only profiles omit it. |
| `reasoning_effort` | No | Effective opaque provider reasoning setting. An empty or explicitly cleared setting is omitted. |
Reference `stage` is `chunk`, `extract`, `merge`, or `normalize`. `lane_id` is These values describe observed execution; they are not a backend-registration
omitted for chunk references and present for extract, merge, and normalize interface. Entries that differ by backend or effective reasoning remain
references. distinct even when their profile, provider, and model are otherwise equal.
`validation_status` is `approved` when no outputs were rejected and `rejected` ## Rejections And Warnings
when one or more outputs were rejected.
Producer warnings and the current run's chunk-validation warnings remain in `rejected.json` is always an object with a `rejected` array. Each entry has
`warnings.json`. The manifest records only provenance and decision summaries; required `stage` and `message`; `step_id`, `lane_id`, `module_key`, `chunk_id`,
empty producer-only values are omitted for compatibility with existing readers. `chunk_index`, `validator_name`, `reason_code`, `attempt_count`, and
`diagnostic_artifact_path` are present only when applicable.
`validator_chains` records the resolved validator chain for each validation `warnings.json` is always an object with a `warnings` array. Each warning has
point. Entries include stage, lane ID when applicable, module key, and validators `reason_code` and `message`; `scope` is optional. Both arrays are empty when
with key and execution class. Empty chains are recorded with an empty there is nothing to report.
`validators` array, including chains resolved from explicit empty config
overrides.
`normalized_outputs` summarizes each normalized lane output without embedding ## Compatibility
payload bytes. Entries include lane ID, normalizer module key, source ID, media
type, and response schema provenance where available.
`rejected_outputs` summarizes rejected module outputs without embedding raw The index is the authoritative map from a logical lane to its published
payload bytes. Entries include stage, lane, module, chunk, validator or reason, payload. Consumers must tolerate omitted optional manifest and descriptor
message, attempt count, and optional diagnostic artifact path. fields, and should rely on the linked artifact contract for each lanes JSON
shape. This contract describes the published logical bundle only; it does not
## Output Payload Files promise a filesystem layout or expose internal state formats.
Each normalized serialized artifact is written to
`lanes/<sanitized-lane-id>.json`. The JSON output encoder is domain-neutral and
accepts only artifacts whose codec media type is `application/json`. The file
contains the codec-owned JSON bytes pretty-printed.
The schema of each lane payload is owned by that artifact contract. For the
current D&D spell lane, see [D&D Spell Artifact](dnd-spell-artifacts.md).
## `rejected.json`
Shape:
```json
{
"rejected": []
}
```
When output validation rejects an output, each entry contains `stage` and
`message`. It includes `lane_id`, `module_key`, `chunk_id`, `chunk_index`,
`validator_name`, `reason_code`, `attempt_count`, and
`diagnostic_artifact_path` when applicable.
## `warnings.json`
Shape:
```json
{
"warnings": [
{
"scope": "extract",
"reason_code": "example",
"message": "human-readable warning"
}
]
}
```
`warnings` is an empty array when no warnings are reported.
Each warning requires `reason_code` and `message`; `scope` is omitted when it is
empty.

View File

@@ -0,0 +1,108 @@
# PromptKit Integration
Notarius pins
[`gitea.maximumdirect.net/eric/promptkit` v0.5.0](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.5.0)
as its in-process prompt engine. The upstream
[Go package consumer guide](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.5.0/docs/consumers/pkg-promptkit.md)
owns the public engine API, and the upstream
[format reference](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.5.0/docs/formats.md)
owns prompt, profile, and schema file contracts.
## Supported Boundary
Notarius relies on the root `promptkit` package to:
- construct an `Engine` with filesystem-backed prompt, schema, and optional
operator and application-fallback profile sources;
- prepare one frozen execution from a `RunRequest` with named inline artifacts,
variables, a direct session ID, prompt identity, and profile selection, then
record credential-redacted details and run that exact execution;
- return rendered debug material, validated structured output, selected
profile, backend, effective model metadata, and token usage;
- register the optional conventional `local` backend through `BackendLocal`,
`LocalBackend`, and `WithBackend`;
- distinguish structured-output validation failure from execution failure; and
- identify a missing explicit profile through `ErrProfileNotFound` and backend
admission exhaustion through `ErrCapacityExceeded`.
The pinned
[`BackendLocal`, `LocalBackend`, and `WithBackend` API](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.5.0/backends.go)
owns the registration and backend-capacity contract.
For one completion, the adapter calls `PrepareExecution`, takes a
caller-owned `Details` snapshot, and calls `RunPrepared` for that same opaque
prepared execution. It defers `Discard` for every unexecuted handle. Explicit
profile preflight uses `Engine.InspectProfile`; it does not prepare a synthetic
prompt. PromptKit's prepared handle, inspection result, and capacity-error
types stay inside the Notarius LLM adapter.
When a PromptKit profile and runtime override leave `temperature`, `max_tokens`,
or `top_p` unset, Notarius leaves that control unset as well. Compatible
providers therefore apply their own defaults; an operator that requires a
specific sampling value must select it explicitly in the profile or runtime
override.
Notarius does not use PromptKit's optional `ArtifactReader`. It materializes
source and reference content itself and supplies owned inline artifacts at the
adapter boundary. It also retains responsibility for pipeline retries,
scheduling, debug persistence, redaction, profile provenance, and conversion
from private model responses into durable domain artifacts.
Notarius sends one stable effective session through PromptKit's direct session
field, which is authoritative for provider session behavior. It also retains
the same value as the `session_id` prompt variable for maintained prompt
compatibility. The generated identifier is 76 ASCII characters, within
PromptKit v0.5.0's 256-code-point session limit. Session IDs are non-secret
correlation identifiers and may be exposed to providers and provider
observability. The CLI contract owns generation and override behavior.
Notarius records PromptKit's selected backend ID and effective reasoning
setting as optional run-manifest provenance. Endpoint-only profiles have no
backend ID. Debug prompt material also retains the selected backend ID and
PromptKit's stable lower-case `effective_model_params` JSON, which may include
`backend_id`. Notarius production configuration exposes one optional
conventional `local` registration. It does not expose a general user-defined
PromptKit backend registry. Endpoint-only profiles remain supported unchanged.
Notarius retains its application-wide scheduled client around the PromptKit
adapter. PromptKit may apply a narrower limit for the selected backend;
endpoint-only profiles have no such backend limit. The adapter translates
PromptKit capacity rejection into the provider-neutral Notarius
`ErrLLMCapacityExceeded` contract. It may include the normalized selected
backend ID in safe diagnostic context, without exposing PromptKit's capacity
error type, and leaves retries to the calling pipeline stage.
## Profile Sources And Compatibility
Notarius gives PromptKit the configured operator profile source, registered
application fallback profile assets, and optional backend registration through
the same construction path for inspection and execution. PromptKit owns the
resulting source precedence and strict profile parsing: a matching operator
profile is a complete replacement for a fallback or built-in profile, while an
invalid matching document fails instead of falling through. The operator
configuration and deployment workflow are defined in
[Configuration](../config.md#promptkit-profiles) and
[Operations](../operations.md#promptkit-profile-deployment).
Notarius supports this boundary against PromptKit v0.5.0. Its fallback source,
prepared-execution, inspection, and typed capacity APIs are used as public
upstream contracts; other PromptKit APIs or file-format behavior are not
implicitly supported. A dependency upgrade requires reviewing the adapter,
profile-source construction, and this compatibility statement against the
pinned upstream documentation.
## Notarius Ownership
[LLM Runtime Internals](../internal/llm.md) describes how Notarius mounts
module assets, maps its transport-neutral completion contract, prepares and
executes requests, validates output, records provenance, captures debug
material, redacts errors, and preserves timeout ownership.
[D&D Module Internals](../internal/dnd.md) owns the embedded
`dnd-extraction` fallback profile and the maintained D&D prompt defaults.
[Configuration](../config.md#promptkit-profiles) defines how a Notarius
configuration selects one PromptKit profile source and optionally registers
the conventional local backend.
PromptKit API or format changes outside this boundary are not implicitly
supported. Updating the pinned version requires reviewing the adapter and
profile/configuration contracts against the upstream documentation.

View File

@@ -0,0 +1,68 @@
# Run Result Receipt
`notarius run --json` writes this receipt to standard output when a run
completes successfully. It lets a subprocess caller discover the physical root
of the published output bundle without parsing interactive command output.
Command syntax, streams, and exit statuses are defined in the
[CLI reference](../cli.md); logical files within the bundle are defined in the
[Published JSON Output contract](json-output.md).
## Schema
The current schema version is `notarius.run-result.v1`.
| Field | Required | Meaning |
| --- | --- | --- |
| `schema_version` | Yes | Exactly `notarius.run-result.v1`. |
| `run_id` | Yes | The finalized Notarius run identifier. |
| `pipeline_id` | Yes | The effective pipeline identifier. |
| `output_directory` | Yes | Absolute path to the published, run-specific output bundle. |
| `index_file` | For the production JSON output | Logical path `index.json`; omitted for other output modules. |
| `normalized_output_count` | Yes | Number of final normalized outputs. |
| `rejected_output_count` | Yes | Number of recorded rejected outputs. |
| `warning_count` | Yes | Number of final run warnings. |
| `validation_status` | Yes | The final run manifest validation status. |
| `debug_directory` | No | Absolute path to the run-specific debug bundle when requested debug capture completed. |
For the production `json` output module, `index_file` is present only when the
completed run returned exactly one logical output file named `index.json`.
For another output module, its absence does not indicate a failed run.
```json
{
"schema_version": "notarius.run-result.v1",
"run_id": "run-1770000000000000000-0123456789abcdef0123456789abcdef",
"pipeline_id": "dnd-session",
"output_directory": "/work/results/run-1770000000000000000-0123456789abcdef0123456789abcdef",
"index_file": "index.json",
"normalized_output_count": 6,
"rejected_output_count": 2,
"warning_count": 1,
"validation_status": "rejected"
}
```
## Paths And Bundle Discovery
`output_directory` and `debug_directory`, when present, are lexical absolute
paths. They identify the paths used by Notarius and do not resolve symlinks.
`output_directory` is the run-specific bundle, not the configured output root.
The receipt is a summary and discovery document. It does not contain lane
descriptors, payloads, manifest data, rejections, warnings, or file contents.
For the production JSON output, resolve `index_file` beneath
`output_directory`, reject path escapes, and use the
[Published JSON Output contract](json-output.md) to discover logical files and
lane payloads.
## Delivery And Compatibility
Notarius writes the receipt only after the output bundle has been published and
any requested debug terminal reporting has completed. Standard output is not
transactional: a result-write failure returns a nonzero status and can leave
partial bytes. Consumers must ignore standard output unless the process exits
with status 0.
Future versions may add optional fields to this schema. Consumers must tolerate
unknown fields. An incompatible field or semantic change requires a new
`schema_version` value.

View File

@@ -1,69 +1,73 @@
# Seriatim Transcript JSON # Seriatim Transcript Input
This document is the external input contract consumed by the production This document defines the JSON transcript accepted by the production Seriatim
Seriatim input adapter. Selectable input-adapter keys are cataloged in input adapter. It is a source input, not a durable lane artifact. Configure the
[Configuration](../config.md#implemented-production-modules). input adapter through [Configuration](../config.md#production-module-keys).
## Adapter ## Contract Identity
- Source format: `application/vnd.seriatim+json` | Property | Value |
| --- | --- |
| Consumer | Seriatim input adapter |
| Media type | `application/vnd.seriatim+json` |
| Source document kind | `transcript` |
| Source-unit kind | `transcript_segment` |
## Accepted Shape ## Accepted Shape
The input must be one JSON object with top-level `metadata` and `segments` The input is one JSON object containing `metadata` and a non-empty `segments`
fields. This covers the maintained minimal fixture and Seriatim intermediate array. This minimal document is valid:
output that provides the same required segment fields.
The maintained example is ```json
[examples/seriatim-minimal-transcript.json](../../examples/seriatim-minimal-transcript.json). {
"metadata": {"id": "session-alpha"},
"segments": [
{
"id": 1,
"start": 0,
"end": 4,
"speaker": "Aria",
"text": "Aria casts Cure Wounds."
}
]
}
```
Required top-level fields: The maintained two-segment input is
[seriatim-minimal-transcript.json](../../examples/seriatim-minimal-transcript.json).
- `metadata`: an object. Its entries are accepted as source metadata. | Field | Required | Meaning and constraints |
- `segments`: a non-empty array of segment objects. | --- | --- | --- |
| `metadata` | Yes | JSON object. Its entries become source metadata; no particular metadata key is otherwise required. |
| `segments` | Yes | Non-empty array of segment objects, kept in input order. |
| `segments[].id` | Yes | Positive canonical decimal integer, supplied as a JSON number or string. IDs must be unique. |
| `segments[].start` | Yes | Finite, non-negative numeric value, supplied as a JSON number or string. |
| `segments[].end` | Yes | Finite, non-negative numeric value that is not earlier than `start`. |
| `segments[].speaker` | Yes | String that is non-empty after trimming. |
| `segments[].text` | Yes | String that is non-empty after trimming. Its original text is retained. |
Required segment fields: Additional top-level and segment fields are ignored. A missing required field,
`null` in place of an object or array, malformed JSON, or more than one
top-level JSON value is rejected.
- `id`: a positive integer JSON number or canonical decimal string without ## Source Identity And References
leading zeros or surrounding whitespace;
- `start`: a finite, non-negative JSON number or numeric string;
- `end`: a finite, non-negative JSON number or numeric string that is not less
than `start`;
- `speaker`: a non-empty string;
- `text`: a non-empty string.
Other top-level and segment fields, such as `categories`, are ignored. The adapter chooses the source ID in this order:
Multiple top-level JSON values are rejected. 1. a non-empty source ID supplied by the calling request;
2. non-empty string `metadata.id`;
3. non-empty string `metadata.source_id`;
4. `seriatim:` followed by the first 16 hexadecimal characters of the raw
inputs SHA-256 digest.
## Validation Each accepted segment becomes one source unit whose unit ID is `segments[].id`.
Its self-reference uses the derived source ID and the same segment ID for both
range endpoints. Artifact contracts use those segment IDs when they cite
transcript evidence.
The adapter rejects empty input, malformed JSON, multiple top-level JSON values, ## Compatibility
non-object segment values, duplicate segment IDs, and any violation of the
shape or field constraints above.
Segment text is preserved as provided, but it must not be empty after trimming. This adapter accepts only the shape described here. A broader Seriatim export
is usable only when it supplies this object, metadata, and segment shape with
## Derived Identity the stated types and constraints. Unknown additional fields do not add
Notarius behavior.
Notarius identifies the parsed source in this order:
1. `metadata.id`, when it is a non-empty string after trimming;
2. `metadata.source_id`, when it is a non-empty string after trimming;
3. `seriatim:<first-16-hex-chars-of-raw-sha256>`.
The exact raw input SHA-256 remains the basis of the fallback source ID. The
source digest recorded in output provenance is instead the SHA-256 of the
canonical generic source document, excluding the digest field itself. It covers
the derived source identity, document kind and format, ordered units and their
self-references, and accepted metadata. Segment IDs become the unit IDs used by
artifact source references; each produced unit carries a self-reference whose
source ID is the derived document ID and whose start and end IDs both equal the
segment ID.
## Compatibility Limit
This contract covers only Seriatim transcript JSON with the top-level
`metadata` object and `segments` array described here. Broader Seriatim output
schemas are compatible only when they provide these required fields with the
accepted types.

172
docs/internal/cli.md Normal file
View File

@@ -0,0 +1,172 @@
# CLI Internals
This document describes **internal/cli**, Notarius's production composition
root. The [CLI reference](../cli.md) owns command syntax and exit statuses;
[Configuration](../config.md) owns configuration values; and
[Operations](../operations.md) owns filesystem layout, recovery, and operator
procedures.
## Inputs, Outputs, And Boundaries
The CLI accepts process arguments, standard streams, and injectable options
used by tests and embedding code. It writes command results to the supplied
streams and returns a process exit status. For a run, it also creates the
production catalog and runtime collaborators, hands a prepared pipeline and
source bytes to the framework, and places the logical files returned by the
runner.
It is the only boundary allowed to compose concrete registries, LLM clients,
cache/checkpoint collaborators, debug recorders, and physical output paths.
Pipeline modules receive interfaces and request data rather than CLI streams or
filesystem roots. The [Architecture](../policy/architecture.md) defines this
composition-root boundary; [Pipeline Internals](pipeline.md) owns resolution,
preparation, and runner mechanics after their inputs are supplied.
## Dispatch And Configuration Handoff
The root dispatcher handles help, configuration validation, pipeline listing,
and a pipeline run. It normalizes injectable options before dispatch so that a
missing production dependency fails as a command error rather than reaching
execution.
Commands that need configuration use one shared loader. The CLI discovers the
file, parses it through **internal/core/config**, starts from defaults, applies
the file and supported environment overrides, and then validates it for the
command. The configured discovery and precedence contract is in
[Configuration](../config.md), while the loading and resolution mechanics are
in [Configuration Internals](configuration.md).
Configuration validation without a selected pipeline checks structural
configuration only. Validation with a selected pipeline also builds the
effective catalog, resolves the pipeline, and verifies every explicit effective
PromptKit profile. Selected LLM-backed input, chunk, lane, output, and validator
profiles are inspected
against the configured PromptKit source and backend registrations without
loading a prompt or performing generation, so an unknown or invalid profile
fails before pipeline preparation. Credential availability remains an
execution-time concern. Pipeline listing validates configuration before
returning normalized, sorted identifiers.
## Production Composition
The production composition helper allocates every framework registry and the
prompt-asset registry, then registers the generic, Seriatim, and D&D module
families in that order. The resulting registries provide both the module
catalog used for resolution and the concrete constructors used for preparation.
Tests may provide a catalog or registries instead; production code must not
silently merge an injected partial catalog with production registrations.
The production LLM factory builds one PromptKit-backed client from the resolved
**promptkit.profile_dir** or **promptkit.profile_file** source, attaches the
profile-provenance recorder, creates one scheduler from the effective global
LLM limit, and wraps the client before it reaches modules. Registration and LLM
construction errors are returned before a pipeline is prepared. Configuration
field definitions remain in [Configuration](../config.md#promptkit-profiles);
the D&D registrar's fallback profile assets and the adapter mechanics remain in
[LLM Runtime](llm.md).
The factory also accepts `LLMRuntimeOverrides`, whose reasoning pointer
preserves inherit, replace, and clear states across the composition boundary.
Run orchestration constructs this value from the mutually exclusive
`--reasoning-effort` and `--clear-reasoning-effort` controls. Absence preserves
a nil pointer, replacement is trimmed, and clear uses a non-nil empty string.
The same override reaches the one shared production client, checkpoint
identity, and debug invocation metadata. Persistent reasoning configuration
remains owned by PromptKit profiles; Notarius configuration has no reasoning
field.
## Run Orchestration
After parsing and validating a run invocation, the CLI performs this ordered
handoff:
1. load and validate configuration, then apply command-level operational
overrides;
2. create and validate a safe run identity, then allocate a debug bundle only
when requested;
3. build the effective catalog, resolve requested reference changes, resolve
the effective pipeline, and inspect its explicit effective PromptKit
profiles;
4. materialize external or generated references and record redacted invocation
and resolution provenance when debug capture is enabled;
5. construct registries, the scheduled LLM client, and prepared modules;
6. read the source input once, resolve its effective session from the explicit
override or resolved input module and raw bytes, then construct requested
checkpoint collaborators and invoke the framework runner with that same
value; and
7. write the runner's logical output files only after a successful run, then
complete the command report and user-facing result.
Preparation happens before source parsing, so module construction and
dependency failures cannot begin stage execution. The CLI also preserves the
framework's result and warning information when it writes summaries and the
final command result. Detailed state lifecycle, resume handling, and physical
path confinement are maintained in [Run State Internals](state.md) and
[Operations](../operations.md).
The CLI owns the versioned generated-session policy and resolves the sole
effective value before checkpoint construction. It records that value in the
final debug invocation summary when capture is enabled and passes it unchanged
to checkpoint identity and `pipeline.RunInput`. The public flag and stability
contract are defined by the [CLI reference](../cli.md#run); framework and LLM
packages only transport the supplied value.
For `run --json`, the CLI constructs and encodes its private run-result receipt
after a successful runner result is available, before it publishes logical
output files. It writes the prepared receipt to standard output only after
output publication and requested debug terminalization succeed. A receipt-write
failure exits with runtime status 1 and may leave partial standard-output bytes,
but the already-published output bundle remains complete and requested debug
reporting remains successfully terminalized. The CLI reports a bounded
command-owned error and does not repeat terminal reporting. The receipt remains
a CLI reporting concern rather than a framework or output-module responsibility;
its public contract is the
[run-result receipt](../integrations/run-result.md).
## Failure Mapping And Terminal Reporting
Argument, flag, and invocation-combination failures are reported to standard
error before runtime composition and use the syntax error class. Once an
invocation is syntactically valid, configuration loading and validation,
resolution, registration, profile checks, reference materialization, module
construction, input reads, runner failures, output publication, and requested
debug handling use the runtime failure class. The public status numbers and
stream contract are defined in the [CLI reference](../cli.md#output-streams-and-exit-statuses).
When debug capture has been allocated, one command-state value records the
known run result. Guarded terminalization writes a success report once, or
attempts a failure report and error record once. A persistence failure is
reported in addition to the original failure and never replaces it. If a debug
path exists, failure output includes that path so the retained diagnostic data
is discoverable.
## Invariants To Preserve
- Only the CLI composes production implementations and physical runtime roots.
- Configuration and resolved composition failures occur before module
preparation or source parsing.
- A runner's logical files are published only after a successful run.
- Production registries and a caller-supplied catalog or registries are
alternative composition sources, not an implicit mixture.
- A requested debug bundle has one terminal report attempt; its persistence
errors supplement rather than obscure the primary command error.
- User-facing flags, paths, exit codes, and configuration fields are defined
by their public documentation, not duplicated here.
## Focused Tests
- **internal/cli/command_contract_test.go** covers dispatch, help, syntax and
runtime error classes, discovery, validation, and listing.
- **internal/cli/run_contract_test.go** covers the run handoff, publication,
debug reporting, and command-owned state collaborators.
- **internal/cli/production_contract_test.go** covers registrar composition,
production catalog contents, assets, and representative configuration
validation.
- **internal/cli/reference_contract_test.go** covers CLI reference overrides,
origin separation, and materialization boundaries.
- **internal/cli/state_hardening_test.go** covers safe run identity, state
roots, and failure ordering.
Run **go test ./internal/cli** after changing command composition or command
behavior. Pair it with **go test ./internal/core/config** when the configuration
handoff changes.

View File

@@ -0,0 +1,144 @@
# Configuration Internals
This document describes the maintainer-facing configuration boundary in
**internal/core/config**. The [Configuration](../config.md) reference owns the
file format, fields, defaults, precedence contract, and selectable keys. The
[CLI reference](../cli.md) owns command syntax; this document does not redefine
either interface.
## Boundary
The configuration package turns a selected YAML file and supported environment
values into a validated, independently owned configuration. It then resolves a
requested pipeline against a module catalog before the framework prepares or
runs anything.
| Boundary | Inputs | Outputs | Does not own |
| --- | --- | --- | --- |
| Loading | Selected file path and environment lookup | Parsed file model and a populated **Config** | Choosing the file path or reporting a command result. |
| Validation | **Config** | Structural configuration errors with pipeline, lane, or binding context | Module availability, capabilities, or construction. |
| Resolution | Valid **Config**, selected pipeline and lanes, runtime reference changes, LLM override, and module catalog | **EffectiveConfig** with a **ResolvedPipeline** | Materializing reference bytes, preparing modules, execution, or filesystem state. |
| Summary | **Config** or **EffectiveConfig** | Detached redacted payload suitable for debug summaries | Redacting arbitrary process state or provider traffic. |
The CLI discovers a configuration file, invokes this package, and supplies the
result to the framework. Configuration never reads an input file, constructs a
module, or creates output, cache, or debug paths. Those responsibilities remain
at their respective [CLI](cli.md), [pipeline](pipeline.md), and
[run-state](state.md) boundaries.
## Loading And Validation
The CLI loads configuration in this order:
1. parse the selected YAML file strictly into the file model;
2. start from **Default**;
3. apply the file model; and
4. apply the supported environment overrides.
This establishes the public precedence order without giving environment input a
second file schema. Loading and application reject malformed YAML, unsupported
file versions, unknown fields, invalid values, and identifiers that are empty
or collide after whitespace normalization. The file application also makes the
effective extraction-worker default follow the effective LLM limit. A present
PromptKit local-backend object requires and trims its endpoint, defaults its
omitted concurrency limit to zero, and is copied so the parsed file model
cannot alias the populated **Config**. A pipeline `llm_profile` is
presence-aware: omission remains empty, while a present blank value is
rejected and a non-empty file value is trimmed before it reaches **Config**.
**Config.Validate** checks configuration-only invariants before resolution. It
rejects incompatible profile sources, invalid state-surface values, unsupported
concurrency settings, malformed bindings and references, invalid retries, and
invalid pipeline, step, or lane structure. PromptKit local-backend validation
accepts only an absolute HTTP or HTTPS endpoint with a host and no user
information, query, or fragment, and rejects a negative local concurrency
limit. Its errors retain the closest known pipeline, lane, and binding context.
It deliberately does not require modules to be registered: that requires a
catalog and belongs to resolution.
The exact user-selectable values and validation rules are defined in
[Configuration](../config.md). Keep additions to the file model, an
environment override, its validation, and that reference in the same change.
## Effective Resolution
**Config.Resolve** first recomputes derived concurrency defaults and validates
the configuration. It normalizes the requested pipeline ID, copies the selected
profile, and passes the non-empty command-level LLM profile override, requested
lane selection, and reference changes to the framework resolver.
After module and validator selection, the resolver applies the effective
profile policy to LLM-backed bindings only: command override, binding profile,
pipeline profile, then the prompt default. Deterministic bindings remain
profile-free, and no second inheritance decision occurs during execution. The
public field definitions and precedence are owned by
[Configuration](../config.md#pipelines).
The framework resolver supplies defaults, selects lanes, resolves validator
chains, checks registered module and artifact compatibility, validates module
options, and returns the fixed ordered pipeline shape. The resulting
**EffectiveConfig** retains the selected ID, requested selection and reference
changes, a clone of the input configuration, and the resolved pipeline.
Callers may therefore retain or modify their input slices and maps without
changing the resolved result, and later consumers cannot mutate the original
configuration through the effective value. This ownership includes the nested
PromptKit local-backend value.
Resolution failures stop before module construction and source parsing. They
include an error path for an unconfigured pipeline, missing module, missing
capability, incompatible artifact variant, invalid option, invalid reference,
or invalid lane selection. CLI code maps these valid-invocation failures to the
runtime error class described in the [CLI reference](../cli.md#output-streams-and-exit-statuses).
## Resolved Identity And Redaction
The framework assigns the resolved pipeline a deterministic SHA-256 digest
after defaults, lane selection, module bindings, reference bindings, validator
chains, effective LLM profiles, and artifact schema identity have been
resolved. The digest excludes
its own stored value. It identifies resolved composition rather than raw YAML
bytes, a debug payload, or all runtime state. The CLI records it as invocation
provenance before execution; cache and checkpoint identity have additional
owners in [Run State Internals](state.md).
Configuration summaries must use **Redacted**, **RedactedSummaryPayload**, or
**RedactedResolvedPipelinePayload**, never a direct configuration marshal.
Those methods copy every binding and nested option container, replace values
whose key is credential-shaped with **[REDACTED]**, and omit materialized
reference content while retaining safe binding and reference provenance. The
payload must not alias the source configuration or resolved pipeline.
PromptKit's local endpoint and concurrency limit are preserved as non-secret
configuration metadata in the independently owned summary; the object contains
no credential value. This redaction is deliberately narrow: it protects
configuration summaries and does not authorize recording arbitrary environment
values or provider requests.
## Invariants To Preserve
- Defaults, YAML values, and environment values are applied in one direction;
later sources may override only their supported operational settings.
- A configuration is structurally valid before it is resolved, and a resolved
pipeline is compatible with the supplied catalog before preparation begins.
- Whitespace-normalized identifiers are unique wherever they identify a
pipeline, step, lane, worker, or reference slot.
- Resolution and summary generation return detached data. Redaction must cover
every configured and resolved binding, including nested validator bindings.
- The resolved digest changes when resolved composition changes and never
includes itself.
## Focused Tests
- **internal/core/config/file_config_contract_test.go** covers strict file
parsing, normalization, file application, and structural rejection.
- **internal/core/config/env_contract_test.go** covers supported operational
overrides and their precedence.
- **internal/core/config/validation_contract_test.go** covers configuration
invariants and contextual failures.
- **internal/core/config/effective_config_contract_test.go** covers defaults,
selections, overrides, resolution context, digest changes, and ownership.
- **internal/core/config/redaction_test.go** covers recursive credential
redaction, reference-content exclusion, and non-aliasing payloads.
Run **go test ./internal/core/config** after changing this boundary. Changes to
the handoff or resolved-composition semantics also need the focused framework
pipeline tests.

247
docs/internal/dnd.md Normal file
View File

@@ -0,0 +1,247 @@
# D&D Module Internals
This guide records the conventions shared by the production D&D module family.
It complements [Module Internals](modules.md), which owns generic registration
and extension mechanics, and [Configuration](../config.md), which owns the
selectable keys, bindings, reference syntax, and default validator chains.
## Durable Artifact Contracts
The ten lanes have separate durable wire contracts. This guide deliberately
does not repeat their JSON shapes or schemas.
| Lane | Durable contract |
| --- | --- |
| Spells | [spell artifacts](../integrations/dnd-spell-artifacts.md) |
| NPC registry | [NPC registry artifacts](../integrations/dnd-npc-registry-artifacts.md) |
| Combat turns | [combat-turn artifacts](../integrations/dnd-combat-turn-artifacts.md) |
| Item occurrences | [item-occurrence artifacts](../integrations/dnd-item-occurrence-artifacts.md) |
| Item registry | [item-registry artifacts](../integrations/dnd-item-registry-artifacts.md) |
| NPC occurrences | [NPC-occurrence artifacts](../integrations/dnd-npc-occurrence-artifacts.md) |
| Scene descriptions | [scene-description artifacts](../integrations/dnd-scene-description-artifacts.md) |
| Enemy events | [enemy-event artifacts](../integrations/dnd-enemy-event-artifacts.md) |
| Location registry | [location-registry artifacts](../integrations/dnd-location-registry-artifacts.md) |
| Location occurrences | [location-occurrence artifacts](../integrations/dnd-location-occurrence-artifacts.md) |
## Family Composition
The D&D registrar registers the familys artifact codecs, extractors, typed
append-order mergers, normalizers, validators, prompt assets, fallback LLM
profile asset, and default validator chains. Each extractor and normalizer has
a stable module spec, explicit execution class, strict option decoding, and a
typed builder. Scene chunking, every extractor, and NPC, location, and item-registry
normalization are registered as `llm_backed`; the remaining current D&D mergers
and normalizers are `deterministic`. The metadata is available to catalog inspection and
resolved-pipeline debug data and determines which selected bindings inherit the
pipeline profile. Configuration remains the canonical owner of the exact keys,
profile precedence, and validator order.
Private structured-LLM response schemas are deliberately minimal. They reject
invalid JSON structure, missing required fields, incompatible types, and
unknown fields, while preserving semantic candidates for deterministic
validation. Do not promote a private response envelope into a durable schema;
the contracts above define durable data.
## Prompt Construction
D&D LLM-facing content lives beneath `assets/dnd/`. Each module contributes a
local `prompt.yaml` declaration and `instructions.md`; input-specific files
such as a catalog, registry, grounding projection, or candidate collection are
local only when that module needs them. New extractor content uses its feature
subtree, while families with both extraction and normalization content use their
`extract` and `normalize` subtrees. Shared visual-provenance fragments use
the `common-dnd-` prefix. Production lane code belongs with its D&D codec,
extractor, normalizer, and validator packages; registry projections and
identity helpers remain in their owning entity packages rather than in a
consumer lane.
The owning modules manifest is the source of truth for which local and shared
assets are selected, their mount paths, their message order, cache controls,
and the files included in its prompt fingerprint. Shared fragments belong to
the D&D shared implementation and are selected by name rather than copied into
module directories. The root `assets` package is a content-only boundary; its
physical ownership and rationale are defined by
[ADR-0011](../adr/0011-centralize-llm-assets.md).
Put each rule at its narrowest owner:
- universal behavior belongs in the shared system asset;
- D&D-family behavior belongs in a selected `common-dnd-` asset;
- rules for an input projection belong with that input asset;
- lane-specific policy belongs in the modules `instructions.md`; and
- transport-envelope shape belongs in the private response schema.
A rule is eligible for the system prompt only when every D&D LLM prompt needs
it regardless of lane, inputs, or response shape. Module instructions must not
repeat rules selected from shared assets or schemas. Reintroduce such repetition
only after observational evaluation with representative transcripts shows that
it improves results at the intended target models and cost; structural prompt
tests alone are not that evidence.
Every maintained D&D LLM prompt selects `dnd-extraction` as its default
profile. The D&D registrar registers the fallback, while an operator can
replace it with a complete profile of the same ID from the configured PromptKit
source. Deployment profile selection is documented in
[Configuration](../config.md#promptkit-profiles).
The D&D transcript assets have distinct consumers. Scene chunking consumes the
complete-session `common-dnd-transcript-full.md`, while extraction prompts
consume the current-chunk `common-dnd-transcript-chunk.md`. NPC, location, and
item normalization instead mount the generic semantic-reconciliation
candidate and transcript-window presentation assets. Player, party, glossary,
and compatible campaign references provide disambiguating context only when
declared by the active prompt; they never establish evidence. Reference
material is canonically ordered before rendering so equivalent inputs remain
stable.
Extraction prompts render the common system and identity messages first, then
cached campaign references and the cached chunk transcript. Evidence policy and
any lane-specific registry, catalog, or grounding projection follow that
prefix. The final module instructions message is ephemeral. This keeps the
reusable extraction prefix identical while preserving the lane-specific suffix.
Scene chunking intentionally uses a different order: system, cached campaign
references, uncached module instructions, then the final ephemeral full
transcript. Entity normalization also has its own order: D&D system, mandatory
generic protocol, ephemeral domain semantic instructions, generic candidate
presentation, and final ephemeral generic transcript windows. These orders and
cache controls are prompt behavior; change them only through the owning
manifest and prompt declaration.
## Evidence, Candidates, And Normalization
The current transcript is the only durable evidence source. Extractors assign
the current source identity, preserve candidate evidence ranges for validators,
and canonically order or remove exact duplicate ranges without asking the
model to repair semantic errors. Campaign context and generated artifacts may
ground names or control routing, but they never establish evidence for a D&D
result.
Default chains keep responsibilities separate: structural validators assess the
candidate, source-reference validators resolve cited ranges against the current
source and require extraction evidence to stay within the current chunk,
durable-schema validation checks an approved representation, and
relatedness validators report advisory evidence concerns. The configured order
is documented in
[Configuration](../config.md#production-validator-keys-and-default-chains).
Enemy-event extraction additionally rejects a second `engaged` observation for
the same comparison identity within one scene-scoped result. Normalization may
combine results from distinct scenes, so it intentionally does not apply that
rule. Configuration owns the exact validator key and chain position.
Normalizers are deterministic for spells, combat turns, item occurrences, NPC
occurrences, scene descriptions, enemy events, and location occurrences. They
canonicalize display values and evidence, use source-document order for stable
output, and issue bounded warnings for changes or collapsed duplicates. NPC,
item, and location registry normalizers are intentional exceptions: each first
produces a deterministic candidate set, then may use a bounded structured-LLM
proposal to reconcile identity groups.
## Semantic Registry Reconciliation
The three registry normalizers instantiate the domain-neutral
`internal/framework/semanticreconcile` engine with default bounds. Each
eligible candidate receives a contiguous, one-based `candidate_id` for that
request. The model sees that handle, the candidate label and source-free
evidence ranges, plus bounded transcript windows; it returns only duplicate
groups of supplied handles and one supplied canonical handle per group. It
never returns names, evidence, durable IDs, or replacement records. Identical
labels and evidence remain independently selectable because their handles are
distinct.
The generic core owns the mandatory handle protocol, candidate and transcript
presentation, the private response schema, source-reference validation,
candidate and combined-material limits, structured completion, proposal
assessment, stable group ordering, and typed plan-application mechanics. The
D&D prompt contributes its system message and registry-specific semantic
instructions. The generic registrar registers the shared prompt and schema;
the D&D registrar registers each consuming prompt and the fallback profile.
Fewer than two eligible candidates skips the LLM without a semantic warning.
An exceeded bound also skips the call and preserves the deterministic
preprocessed registry, adding the registry's bounded fallback warning. Invalid
structured output or discarded proposal groups use the normalizer's existing
retry contract; retry exhaustion preserves the safe deterministic or
partially applied result and emits its bounded fallback warning. Provider,
transport, cancellation, and context-material failures remain execution
errors.
Application remains typed and registry-owned. All three policies select the
canonical member's normalized display name, union member evidence in source
order, preserve ungrouped records, and derive durable identity only after
consolidation. NPC IDs derive from the final name. Item IDs also derive from
the final name, and a typed guard prevents currency aliases from crossing
denominations or mixing currency with non-currency records. Location IDs
derive from the final name and final evidence, preserving same-name,
parent/child, and distinct physical-place identities. Registry warning scopes,
reason codes, and postconditions remain outside the generic core.
## Generated References And Grounding
Normalized D&D artifacts can be handed to a later step through a generated
reference binding. The framework verifies artifact compatibility and retains
producer provenance; consumers resolve the handed-off artifact into an
immutable, validated projection for each operation. External files are checked
during preparation, while generated artifacts are resolved at the handoff.
NPC and item registry consumers receive names-only grounding. Location
consumers receive a contextual selector containing the canonical name and the
registry references needed to distinguish same-name places. The calling module
resolves those supplied selections locally and maps them into the unchanged
durable ID/name pair; an unknown or ambiguous selection rejects the complete
occurrence result rather than accepting a partial mapping. The NPC registry
additionally supplies names-only actor grounding to spells, combat turns, and
enemy events.
Registry references establish a registry identity and may disambiguate a
selection, but never become occurrence evidence. Each occurrence keeps its own
current-transcript source references, even when it was grounded through the
same registry record.
Scene descriptions are eligibility-only projections: they retain current-chunk
classification data, not scene prose or evidence, and exist to route combat
extraction. Enemy-event extraction also projects combat turns to `actor` and
`turn_kind` and filters NPC occurrences to `combat_opponent` names and kinds.
These projections are guidance only and never event evidence.
## Lane-Specific Rules
The following differences are intentional and should remain explicit when a
shared helper changes.
Shared D&D text comparison is identified by `dnd.text_comparison.v1`. Any
semantic change requires an explicit policy-version review for every affected
identity, mapping, normalization, and validator policy; helper source is not a
checkpoint fingerprint.
| Lane | Intentional behavior |
| --- | --- |
| Spells | May use a spell-catalog overlay and optional NPC grounding; the catalog validator supplies domain-specific semantic checks. |
| NPC registry | Establishes transcript-grounded NPC identities, including factual third-party mentions, without assigning occurrence categories. It does not consume an NPC registry, and its normalizer is the LLM-assisted reconciliation exception described above. |
| Combat turns | Requires a scene-description artifact. It calls the LLM only for an exact `combat` classification; exact non-combat classifications return an accepted empty result, while missing or mismatched classifications return an empty result with a bounded warning. Optional NPC grounding never becomes evidence. |
| Item occurrences | Requires the normalized item registry for exact deterministic grounding at extraction and normalization. Campaign context may disambiguate, but the registry never becomes occurrence evidence. |
| Item registry | Produces source-grounded item types and unique designations. Its LLM-assisted reconciliation is proposal-only, preserves distinct currency denominations and item types, and does not create per-instance identities. |
| NPC occurrences | Requires the normalized NPC registry at extraction and normalization, using it for canonical actor grounding only. It separately emits cited current-transcript occurrence facts, including `mentioned`, rather than deriving them from registry provenance. |
| Scene descriptions | Produces the classifications consumed by combat routing; it does not consume an NPC registry or provide evidence for combat artifacts. |
| Enemy events | Requires NPC, scene-description, combat-turn, and NPC-occurrence artifacts. It calls the LLM only for an exact `combat` classification, records ordered observations rather than terminal state, and normalizes recognized names through the NPC registry while preserving grounded collective labels. |
| Location registry | Produces a source-anchored, session-scoped registry from stable proper names or unique in-world designations. Its LLM-assisted reconciliation is proposal-only and never collapses same-name places without validated identity and evidence rules. |
| Location occurrences | Requires the normalized location registry for both extraction and normalization. Its [durable occurrence categories](../integrations/dnd-location-occurrence-artifacts.md#occurrence-categories) distinguish explicit speculation from unsupported inference; the deterministic normalizer enforces exact registry grounding and never turns registry provenance into occurrence evidence. |
The combat and scene-description contracts describe their exact handoff and
empty-result behavior in more detail:
[combat turns](../integrations/dnd-combat-turn-artifacts.md) and
[scene descriptions](../integrations/dnd-scene-description-artifacts.md).
The [enemy-event contract](../integrations/dnd-enemy-event-artifacts.md)
defines its durable semantics; [Configuration](../config.md) owns its
selectable bindings and validation chains.
## Focused Verification
When changing D&D behavior, test the affected codec, extractor, normalizer,
validator, prompt-asset manifest, and registry projection. Also test generated
handoffs at the integration boundary and run the full D&D module suite:
~~~sh
go test ./internal/modules/dnd/...
go test ./internal/modules/integration/...
~~~

View File

@@ -1,154 +1,282 @@
# LLM Runtime Internals # LLM Runtime Internals
`internal/framework/llm` implements Notarius's transport boundary for structured `internal/framework/llm` is Notariuss provider-independent structured
completion. It contains the Scriptorium adapter, concurrency scheduler, completion boundary. It adapts framework requests to PromptKit, bounds
prompt/schema registries, selected-profile recording, and provider-error provider calls, assembles registered prompt and schema assets, records selected
redaction. profiles, and redacts provider errors. The architectural boundary is defined in
[Architecture](../policy/architecture.md#llm-boundary); profile sources,
credentials, and concurrency settings belong in
[Configuration](../config.md#promptkit-profiles) and
[Configuration](../config.md#concurrency-output-cache-and-debug).
Provider-neutral ownership rules are defined in ## Structured Completion Boundary
[Architecture](../policy/architecture.md#llm-boundary). Profile sources,
credentials, and concurrency settings are defined in
[Configuration](../config.md).
## Structured Contract Modules and LLM-backed validators depend only on
`contracts.StructuredLLMClient`. A completion request supplies a prompt ID and
version, optional profile and session IDs, named input material, variables, and
a caller-owned decode target. The successful response returns the validated raw
structured bytes together with non-secret provider, model, profile, and token
metadata.
Modules and LLM-backed validators depend on The caller owns the domain behavior: it chooses the prompt, prepares inputs,
`contracts.StructuredLLMClient.CompleteStructured`. A request identifies a selects the private response schema, and interprets the decoded result. The
prompt and optional profile/session, supplies named input materials and adapter does not own source evidence, artifact conversion, normalization, or
variables, and provides a caller-owned decoding target. A successful response durable schemas. Those responsibilities remain with the module and its
contains the validated raw structured bytes plus non-secret provider, model, [integration contract](../integrations/).
profile, and token metadata.
The caller owns prompt selection, response-schema selection, and interpretation The calling module also resolves contextual entity selections and attaches any
of the decoded result. `LLMInputMaterial` keeps source and reference bytes with application identity; PromptKit and this adapter do not own entity identity.
their origin metadata so the adapter can pass named artifacts to Scriptorium
without exposing Scriptorium types through stage contracts.
## Production Construction `PromptKitClient` validates the request target and prompt identity, maps each
named material to a PromptKit inline artifact while preserving its origin URI,
passes the supplied request session through to PromptKit's direct per-run
session field, retains the same value as the `session_id` prompt variable for
maintained prompt compatibility, and forwards profile selection. It does not
derive or replace session values; the CLI owns that policy. It then creates one
frozen prepared execution, captures its caller-owned credential-redacted
details for debug material, and executes that exact snapshot through
PromptKit's prepared-execution boundary. The direct field
is authoritative for provider session behavior. A session ID is a stable,
non-secret correlation identifier and may be exposed to providers and provider
observability. The adapter returns PromptKits validated raw bytes rather than
re-encoding the decoded target. An empty optional material is represented as
one space so its named input is retained by PromptKit.
`internal/cli` constructs the production runtime by: Client construction may also receive a run-wide reasoning-effort override from
the CLI factory boundary. The adapter copies the caller-owned pointer and
creates a fresh PromptKit execution override for each request: a nil pointer
inherits the selected profile, a non-empty value replaces it, and an empty
value clears inherited reasoning. The CLI's mutually exclusive
`--reasoning-effort` and `--clear-reasoning-effort` controls select those
states. With neither flag, profile behavior remains unchanged. Because
production constructs one shared client, the selected state applies uniformly
to module calls, retries, and LLM-backed validators for the whole run.
1. allocating the asset registry populated by the generic, Seriatim, and D&D An empty request profile lets the prompt select its configured default. Before a
package-family registrars; run begins, the CLI asks the adapter to inspect every explicit profile on the
2. creating a `ScriptoriumClient` from the effective profile source; resolved selected LLM-backed bindings and validators, including inherited
3. attaching an `LLMProfileRecorder`; pipeline profiles. Inspection resolves the profile and its selected backend and
4. creating a scheduler from the effective concurrency limit; target without loading a prompt, reading credentials, admitting capacity, or
5. returning a `ScheduledClient` wrapper; contacting a provider, so a missing or invalid explicit profile fails before
6. decorating that shared client before preparation when debug recording is stage execution while a valid `api_key_env` may remain unset. Calls record the
enabled; and profile actually selected by PromptKit. The recorder trims and deduplicates
7. injecting that one shared client into complete pipeline preparation before non-secret profile identity, provider, model, selected backend ID, and
the source file is read or the runner is invoked. effective reasoning values for manifest use. Entries that differ in backend or
reasoning remain distinct and deterministically ordered. Endpoint-only profiles
retain an empty backend ID, which the published JSON omits. Successful
completion responses and recorded profile manifests identify the adapter
provider as `promptkit`.
The D&D scene chunker and spell extractor retain this injected client and use The CLI's profile-inspection engine and the production adapter use the same
it for every structured completion. Operation requests do not carry an LLM profile-source construction to apply the configured profile directory or file,
client. the optional registered fallback profile assets, and the optional conventional
`local` backend. Preflight therefore resolves the same profile sources and
backend membership as runtime without performing generation. Fallback assets
are mounted only when at least one source is registered. The production D&D
registrar contributes its `dnd-extraction` fallback, and the maintained D&D
prompts select that logical ID by default. PromptKit owns source precedence and
profile parsing: an operator-provided matching profile takes precedence over a
fallback profile without Notarius merging either document.
When the registration is absent, a profile selecting `backend: local` fails
inspection instead of falling back to a built-in or endpoint-only target.
The CLI separately gathers explicit profile IDs from resolved LLM-capable stage Before execution, the adapter also contributes a non-secret checkpoint
and validator bindings. It prepares a small internal check prompt for each ID so fingerprint for the effective PromptKit profile source. It combines the
missing or invalid profiles fail before pipeline execution. The runtime profile identity of PromptKit's compiled-in profile catalog with a deterministic digest
override syntax and scope are defined in the of every YAML profile in the configured profile directory, or of the configured
[CLI reference](../cli.md#run); binding rules are defined in profile file, and a deterministic digest of the flattened fallback profile
[Configuration](../config.md#module-bindings). assets. The fingerprint contains neither profile content nor source paths. It
covers inherited pipeline profiles, explicit binding profiles, and
prompt-selected defaults, so changing a model or other profile setting cannot
reuse checkpoints created under the
prior profile source. This cache identity is independent of durable
profile provenance: run manifests continue to list only profiles actually
observed during LLM calls. When the local backend is registered, a second
fingerprint hashes its trimmed endpoint behind a stable marker. Changing that
semantic execution target invalidates checkpoint reuse. The raw endpoint is not
stored in checkpoint identity, and the local concurrency limit is excluded
because it changes scheduling rather than execution semantics.
## Scriptorium Adapter ## Shared Provider-Call Limit
`ScriptoriumClient` converts a Notarius request into a Scriptorium `RunRequest`. Production construction creates one PromptKit client and wraps it in one
It validates the decoding target and prompt identity, maps named input materials scheduled client. The scheduler has a fixed, positive permit limit, serves
to inline artifacts, forwards explicit profile and session context, delegates queued calls in FIFO order, and removes a queued call when its context is
rendering/provider execution/structured validation, and unmarshals successful cancelled. It rechecks the caller context after admission and before dispatch.
JSON into the caller target. A granted permit is released exactly once on every completion path.
Empty optional input material is represented by a single space so Scriptorium The scheduled wrapper surrounds every `CompleteStructured` call, so concurrent
retains the named input. The client returns Scriptorium's validated structured lanes, pipeline retries, and LLM-backed validators share the same provider-call
bytes rather than re-encoding the caller target, allowing modules to preserve ceiling. This ceiling is independent of pipeline worker concurrency; changing
the runtime result exactly. worker counts cannot exceed the configured LLM limit. The configuration field
and its effective default are owned by
[Configuration](../config.md#concurrency-output-cache-and-debug).
Selected profile, provider, model, and token metadata are mapped into the PromptKit applies a second, independent admission limit when the selected
Notarius response. The recorder deduplicates profiles by identity and supplies profile names a limited backend. It sits beneath the Notarius scheduled client,
manifest-safe profile summaries after actual calls; manifest population does so it may narrow but cannot expand the application-wide limit. Built-in
not guess the selected prompt default in advance. OpenRouter profiles select PromptKit's reserved backend and its upstream
capacity policy. A positive configured local-backend limit bounds active local
Generated-output validation failures and provider failures are wrapped with generations inside PromptKit; zero leaves that backend unlimited there.
prompt context. Error strings pass through bearer-token redaction before they Endpoint-only profiles do not select a PromptKit backend and remain limited
cross the runtime boundary. only by the Notarius scheduler.
## Scheduling
`Scheduler` uses a bounded permit count and a FIFO waiter queue. Immediate
acquisition increments the in-flight count; queued acquisition waits for a
permit or context cancellation. Cancellation removes a queued waiter, while a
cancelled waiter that has already received a permit releases it.
`ScheduledClient` acquires a permit around each structured completion and
defers release on every result path. The effective limit and default are
configuration facts in [Configuration](../config.md#defaults).
This provider-call ceiling is independent of the pipeline's extract worker
limit. Concurrent lanes, retries, and validators all use the same scheduled
client, so increasing framework workers cannot exceed `total_llm`. Pipeline
dispatch and cancellation mechanics are documented in
[Pipeline Internals](pipeline.md#execution-flow).
## Prompt And Schema Assets ## Prompt And Schema Assets
`AssetRegistry` combines caller-owned prompt filesystems under stable prefixes An `AssetRegistry` collects prompt, schema, and optional fallback-profile
and rejects invalid or conflicting registrations. Production module packages filesystems from production module families. It flattens registered roots into
register their own prompt and schema assets; generic framework code contains no the corresponding PromptKit filesystems and rejects invalid roots, unreadable
D&D prompt content. `internal/framework/promptfs` provides the domain-neutral assets, duplicate paths, and missing prompt or schema files during preparation.
filesystem composition helper used to combine module-owned files with shared Fallback assets receive a safe content digest for checkpoint identity; raw
domain prompt fragments. paths and bytes are never included. The frameworks `promptfs` helper combines
module-selected prompt files with reusable domain fragments without making the
framework depend on D&D content.
Schema helpers load embedded JSON Schema with identity and digest metadata, LLM-facing content is embedded once by the root `assets` package. Each consumer
return defensive copies, and expose a diagnostics map that omits schema bytes. uses only its scoped subtree, while the module retains ownership of its prompt
The small framework registry contains only generic test schemas; production declaration, ordered manifest, private response-schema identity, and
schemas remain package-owned. registration. Shared D&D fragments are selected by D&D's shared implementation;
the detailed convention is in [D&D Module Internals](dnd.md). This physical
arrangement and its data-only boundary are defined by
[Architecture](../policy/architecture.md) and
[ADR-0011](../adr/0011-centralize-llm-assets.md), rather than by this runtime
guide.
The spell extractor's package-owned prompt declares a required The generic registrar is the sole production registration owner for the
`application/json` `spell_catalog` input. The extractor generates that input semantic-reconciliation default prompt and private response schema. The
from its prepared effective catalog as `{"spell_names":[...]}` using sorted domain-neutral reconciliation package also exposes only its mandatory protocol
canonical names only. Its input digest covers those generated bytes; manifests and candidate/transcript presentation files for domain prompt manifests. D&D
record catalog identity and digest rather than names, aliases, overlay bytes, registry normalizers mount those files while retaining ownership and hashing
or source metadata. of their D&D system message, semantic instructions, and complete prompt
declaration. The response schema is therefore registered once even though
several typed normalizers select it.
## Debug And Redaction Boundaries Mounted prompt assets determine a module's fingerprint. The fingerprint hashes
only the module and shared files explicitly selected by its manifest, so an
unrelated asset does not invalidate a checkpoint. Schema loaders validate JSON,
attach identity and digest metadata, make defensive copies, and expose
diagnostics without raw schema bytes.
The pipeline may wrap the client with a debug recorder that captures prepared Semantic-reconciliation normalizers extend this identity with the shared
prompt/response material for an explicitly requested debug run. Debug summaries response-schema digest, framework policy version, and complete limit-policy
and manifests receive identities, hashes, usage, and selected profile summaries digest. Their manifest metadata records the same content-free prompt, schema,
rather than prompt, source, reference, schema, or response content. policy, and limit identities together with domain identity and normalization
policies. Request-local handles, source material, proposal content, and raw
asset bytes are not checkpoint metadata.
The Scriptorium error wrapper removes bearer credential values from surfaced Private response schemas validate a model transport envelope. They are not the
provider errors; `RedactSecrets` and `ErrorWithSecretsRedacted` support known durable artifact schema and should not be documented as an external wire
secret values elsewhere in the runtime. Config summaries use a separate contract. Durable formats and compatibility rules remain in the
clone-and-redact path in `internal/core/config`. These mechanisms implement the [integration contracts](../integrations/).
security invariant in
[Architecture](../policy/architecture.md#state-output-and-safety); operator
handling of debug data is defined in [Operations](../operations.md#debug).
## Failure Behavior ## Prompt Maintenance And Backend Caching
- Invalid targets, missing prompt IDs, malformed structured output, and Prompt message order and shared asset bytes are runtime behavior. Backend cache
Scriptorium failures return contextual errors to the calling module. reuse depends on identical preceding roles, rendered bytes, and cache-control
- Scheduler construction rejects non-positive limits; acquisition respects metadata—not merely equivalent meaning. Keep reusable shared assets
context cancellation. byte-identical and preserve each prompts declared ordering and cache controls
- Asset registration rejects invalid roots, missing content, and path conflicts. when editing it.
- Schema loading distinguishes missing assets, invalid JSON, and invalid
metadata.
- Profile validation errors occur during CLI preparation when an explicit
selected ID cannot be prepared.
## Tests To Inspect For sibling prompts that can reuse the same source material, order universal
shared context first, request source material next, and module-specific
suffixes last. Put a cache boundary at a reusable prefix that is useful to the
backend. Redundant intermediate cache boundaries do not extend that reusable
prefix and add no value.
- `internal/framework/llm/scriptorium_client_test.go`: adapter mapping and local Prompt-family owners may choose a different sequence when their inputs and
HTTP integration. reuse pattern differ. The D&D familys extraction, scene-chunking, and NPC
- `internal/framework/llm/scheduler_test.go` and normalization policies are maintained in [D&D Module Internals](dnd.md#prompt-construction).
`scheduled_client_test.go`: permits, FIFO behavior, cancellation, and wrapper Do not add tests that enforce prompt prose; prompt tests should verify the
release. meaningful input placement and cache controls of the prompt being changed.
- `internal/framework/llm/asset_registry_test.go` and
`schema_registry_test.go`: asset composition, validation, and defensive ## Validation, Repair, And Retries
copies.
- `internal/framework/llm/secrets_test.go`: provider-error redaction. PromptKit performs prompt rendering, provider execution, and the prompts
- `internal/cli/run_contract_test.go`: profile validation, production client structured-output validation. The adapter reports an empty result, validation
wiring, manifest recording, and debug integration. failure, empty structured body, or decode failure as
- Module-local `scriptorium_assets_test.go` files: prompt inputs and package `ErrInvalidStructuredOutput`, while retaining the returned raw bytes and debug
asset registration. material when they exist. Provider failures remain operational errors rather
than output-validation failures. Apart from documented context, capacity, and
invalid-output categories, provider error values and types do not cross the
adapter error chain; callers receive only a credential-redacted diagnostic.
When PromptKit rejects backend admission before generation, the adapter maps
`promptkit.ErrCapacityExceeded` to
`contracts.ErrLLMCapacityExceeded`, retaining prompt context and a redacted
upstream diagnostic without exposing the PromptKit sentinel or capacity-error
type as a framework contract. When supplied, the normalized selected backend
ID appears only in that safe application-owned diagnostic context. A canceled
caller context takes precedence. The adapter does not retry capacity failures;
the pipeline's existing binding attempt policy sees the operational error and
decides whether to rerun the complete operation.
Prompt-declared repair is executed within PromptKits structured-output flow.
The current production D&D prompt manifests set repair attempts to zero. That
setting does not replace pipeline retry behavior: a bindings configured retry
count reruns its stage attempt after an error or rejection, and an exhausted
rejection is a recorded output rather than a provider error. The pipeline owns
attempt lifecycle, validation chains, and retry diagnostics; see
[Pipeline Internals](pipeline.md#validation-retries-and-output) and the
[binding reference](../config.md#module-bindings-and-validators).
## Timeout Ownership
The caller context remains the outer cancellation authority. PromptKit applies
a positive effective generation timeout as an inner request deadline; an
explicit zero disables only that generation deadline. The HTTP client timeout
is a separate transport-wide cap. Notarius forwards the caller context and
does not install another timeout wrapper around PromptKit.
The selected PromptKit profile owns generation settings. Notarius binding
retries remain outside the adapter and repeat the complete module operation
and validation chain. PromptKit does not add a provider retry loop.
Operator-facing behavior is summarized in
[Operations](../operations.md#operational-limits), and the pinned upstream
contract is identified in
[PromptKit Integration](../integrations/pkg-promptkit.md).
## Observability And Redaction
When debug recording is enabled, the pipeline decorates the shared client. The
wrapper records prepared prompt and response material, timing, selected profile
and backend, effective model parameters, and call identifiers in the runs
debug bundle, including material available from a failed structured completion.
Effective parameters use PromptKit's stable lower-case JSON field names and may
include `backend_id`. For a successful completion, a debug-write failure is
surfaced; when the completion already failed, its call error remains the
result. Debug-bundle location, retention, and handling are operational concerns
documented in [Operations](../operations.md#debug-bundles).
Run manifests receive selected profile summaries, including optional effective
backend and reasoning provenance, and component identities—not prompt, schema,
source, reference, or response content. The published field semantics belong
to the [JSON output contract](../integrations/json-output.md#manifestjson).
Provider error text is wrapped with prompt context and bearer credentials are
redacted before it crosses the runtime boundary. Known-secret redaction is
available to other runtime collaborators; it does not make prompt or response
contents safe for general logging.
## Failure Boundaries
- Construction fails for missing asset registries, mutually exclusive profile
sources, invalid asset registration, or a non-positive scheduler limit.
- Preparation failures, unavailable explicit profiles, provider failures, and
context cancellation propagate to the calling stage with context.
- Backend admission exhaustion is a provider-neutral operational error and is
not classified as invalid structured output or validator rejection.
- Malformed or schema-invalid provider output is classified separately as
invalid structured output so the module or pipeline can apply its own retry
and rejection policy.
- Domain semantic checks, evidence decisions, and deterministic normalization
run outside the provider adapter.
## Focused Verification
Read the LLM adapter, scheduler, asset registry, schema loader, and redaction
tests when changing this boundary. Prompt changes also require the owning
modules asset tests, and retry or debug changes require focused pipeline or
CLI coverage. The focused runtime and D&D checks are:
~~~sh
go test ./internal/framework/llm/... ./internal/modules/dnd/...
~~~

View File

@@ -1,318 +1,152 @@
# Module And Validator Internals # Module Internals
Production module and validator implementations live under their domain-first This guide owns the mechanics for implementing and registering production
trees in `internal/modules`. modules. [Configuration](../config.md) owns selectable keys, binding syntax,
The selectable keys, configuration options, reference slots, and default reference configuration, and default validator chains. Durable input and output
validator chain are canonical in the shapes belong in [integration contracts](../integrations/).
[module](../config.md#implemented-production-modules) and
[validator](../config.md#implemented-production-validators) catalogs in
Configuration.
## Extension Pattern The D&D family has additional shared conventions and domain-specific
exceptions. See [D&D Module Internals](dnd.md) rather than adding them here.
A stage module package provides a stable key, constructor, contract ## Module Boundary
implementation, `ModuleSpec`, `Register`, and focused behavior and registration
tests. A validator package follows the same pattern with `ValidatorSpec` and the
validator registry. Package-family registrars compose those leaf registrations
into the production catalog and own family-level policy such as default
validator chains and prompt asset collection.
Production input, chunk, output, and D&D spell-extract packages register strict A module is a typed implementation registered for one pipeline stage. Its
option decoders and run-local builders. Preparation decodes their options into `ModuleSpec` is the public-to-the-framework declaration of its stable key,
implementation-owned values and injects dependencies plus the materialized stage, execution class, required and provided capabilities, artifact kind, and
reference set for the selected target. Each builder receives an isolated clone accepted reference slots. The execution class states whether a module is
of that set; input and output builders receive no references. The spell `deterministic` or `llm_backed`; registries retain it for catalog inspection and
extractor is typed over the canonical D&D model. D&D validators, merge, and resolved-pipeline debug data without constructing the module. The framework
normalize use typed variants; JSON representation validators use serialized uses the declaration to resolve a configured binding before it builds the
requests; and unconditional validators expose separate chunk and typed implementation. After selection, the resolver applies profile inheritance only
variants. The D&D production registrar registers only the canonical typed spell to bindings whose declared execution class is `llm_backed` and rejects a
implementations. binding-specific profile on a deterministic module. The user-facing precedence
contract belongs in [Configuration](../config.md#pipelines).
Prepared extractors, extract validators, and codecs may be reused concurrently Implementations that accept options must provide both an option validator and
by the run-wide extract pool. Production implementations are immutable after a builder. The validator is used while resolving configuration; the builder
construction: they retain only typed options, immutable assets, or the shared decodes the same options and constructs the implementation from the prepared
concurrency-safe LLM client. Implementations that introduce mutable state must `BuildRequest`. Reject unknown options in both paths. A builder receives only
synchronize that state without creating a separate provider scheduler. the dependencies and materialized references that the framework prepared for
that operation, so it must not re-read configuration or files.
Specs expose capability and execution metadata without constructing an Registry helpers register the typed builder for a stage-specific registry.
implementation. Registry entries separately expose option validation and They are preferable to hand-written untyped registration because they retain
run-local construction. Chunk, extract, merge, and normalize modules that accept the artifact type at the framework boundary. Registrars validate the registries
auxiliary material declare identical reference slots from both they need, register each leaf implementation, and add any family-owned assets
`ReferenceSlots()` and `ModuleSpec().ReferenceSlots`; registration tests enforce or default validator chains. They return contextual errors so production
that agreement. Runtime delivery uses the corresponding stage request's composition fails at startup rather than at the first run.
`References` field.
LLM-backed extensions own their prompt definitions and response schemas under An artifact family can register an optional typed evidence projector alongside
package-local embedded assets. Shared filesystem composition belongs in its codec. The projector returns defensive copies of the artifact's direct
`internal/framework/promptfs`; reusable D&D prompt fragments, reference generic source references and must use the codec's exact Go type. It does not
declarations, prompt-input assembly, and source-unit helpers belong in interpret surrounding context or publish files; the pipeline validates the
`internal/modules/dnd/shared`. Stage contracts expose only Notarius structured- capability during preparation and the output boundary owns publication. See
completion types, not Scriptorium public types. the [Published Evidence Context contract](../integrations/evidence-context.md)
for the durable source-unit excerpt. Lane artifacts retain citation and lane
provenance; the framework does not add either to that published excerpt.
Reference material may inform a module or prompt but must not become source An artifact family is broader than a module: it owns the cohesive domain
evidence. The resolver and materializer behavior is described in feature across its artifact type, codec, stage modules, validators, prompt
[Pipeline Internals](pipeline.md#reference-materialization). policy, schemas, identity helpers, and reference projections. An extractor and
normalizer in one artifact family remain independently registered modules in
their respective pipeline stages. This ownership vocabulary does not create a
new registry or change the fixed pipeline.
## Domain Reference Data ## Production Composition
### `internal/modules/dnd/spells/catalog` Production composition is intentionally split by family:
The spell catalog package owns the embedded, versioned D&D 5e 2014 SRD spell - The generic registrar provides the unit chunker, generic JSON validators,
reference data. Its strict JSON asset contains one canonical record per spell, JSON output encoder, and shared semantic-reconciliation prompt and response
including spell level and all applicable class memberships. `LoadSRD5E2014` schema assets.
validates catalog identity, provenance metadata, ordering, uniqueness, levels, - The Seriatim registrar provides the transcript input adapter. Its external
classes, aliases, and lookup-key collisions before exposing immutable copies. input behavior is defined by the [Seriatim contract](../integrations/seriatim.md).
- The D&D registrar provides its codecs, extractors, mergers, normalizers,
validators, prompt assets, fallback profile asset, and default chains. Its behavioral conventions
are documented in [D&D Module Internals](dnd.md).
Lookup is case-insensitive and normalizes whitespace and common apostrophe The CLI owns the composition that invokes these registrars. A module package
variants while preserving source punctuation in canonical display names. The may register its own family but must not assemble the CLI or make framework
catalog contains 319 unique spells and 779 class memberships. Source and packages depend on production extensions.
license details live beside the asset in `SOURCES.md`. This domain-owned data is
separate from `internal/modules/dnd/shared`, which is reserved for reusable
prompt and source-reference machinery.
`ResolveEffectiveCatalog` builds the immutable recognition view used by the ## Semantic Reconciliation
spell extractor and catalog validator. It starts with the embedded SRD catalog
and optionally applies one strict JSON overlay from the `spell_catalog` item in
a materialized reference set. Overlay catalogs are ordered by ID, may add names
and aliases, and may augment an existing canonical spell without replacing its
display name. Cross-spell lookup collisions are errors. The effective view
exposes sorted canonical names, normalized lookup, overlay identities, and a
semantic digest; overlay content remains contextual reference material rather
than source evidence. Its external JSON contract is defined in the
[spell-catalog overlay contract](../integrations/dnd-spell-catalog-overlays.md).
## Input Adapter `internal/framework/semanticreconcile` is a domain-neutral strategy used by a
typed normalize module; it is not itself a selectable stage module. A
source-backed artifact-family normalizer projects its deterministic records
into contextual candidates and owned typed record envelopes, supplies its
chosen prompt identity and resolved LLM profile, and constructs an engine with
explicit limits. The core filters invalid evidence, assigns contiguous
request-local integer handles, renders bounded candidate and transcript
materials, invokes the structured-completion boundary, and assesses the
returned duplicate groups into a stable non-overlapping plan.
### `internal/modules/seriatim/input/transcript` The normalizer then applies that plan through a typed `ApplicationPolicy`. The
core preserves ungrouped records, contribution order, and provenance while the
artifact family owns group guards, field and evidence consolidation, durable
ID derivation, retry and fallback presentation, warnings, and postconditions.
Request-local handles do not enter the typed value or durable artifact. Fewer
than two eligible candidates skips model invocation; exceeding a candidate or
combined-material bound preserves the deterministic result under the family's
fallback policy. Provider, transport, cancellation, and context-construction
failures remain execution errors.
The adapter decodes the supported transcript JSON, selects the source identity, The core supplies a conservative generic prompt and the single private
computes canonical source provenance, validates segments, and maps each segment response schema. A domain prompt may substitute its semantic instructions but
into a generic source unit with a self-reference plus speaker and timestamp mounts the core-owned protocol and candidate/transcript presentation assets.
metadata. It accepts no module options. Its spec advertises the transcript Prompt, schema, policy, and limit identities participate in manifest metadata
capabilities consumed by D&D modules. and checkpoint fingerprints. The generic registrar owns production
registration of those shared assets; a consuming domain registrar owns only
its domain prompt.
Parsing is strict about required values and duplicate unit IDs but deliberately ## Adding Or Changing A Module
ignores unrelated Seriatim fields. The external format and derived-identity
rules are defined in the
[Seriatim contract](../integrations/seriatim.md).
## Chunkers 1. Choose the pipeline stage and the typed artifact boundary. Put external
input or durable artifact formats in the relevant integration contract,
not in this guide or in a private LLM response type.
2. Define a stable `ModuleSpec` with an explicit execution class, the exact
capabilities, and reference slots needed for the operation. Model a
producer/consumer handoff as an artifact-compatible slot; configuration
then chooses an external file or a generated binding.
3. Implement strict option decoding, construction, and the typed stage
interface. Preserve caller ownership: do not retain mutable request data
and return defensive copies where an implementation exposes stored data.
4. Register the module through its typed registry helper and add it to the
owning family registrar. Add a default validator chain only when that
family owns the behavior; otherwise require an explicit compatible chain.
5. Update the selectable-key and chain reference in
[Configuration](../config.md#production-module-keys), the applicable
integration contract, and focused tests. Keep the configuration document
as the sole list of production keys and validator order.
Chunkers implement `contracts.Chunker.Plan`. A plan identifies ordered source ## Validation And References
unit ranges and may carry optional namespaced JSON annotations; it does not
contain materialized chunk content. The framework canonicalizes annotations,
validates ranges against the current source, and materializes chunk IDs,
indexes, references, content, units, and generic metadata. Materialized source
unit metadata is independently owned. Annotation
namespaces remain optional data: generic framework code and downstream modules
must not require D&D scene annotations or import `dnd/scenes`.
### `internal/modules/generic/chunk/units` Validators operate on the value produced at their configured stage. A default
chain is ordered behavior, not a set: JSON parsing, structural checks,
domain-specific checks, durable-schema checks, and advisory checks may have
different responsibilities and failure handling. The active default chains and
override rules are maintained in
[Configuration](../config.md#production-validator-keys-and-default-chains).
The generic chunker validates the source document and returns ranges over units Reference slots are part of the module specification. They describe the
in configured windows. Overlap changes the next window start but never reorders accepted artifact kind, media type, size, and whether a binding is required;
units. Framework materialization derives the resulting chunk identity and the framework validates those constraints before construction. An external
generic metadata from those ranges. reference is materialized during preparation. A generated reference is a
compatible normalized artifact handed from an earlier pipeline step at
operation time. The configuration reference rules, including precedence and
ordered-handoff requirements, are maintained in
[Configuration](../config.md#references-and-ordered-handoffs).
The accepted options and defaults are defined in ## Focused Verification
[Configuration](../config.md#implemented-production-modules). Generic
framework validation canonicalizes the returned unit slices before extraction.
The chunker decodes its options during construction and retains only the typed
window settings used by `Plan`.
### `internal/modules/dnd/chunk/scenes` Exercise the leaf implementation and its registration path when changing a
module. Registry and registrar tests cover duplicate keys, required registries,
and typed construction; pipeline resolution tests cover capabilities, options,
and reference compatibility. Domain packages should additionally test their
codecs, validators, normalizers, and any integration handoffs they own.
The scene chunker prepares a structured Scriptorium request from the full Run the affected package tests while iterating. The complete module suite is:
transcript, session, and optional D&D reference inputs. It validates the model's
scene boundaries against source-unit IDs and converts them into deterministic
plan ranges with optional scene annotations. Preparation injects the shared
structured LLM client into the chunker; `Plan`
supplies only the run-specific profile, session, source, references, and
metadata.
Scene validation requires sequential, contiguous, non-overlapping coverage from ~~~sh
the first source unit through the last. Scene descriptions, boundaries, go test ./internal/modules/...
confidence, and participants are module-owned annotations. Boundary caveats ~~~
become warnings. Malformed
structured output is returned as an error; there is no fallback chunker.
The package embeds its prompt and response schema and reports their non-secret
identity and hashes through singleton module metadata. Shared D&D assets supply
reference declarations and prompt inputs; their user-facing keys and accepted
file types remain canonical in [Configuration](../config.md).
## Extractor
### `internal/modules/dnd/extract/spells`
The spell extractor prepares a structured request from one chunk, the
chunk-scoped source input, the session, and optional D&D reference inputs. It
decodes the model response, assigns the generic source identity to every source
reference, canonicalizes duplicate references, orders spell casts by their
earliest cited unit, and returns `dnd.SpellList`.
The extractor owns its private model-response DTO, embedded prompt, LLM response
schema, strict option decoder, injected shared LLM client, and prompt/schema
manifest metadata. During preparation it resolves the optional `spell_catalog`
reference into an immutable effective catalog and adds a generated
canonical-name-only JSON input to every structured completion request. Overlay
failures therefore stop construction before source parsing or an LLM call;
campaign references remain separate disambiguation inputs and never become
source evidence. Both the extractor and deterministic catalog validator expose
the effective base-plus-overlay semantic digest as scoped prepared-component
checkpoint identity. Raw overlay provenance independently covers file-byte
changes, while the semantic digest also invalidates reuse when the embedded
catalog or catalog composition changes. The separate
`internal/modules/dnd/codec/spells` package
owns the durable schema and stable JSON representation for artifact kind
`dnd/spell-list`. The runner keeps the result typed through validators and later
stages, using the codec only for checkpoint, debug, and output boundaries.
Shared D&D helpers keep prompt input
names and source-unit reference conversion consistent with the scene chunker.
The durable payload and manifest metadata shapes are defined in the
[D&D spell artifact contract](../integrations/dnd-spell-artifacts.md).
## Merger And Normalizer
### `internal/modules/generic/merge/appendorder`
The merger passes typed values to an injected combine function in framework
source-chunk order. The D&D registrar specializes it with a spell-list append
function.
### `internal/modules/generic/normalize/noop`
The normalizer returns the merged domain value unchanged and is reusable for
any registered artifact type.
### `internal/modules/dnd/normalize/spells`
The typed spell normalizer resolves the optional `spell_catalog` reference into
the same immutable SRD-plus-overlay effective catalog used by spell extraction
and catalog validation. It performs no LLM calls. For each spell cast it
canonicalizes recognized names using the catalog's case, whitespace,
apostrophe, and alias rules; sorts source references by source identity and
unit boundaries; removes only exact reference duplicates; and emits bounded,
scoped warnings for each mutation or unresolved name.
After those per-cast changes, it collapses only casts with the same canonical
spell, case-folded and whitespace-normalized caster, and complete non-empty
valid source-reference set. It retains the first occurrence and its caster,
effect, narrative description, and stable order. Unknown names, empty or
invalid evidence, and adjacent or overlapping but different ranges remain
unchanged for validation.
The normalizer exposes the effective catalog digest as its independently scoped
`effective_catalog` checkpoint fingerprint and reports catalog base ID, digest,
and overlay IDs as manifest metadata. Catalog contents, reference paths, and
raw overlay bytes are not included in either surface. The normalize-stage
reference is stage-local, so an overlay-capable pipeline binds the catalog
independently for extraction and normalization.
## Output Encoder
### `internal/modules/generic/output/json`
The JSON encoder sorts normalized results by lane, derives collision-checked
safe logical names, pretty-prints JSON payloads, and assembles the logical index,
manifest, rejected-result, warning, and lane files. Invalid JSON, unsupported
media types, unsafe names, and sanitized-name collisions are errors.
The encoder returns logical files only. The CLI places them on disk, and the
[JSON output contract](../integrations/json-output.md) defines their external
paths and schemas.
## Generic Validators
The generic validator implementations live under
`internal/modules/generic/validate`.
The unconditional accept and reject validators provide explicit chunk and
typed-artifact variants used primarily for controlled composition and tests.
The serialized JSON syntax validator uses `encoding/json` to reject malformed
representation bytes. The serialized JSON Schema validator requires schema
bytes, parses the instance and schema with `jsonschema`, and distinguishes
payload rejection from schema loading or compilation errors. The framework
serialized-validation request carries either canonical chunk bytes or artifact
codec bytes according to its target context. Neither validator calls the LLM.
## D&D Spell Validators
All four validators receive `dnd.SpellList` directly. The shape validator
rejects missing or empty spell fields and empty reference lists. The catalog
validator defers when shape is invalid, then checks every non-empty spell name
against the immutable effective SRD and overlay catalog. It accepts normalized
canonical names and aliases without rewriting the artifact; unknown names
reject the complete result with bounded, stable index/name diagnostics. The
source-reference validator applies generic source-reference validation to every
cited range. The relatedness validator warns when a case-insensitive spell name
is absent from all cited source text.
These validators are deterministic. Their selectable keys and production order
are defined in
[Configuration](../config.md#implemented-production-validators); their durable
payload rules are defined in the
[artifact contract](../integrations/dnd-spell-artifacts.md).
## Production Registration
Production composition occurs through family registrars. The CLI allocates one
complete framework registry set and one LLM asset registry. It invokes
`internal/modules/generic/register`,
`internal/modules/seriatim/register`, and `internal/modules/dnd/register` in
that order, then exposes the matching catalog for resolution. The generic and
Seriatim registrars own their production leaf registrations. The D&D registrar
owns D&D leaf registrations, the typed spell default-validator chains, and D&D
prompt/schema asset collection.
Concrete implementation packages do not import generic implementation
packages directly. A concrete family's `register` package is its composition
point for specializing reusable generic implementations, while the generic
registrar composes only generic children.
Core and framework production packages do not import production extensions.
CLI production code is the sole application composition root for extensions
and imports only exact family registrar packages. Other production packages,
including commands and newly introduced package trees, do not import module
packages directly. Compatibility tests in the CLI, core, and framework trees
may import roots and implementation leaves directly. Other non-module tests do
not receive that exemption. White-box tests within module families retain the
production family boundaries. `internal/modules/integration` is test
infrastructure: its black-box tests may compose multiple families, but it is
not a production module family or production dependency target.
## Adding An Extension
When adding a production module or validator:
1. implement the stage or validator contract and package-local key;
2. expose and test its spec, constructor, and registration function;
3. keep format or domain parsing inside the concrete package;
4. add package-owned prompt/schema assets when the extension is LLM-backed;
5. register it through its package-family registrar and add a default chain
there only when production policy requires one;
6. add resolution and composition coverage for capabilities, options,
references, and validation behavior;
7. update the selectable-key catalog in [Configuration](../config.md), the
relevant external contract, this inventory, and maintained examples when
user-visible behavior changes.
Do not add the extension to `docs/development.md`; that file routes by task and
does not inventory implementations.
## Tests To Inspect
- Package-local `*_test.go` files under the module or validator being changed.
- `internal/framework/pipeline/typed_resolution_test.go`: typed registry, spec,
and heterogeneous artifact composition.
- `internal/framework/pipeline/profile_test.go`: framework binding defaults and
profile resolution.
- `internal/cli/production_contract_test.go`: production catalog, config
resolution, and composition smoke coverage.
- `internal/cli/example_contract_test.go`: maintained example ownership.
- `internal/framework/promptfs/*_test.go` and
`internal/modules/dnd/shared/*_test.go`: shared prompt and reference assembly.
- `internal/modules/integration/*_test.go`: black-box composition across
production extension domains.

View File

@@ -1,144 +1,61 @@
# Internal Overview # Internal Overview
This document inventories the implemented Notarius components. Normative This document is the implemented component map for Notarius. Normative
boundaries and dependency direction belong in boundaries and dependency direction belong in
[Architecture](../policy/architecture.md); external behavior belongs in the [Architecture](../policy/architecture.md). User and operator contracts belong
[CLI](../cli.md), [Configuration](../config.md), in the [CLI](../cli.md), [Configuration](../config.md),
[Operations](../operations.md), and [integration contracts](../integrations/). [Operations](../operations.md), and [integration contracts](../integrations/).
## Execution Path ## Execution Path
`cmd/notarius` delegates to `internal/cli`, the production composition root. ~~~
The CLI loads configuration, builds the production catalogs and runtime cmd/notarius -> internal/cli -> configuration and production composition
collaborators, invokes `internal/framework/pipeline`, and places the logical -> internal/framework/pipeline -> logical output files
output files returned by the runner. Cache and debug collaborators are supplied -> internal/cli -> durable output and optional state/debug data
at this boundary. ~~~
Resolution produces a fixed ordered workflow and a sorted set of artifact The CLI is the application boundary: it discovers configuration, composes
lanes. Preparation constructs the complete module and validator set before the production registries and runtime collaborators, invokes the framework, and
runner receives source bytes. Source parsing and chunking are serial; extraction places returned files. The framework resolves and prepares a fixed extraction
uses a bounded run-wide worker pool, followed by serial per-lane merge and pipeline, then returns logical results without owning process behavior or
normalize continuations that may overlap across lanes. physical state roots.
## Application Boundary ## Components
| Package | Implemented responsibility | | Area | Implemented owners | Responsibility |
| --- | --- |
| `cmd/notarius` | Executable entry point and process exit delegation. |
| `internal/cli` | Command parsing, config discovery, package-family registrar invocation, LLM client construction, reference materialization, state collaborator setup, durable writes, and user-facing results. |
## Core Packages
| Package | Implemented responsibility |
| --- | --- |
| `internal/core/artifacts` | Run-manifest and provenance models. |
| `internal/core/config` | Defaults, YAML parsing, environment overrides, validation, redaction, and effective pipeline resolution. |
| `internal/core/debugbundle` | Explicit per-run debug-bundle allocation and redacted summary writing. |
| `internal/core/fileio` | Generic confined atomic file and JSON writes with caller-selected permissions. |
| `internal/core/source` | Generic source documents, units, chunks, canonical references, validation, deterministic source digests, and independent metadata materialization. |
## Framework Packages
| Package | Implemented responsibility |
| --- | --- |
| `internal/framework/contracts` | Source-stage contracts plus artifact identity, schema, serialized representation, codec, validator, reference, output, and structured-completion interfaces and data types. |
| `internal/framework/pipeline` | Module and artifact-codec registries, option validation, profile resolution, capability checks, reference materialization, complete pipeline preparation, retries, orchestration, warnings, and manifest population. |
| `internal/framework/validate` | Shared validator decision and cardinality helpers. |
| `internal/framework/llm` | Scriptorium-backed structured completions, prompt/schema registration, scheduling, profile recording, and secret redaction. |
| `internal/framework/promptfs` | Builds module prompt filesystems from module-owned and caller-provided shared prompt assets. |
| `internal/framework/checkpoint` | Root-based checkpoint loading, recording, identity, and payload serialization. |
| `internal/framework/chunkplan` | Source-addressed chunk-plan filesystem storage, envelope validation, and atomic publication. |
| `internal/framework/debug` | Root-based framework and LLM debug recording. |
Framework contracts provide typed artifact, provenance-wrapper, chunk-validator,
serialized-validator, and
typed-validator interfaces. The runner owns handoff provenance, validation
sequencing, rejection handling, checkpoint and debug boundaries, and final
manifest assembly.
Artifact registries support heterogeneous typed extraction entries and
kind-specific merger, normalizer, and validator variants. Resolution derives a
lane's kind from its extractor, requires the matching codec, verifies exact Go
type equality across the lane, and records schema identity in the resolved lane
and pipeline digest. Registry entries carry separate option-validation and
run-local construction closures. Preparation injects shared dependencies and
constructs input, chunk, validators, ordered lanes, and output before source
parsing. Production modules use strict construction-time option decoding, and
LLM-backed modules retain the injected shared client. The D&D family registers
the canonical `dnd/spell-list` codec, typed spell extractor, normalizer, and
validators, plus kind-specific generic merge strategies; generic JSON validators
use the serialized-validation contract. The runner executes lanes through
private exact-type-checked closures, coordinates extract results independently
of completion timing, and serializes artifacts only through their codec at
checkpoint, debug, and output boundaries.
## Production Extensions
The canonical catalogs of user-selectable
[module](../config.md#implemented-production-modules) and
[validator](../config.md#implemented-production-validators) keys are in
Configuration. The implemented module packages are:
| Package | Implemented responsibility |
| --- | --- |
| `internal/modules/seriatim/input/transcript` | Parses the supported Seriatim transcript format into the generic source model. |
| `internal/modules/generic/chunk/units` | Splits ordered source units by unit count and overlap. |
| `internal/modules/dnd/chunk/scenes` | Produces contiguous D&D scene chunks from structured model output. |
| `internal/modules/dnd` | Owns the canonical D&D spell-list and spell-cast artifact types. |
| `internal/modules/dnd/codec/spells` | Strictly decodes and stably encodes the durable D&D spell-list representation. |
| `internal/modules/dnd/extract/spells` | Maps private structured model output to canonical source-grounded D&D spell lists. |
| `internal/modules/dnd/spells/catalog` | Embeds and validates the versioned D&D 5e 2014 SRD catalog, composes optional overlays, and provides immutable effective lookup. |
| `internal/modules/generic/merge/appendorder` | Combines accepted extraction results in chunk order. |
| `internal/modules/generic/normalize/noop` | Preserves accepted merged output. |
| `internal/modules/dnd/normalize/spells` | Canonicalizes catalog-backed spell names and exact source references, conservatively collapses duplicate casts, and reports deterministic warnings and independently scoped catalog checkpoint identity. |
| `internal/modules/generic/output/json` | Encodes manifests, lane payloads, warnings, and rejections as logical JSON files. |
`internal/modules/dnd/shared` owns reusable D&D prompt fragments,
reference declarations, prompt input assembly, and source-unit reference
helpers. Domain-neutral prompt filesystem composition lives in
`internal/framework/promptfs`.
Generic validators under `internal/modules/generic/validate` provide
unconditional test decisions, JSON syntax validation, and JSON Schema
validation. D&D spell validators under `internal/modules/dnd/validate/spells`
consume the canonical spell-list type directly to provide shape,
effective-catalog, source-reference, and source-relatedness decisions.
Production composition is grouped behind package-family registrars, and every
implemented production extension uses its domain-first tree:
| Package | Implemented responsibility |
| --- | --- |
| `internal/modules/generic/register` | Registers domain-neutral chunk, merge, normalize, output, and validator implementations. |
| `internal/modules/seriatim/register` | Registers the Seriatim input adapter. |
| `internal/modules/dnd/register` | Registers D&D modules, validators, default validator policy, and prompt/schema assets. |
The CLI allocates the framework registries and asset registry, then invokes
these registrars in generic, Seriatim, and D&D order.
Implementation details for all production extensions are in
[Module Internals](modules.md).
## Run-State Components
| Surface | Implemented owners | Internal purpose |
| --- | --- | --- | | --- | --- | --- |
| Durable output | Output module, pipeline runner, and CLI writer | Return logical consumer files and place them for a run. | | Executable and command boundary | **cmd/notarius**, **internal/cli** | Process entry, command dispatch, configuration discovery, production composition, runtime collaborator setup, durable file placement, and user-facing reporting. |
| Cache checkpoints | `internal/framework/checkpoint` and `internal/cli` | Validate and serialize reusable extract, merge, and normalize outcomes. | | Configuration | **internal/core/config** | Defaults, strict YAML parsing, environment overrides, structural validation, effective resolution, redaction, and resolved-composition summaries. |
| Chunk-plan cache | `internal/framework/chunkplan` and `internal/cli` | Persist and select source-addressed plans before framework materialization. | | Generic models | **internal/core/source**, **internal/core/artifacts**, **internal/framework/contracts** | Source documents and chunks, manifests and provenance, plus typed artifact, reference, validation, output, and structured-completion contracts. |
| Debug bundles | `internal/core/debugbundle`, `internal/framework/debug`, and pipeline instrumentation | Persist redacted summaries and application-owned traces. | | Pipeline framework | **internal/framework/pipeline** | Registries, profile and reference resolution, typed preparation, validation, retry coordination, ordered execution, handoff, and result assembly. |
| LLM and prompt runtime | **internal/framework/llm**, **internal/framework/promptfs** | Provider-neutral structured completions, scheduling, profile recording, prompt assets, schema registration, and credential-shaped-value redaction. |
| Semantic reconciliation | **internal/framework/semanticreconcile** | Bounded source-backed candidate preparation, request-local handle proposals, deterministic assessment, typed plan application, and reconciliation identity metadata; see [Module Internals](modules.md#semantic-reconciliation) and [D&D Module Internals](dnd.md#semantic-registry-reconciliation). |
| Embedded LLM content | **assets** | Read-only centralized LLM-facing content, scoped by its consuming package; see [LLM Runtime](llm.md#prompt-and-schema-assets) and [D&D Module Internals](dnd.md#prompt-construction). |
| Runtime state | **internal/core/fileio**, **internal/core/debugbundle**, **internal/framework/checkpoint**, **internal/framework/chunkplan**, **internal/framework/chunkmap**, **internal/framework/debug** | Confined atomic files, debug bundles, checkpoint and chunk-plan state, accepted chunk maps, and pipeline-facing debug recording. |
| Production extensions | **internal/modules/generic**, **internal/modules/seriatim**, **internal/modules/dnd** | Domain-neutral extensions, Seriatim input support, and D&D extraction families registered into the production catalog. |
Physical layout, cleanup, recovery, and sensitive-data handling are defined Generic core and framework packages do not depend on production extensions.
in [Operations](../operations.md). Concrete stage modules receive recorder Concrete extensions depend inward on their contracts and are registered only at
interfaces and request data, not physical state roots. the CLI composition boundary.
## Focused Documentation ## Focused Documentation
- [Pipeline Internals](pipeline.md): resolution, execution, validation, retries, - [Configuration Internals](configuration.md): loading, validation, effective
checkpoint/debug hooks, and result assembly. resolution, redaction, and resolved-composition identity.
- [Module Internals](modules.md): production modules, validators, assets, - [CLI Internals](cli.md): command dispatch, production composition, run
registration, and the contributor recipe for adding an extension. orchestration, and terminal reporting.
- [LLM Runtime](llm.md): structured completion contracts, Scriptorium adapter, - [Pipeline Internals](pipeline.md): resolution, preparation, execution,
assets, scheduling, profile recording, and redaction. validation, typed handoff, and framework state hooks.
- [Run State Internals](state.md): output, cache, debug collaborator - [Run State Internals](state.md): output, cache, debug collaborator
composition, and path safety. composition, and path safety.
- [LLM Runtime](llm.md): structured completion, scheduling, prompt assets,
profiles, and secret handling.
- [Module Internals](modules.md): generic extension registration, artifact
families, module construction, semantic reconciliation, validation, and
reference mechanics.
- [D&D Module Internals](dnd.md): shared D&D extractor conventions, generated
reference projections, and lane-specific exceptions. Durable D&D and
Seriatim data shapes remain in the [integration contracts](../integrations/).
Use this map to find an owner, then read the focused document and its tests
before changing behavior.

View File

@@ -1,359 +1,194 @@
# Pipeline Internals # Pipeline Internals
The implemented resolver and runner live in `internal/framework/pipeline`. This document describes the framework-owned pipeline mechanics in
Their fixed workflow and ownership boundaries are defined by **internal/framework/pipeline**. [Configuration](../config.md) owns selectable
[Architecture](../policy/architecture.md#system-shape). Configuration fields, profiles, bindings, and retry settings; [Operations](../operations.md) owns
defaults, and selectable keys are defined in state lifecycle and recovery; and the [integration contracts](../integrations/)
[Configuration](../config.md#pipelines). own durable output shapes. Concrete production extensions are covered by
[Module Internals](modules.md).
Resolution fixes the selected lanes and all stage bindings; preparation ## Boundary
constructs every selected implementation before the runner begins source work.
After serial input parsing and plan selection or generation, the runner
materializes chunks and dispatches extract work to
one bounded run-wide worker pool in chunk-first, lane-second order. Each lane's
merge and normalize operations remain serial and may overlap other lanes once
all extracts for that lane are terminal.
## Resolution The pipeline framework accepts a resolved composition, registries, shared
dependencies, input bytes, a supplied prompt session, and state/debug
collaborators. It returns logical output files, normalized artifacts, recorded
rejections and warnings, manifest provenance, and checkpoint decisions. The
CLI owns process arguments, configuration discovery, session resolution,
physical roots, and placement of returned output files.
`internal/core/config.Config.Resolve` validates the loaded configuration, The framework has one fixed shape:
selects the named profile, applies the runtime inputs supplied by the CLI, and
calls `pipeline.ResolvePipeline`.
`ResolvePipeline`: ~~~
input -> chunk -> extract -> merge -> normalize -> output
~~~
1. selects and sorts artifact lanes; Input and chunking are pipeline-wide. A selected artifact lane owns extract,
2. completes omitted bindings using the documented configuration defaults; merge, and normalize; output aggregates the terminal lane outcomes. A pipeline
3. looks up each module and validator spec without constructing it; is an ordered list of steps, not an arbitrary workflow graph.
4. for a typed extractor, derives its artifact kind, requires the codec, and
selects exact-type merger, normalizer, and validator variants;
5. checks required and provided capabilities in workflow order;
6. resolves target-aware reference bindings and validator chains;
7. validates each selected module and validator option set through its registry
entry; and
8. calculates a digest over the resolved structure, including typed artifact
kind and schema identity and the effective validator policy in its resolved
execution order.
Resolution returns a `ResolvedPipeline` containing ordered lanes, concrete ## Resolve, Materialize, Prepare
bindings, validator chains, reference targets, and the digest. It does not read
reference bytes or construct runtime modules. CLI lane and reference selector
syntax is defined in the [CLI reference](../cli.md#run).
The digest includes each resolved validator chain's stage, lane, owning module, Resolution turns a configured pipeline profile into a **ResolvedPipeline**.
ordered validator bindings, execution classes, targets, and artifact kinds. It normalizes the pipeline and lane identities, applies stage defaults, selects
Changing a default chain or an explicit override therefore changes pipeline requested lanes where that is supported, resolves validator chains, checks
identity whenever it changes the effective validator policy. module capabilities and typed artifact compatibility, validates options, and
assigns a deterministic resolved-composition digest. The resolved pipeline
contains bindings and declared reference targets, not external reference bytes.
After selection, the resolver applies command, binding, and pipeline profile
precedence to LLM-backed bindings and validators only; prompt defaults remain
an empty resolved binding profile. Deterministic bindings remain profile-free.
These effective values are part of the digest, so execution and checkpoint
consumers do not repeat profile inheritance.
Configuration resolution supplies the selected profile and catalog; see
[Configuration Internals](configuration.md).
## Reference Materialization External reference materialization happens before preparation. The materializer
checks that each slot is declared by the selected module, resolves a file path
relative to the correct configuration or working-directory origin, reads
UTF-8 text, verifies media type and size limits, and retains bounded
provenance. For a positive slot limit, it reads at most the limit plus one byte
and rejects overflow before retaining content. A generated-artifact selector
remains declared but has no bytes until its producing step completes.
The CLI calls `MaterializeReferences` after resolution and before constructing Preparation is the construction boundary. It validates the resolved shape and
the LLM client or running the pipeline. The materializer checks each binding registry set, clones the resolved data, then constructs the input adapter,
against its resolved target declaration, reads and validates the file, and chunker, stage-local validators, every typed lane, and output encoder. Each
builds both a `contracts.ReferenceSet` and provenance-only metadata on the registered builder receives its own cloned build request immediately before its
corresponding `ResolvedReferenceTarget`. module-owned code runs. Preparation also collects stable checkpoint
fingerprints. Missing registrations, incompatible typed entries, nil
implementations, and constructor failures are reported before source parsing
or any stage operation begins.
Preparation delivers the materialized set for each target through An output encoder can opt into source-evidence publication through its output
`pipeline.BuildRequest`: chunkers and chunk validators receive the chunk target; policy. Preparation keeps the configured lane allowlist and active lanes
extractors and extract validators receive the lane extract target; mergers and separate, then verifies an exact typed evidence projector and registered codec
merge validators receive the lane merge target; and normalizers and normalize for each active lane. The resulting private plan is immutable; lanes excluded
validators receive the lane normalize target. Input and output builders receive by invocation filtering remain configured but do not acquire a projector for
an empty set because those stages cannot declare references. Every builder gets that run.
an isolated deep clone of its target set, so construction-time mutation cannot
change another builder, the resolved pipeline, or later runtime requests.
The runner continues to clone the resulting set into the chunk, extract, merge, ## Typed Lanes And References
or normalize request that owns the target. LLM-backed extensions may convert
those items into named prompt inputs. Reference content remains separate from
source evidence and source digests.
Binding precedence, path resolution, accepted content, and media-type behavior Each resolved lane has one artifact kind, codec, and exact Go type. The
are configuration contracts; see [Configuration](../config.md#pipelines). framework uses private type erasure only around those typed operations; every
Durable provenance is defined in the handoff checks exact type and codec identity and reports incompatibility as an
[JSON output contract](../integrations/json-output.md#manifestjson), while error rather than panicking. Encoding through the registered codec is the
runtime sensitive-data handling belongs in [Operations](../operations.md). boundary for output, checkpoints, debug records, and generated references.
## Registries And Specs Reference targets are stage- and lane-specific. External reference bytes are
cloned into the operation request. Generated references are built at the next
step boundary from exactly one accepted normalized producer output. The
framework decodes and re-encodes that output with the registered producer
codec, checks its complete schema and media identity, and records a content
digest plus bounded producer provenance. A missing, ambiguous, invalid, or
incompatible producer prevents the consumer step from starting.
`pipeline.Registries` holds option validators and run-local builders used during ## Execution And Ordering
resolution and preparation.
`pipeline.ModuleCatalog` exposes their specs during configuration validation and
resolution. Separate registries exist for every stage and for validators;
`ValidatorChainRegistry` stores production default-chain mappings. Both
containers also carry an `ArtifactCodecRegistry`. Generic registration records
one codec per stable artifact kind, validates its schema metadata and JSON
Schema, retains the exact schema digest and Go type, and safely encodes or
decodes framework-erased values with typed errors on incompatibility.
Typed extractor entries are keyed by module key and declare one artifact kind. The runner validates its input, installs no-op state collaborators when none
Merger, normalizer, and typed-validator variants are keyed by module or were supplied, and serially performs source parsing and chunk-plan selection.
validator key plus artifact kind. Chunk and serialized validators occupy It transports the supplied session unchanged to prompt-facing operations and
separate target namespaces; serialized registrations declare whether they run-manifest metadata; it neither derives a session nor substitutes a parsed
support chunks, artifacts, or both. Duplicate variants and exact Go-type source document identifier. The public session contract is owned by the
mismatches are rejected deterministically. [CLI reference](../cli.md#run).
An accepted plan is materialized into source-addressed chunks and passes the
configured chunk validators before any lane runs. A chunk rejection is a
recorded pipeline outcome: lanes do not start, but the output stage can encode
the terminal result.
Lane-sensitive merger and normalizer spec discovery always supplies the For each ordered step, the runner first builds generated reference sets from
extractor's artifact kind, so variants under one reusable key may declare the accepted normalized outputs of earlier steps. It then executes the step's
different capabilities and reference slots. Kind-neutral registry inspection lanes. Later steps do not begin until the current step is terminal and its
selects the first registered artifact kind in sorted order. generated handoffs have succeeded.
Production composition registers the D&D spell-list codec and typed extractor, Within a step, the lane engine dispatches extraction jobs in deterministic
matching typed merge, normalize, and semantic-validator variants, and chunk-first, lane-second order to a bounded worker group. When all extraction
serialized JSON validators. Every artifact lane resolves through the typed jobs for one lane are terminal, a bounded continuation group can run that
registries and a matching codec. lane's merge and normalize work while extraction for other lanes continues.
The framework does not create an unbounded goroutine per chunk or lane.
A `ModuleSpec` declares its stage plus required and provided capabilities. Completion timing does not determine public results. The coordinator restores
Chunk, extract, merge, and normalize specs may also declare reference slots. lane and chunk order before merging results, and selects a framework error by
Registry implementations defensively copy spec metadata, reject duplicate keys, stable stage, lane, and chunk position. A validator rejection records a lane
and verify that a constructed implementation reports the registered key. outcome without cancelling unrelated work. A framework error or parent
Builder registrations accept `ModuleDependencies` and cloned configuration cancellation cancels derived work, prevents queued work from starting, waits
options through one `BuildRequest`. Builders decode those options and retain for started workers, and prevents output encoding.
typed values or injected dependencies in the constructed implementation.
Extractors declare their artifact kind, and merger, normalizer, and validator
resolution selects the matching typed variant.
A `ValidatorSpec` declares a validator key and execution class. Resolution uses ## Validation, Retries, And Output
the execution class to reject incompatible profile bindings before execution.
The current production catalog and default chain are listed only in
[Configuration](../config.md#implemented-production-validators).
## Preparation And Runner Boundary Every chunk, extract, merge, and normalize candidate passes its resolved
validator chain. Validators receive immutable canonical input appropriate to
their target: chunks, codec-decoded typed candidates, or serialized codec
bytes. Each typed validator receives a newly decoded value from the one
candidate serialization for that attempt, while serialized validators receive
separately owned representation bytes and schema metadata. They may approve,
approve with warnings, reject, or fail. A rejection is an ordinary pipeline
result; a validator error is a framework error.
`pipeline.Prepare` receives a resolved pipeline, the registries, and shared The runner applies the binding's retry policy around a stage operation and its
module dependencies. It constructs input; chunk and its validators; each lane's complete validation chain. It preserves warnings only from the final accepted
extract, merge, and normalize modules and validator chains in resolved order; or rejected attempt. Cancellation stops retries. Normalizer-specific retry
then output. It stops at the first error with pipeline, stage, lane, module, and directives consume this same budget and validate any final safe fallback through
validator context as applicable. It never invokes an operation method. the normalizer chain.
`PreparedPipeline` keeps private constructed executors and exposes cloned After terminal lane work, the runner assembles manifest provenance, normalized
resolved input, chunk, lane, and output identities. Prepared components may artifacts, rejections, warnings, and an optional accepted chunk map. When an
implement `pipeline.CheckpointFingerprintProvider` to contribute explicit output policy selected evidence lanes, it decodes accepted serialized normalize
semantic identities to checkpoint reuse. Preparation trims and validates each outputs through their registered codecs and invokes the prepared typed
non-secret name and value, prefixes it with the component's stage, lane, projectors. Rejected or absent lanes contribute nothing. This reconstruction is
module, and validator scope, rejects duplicates, and retains the resulting also used after normalized-checkpoint reuse, so no second typed output channel
sorted collection behind a defensive-copy accessor. Fingerprints must be is retained. The runner passes the resulting owned artifact to the output
stable and must not contain source content, credentials, local paths, encoder, which returns logical files and does not choose a physical directory.
timestamps, or other invocation-specific values. The CLI publishes those files only after the runner returns without a framework
error. Logical file names and schemas are defined by the [output integration
`pipeline.RunInput` carries that prepared pipeline, raw source input, run identity and timing, optional contracts](../integrations/).
session and profile metadata, a chunk-plan store and mode, and checkpoint/debug
collaborators. The runner
parses source bytes through the already constructed input adapter. Later stage
requests receive the generic source model; extract requests receive
chunk-scoped input material, while chunk, merge, and normalize requests retain
access to the original source material. Input, chunk, and output operation
requests do not carry raw module options. The chunk request also does not carry
an LLM client; an LLM-backed chunker receives the shared client during
preparation. Their operation requests retain run-specific source, reference,
profile, session, and metadata context as applicable.
Prepared lanes retain exact-type-checked erased operation closures. The runner
uses those closures to keep each value typed through extraction, validation,
merge, and normalization.
Source validation requires every unit to carry a canonical self-reference to
its containing document and its own unit ID. Explicit clone, checkpoint, and
debug boundaries retain that reference, and the canonical source digest covers
it deterministically. Chunks use the same source model and carry one canonical
reference spanning the first selected unit through the last.
`pipeline.RunOutput` carries the run manifest, accepted normalized serialized
artifacts with lane and normalizer provenance,
rejected results, warnings, checkpoint events, and logical files returned by the
output encoder. The CLI owns debug-summary and durable filesystem writes after
the runner returns.
## Execution Flow
The runner:
1. validates its prepared input;
2. parses the raw input with the prepared adapter and validates the generic
source document;
3. selects a stored plan or executes the configured chunker's `Plan` operation;
4. canonicalizes and materializes the plan, then validates the resulting
chunks;
5. dispatches extract jobs in source-chunk then resolved-lane order, starting a
bounded lane continuation when all extracts for that lane are terminal;
6. invokes the prepared output encoder and validates its logical file results;
7. returns the assembled manifest, outcomes, warnings, and files.
Within each artifact lane, it reuses the prepared extractor, merger, normalizer,
and validators while performing these transitions:
1. extract once per accepted chunk and add runner-owned lane, source, and chunk
provenance;
2. validate each extract result and omit rejected results from merge input;
3. skip the rest of the lane when no extract result is accepted;
4. merge accepted extract results in their existing order;
5. validate the merge result and skip normalization on rejection;
6. normalize the accepted merge result;
7. validate and append the accepted normalized result.
Module-provided warnings and payload warnings are promoted only from attempts
whose results are accepted and used.
## Chunk Plans And Reuse
`Chunker.Plan` returns a `source.ChunkPlan`: the canonical source digest,
ordered unit-ID ranges, and optional plan or range annotations. The framework
owns plan canonicalization and materialization. It creates the generic chunks
and therefore owns their IDs, indexes, source references, JSON content, units,
media type, and generic metadata. Plan and range annotations are independently
owned raw JSON and become `Chunk.PlanAnnotations` and `Chunk.Annotations`.
In `auto`, the runner looks up the source digest before invoking the chunker. A
valid hit is materialized and sent through the current run's configured chunk
validators; it does not invoke the chunk module, consume its retry budget, or
make a chunk-stage LLM call. A missing, invalid, or unmaterializable record
generates a candidate. `refresh` generates without lookup; `bypass` generates
without cache access. Generated plans are published only after the full chunk
validator chain approves them. A validator rejection is a regular rejected
pipeline outcome and never replaces a cached plan.
The store is source-addressed, not pipeline-addressed. Changes to pipeline
configuration, requested chunker, options, references, lanes, validators, or
LLM profile do not prevent a source-digest hit. The manifest records both the
currently requested chunker and the effective plan producer. Cache state and
paths are configured and operated outside the runner; see
[Configuration](../config.md#state-surfaces) and [Operations](../operations.md).
The extract job channel has the same capacity as the effective extract worker
count, so dispatch applies backpressure. A fixed continuation executor prevents
ready or checkpoint-reused lanes from creating one goroutine each. Workers and
continuations publish lane-local results; the coordinator is the only writer of
aggregate output and merges those results in resolved lane and source-chunk
order.
## Plan Canonicalization And Chunk Materialization
Plan canonicalization requires canonical JSON annotations, a matching source
digest, at least one range, existing ordered boundaries, and increasing range
starts. Ranges may overlap or leave gaps; a chunker may impose stricter policy.
Materialization deterministically reconstructs each range from the current
source document, deep-clones JSON-shaped source-unit metadata, and copies
annotations without interpreting their namespaces. Materialized chunks and
separate materializations do not share mutable unit metadata; unsupported or
cyclic metadata fails materialization with context.
Before lane execution, generic chunk validation checks the materialized chunks'
identities, order, source references, content, media type, units, and metadata.
No chunk checkpoint participates in plan selection: plan storage is the only
chunk-reuse mechanism. Extract, merge, and normalize checkpoints continue to
use materialized chunk digests as their dependencies.
## Validation And Retries
Chunk, extract, merge, and normalize results pass through the resolved validator
chain for their stage and module. Chunk validators receive canonical chunks;
typed validators receive the domain value; and serialized validators receive
canonical chunk JSON or artifact codec bytes. Validators execute in resolved
order and stop at the first error or rejection. An empty chain approves the
result.
`runWithRetry` applies the effective retry policy around module execution and
its complete validation chain. A module or validator error becomes a framework
error when attempts are exhausted. A rejection becomes a recorded
`RejectedOutput` when attempts are exhausted. Cancellation stops retry
processing immediately.
Rejected output is a non-fatal pipeline outcome and does not advance. Warnings
from discarded attempts are not promoted. Configuration owns retry counts and
validator overrides; see [Module Bindings](../config.md#module-bindings).
## Checkpoint And Debug Hooks ## Checkpoint And Debug Hooks
The runner depends on recorder and loader interfaces, using no-op The runner receives checkpoint and debug interfaces rather than roots. It
implementations when collaborators are absent. Each checkpointed workflow records workflow transitions and reuse decisions through the supplied
boundary records a running, succeeded, or failed transition. Reuse decisions collaborators, and clones reusable artifacts before they re-enter normal typed
are consulted in workflow order and accepted payloads are cloned before handoff. Generated-reference dependencies participate in checkpoint decisions.
entering the normal handoff path. Typed extract, merge, and normalize Selective recomputation can require a canonical accepted normalized predecessor
checkpoints store codec bytes with artifact kind, schema ID and version, exact before a dependent lane starts.
schema digest, and media type. Reuse compares that identity with the prepared
codec and decodes through the codec; missing identity, mismatches, corrupt
bytes, and decode failures become explicit reuse misses and execute the step
normally. Dependency fingerprints and debug content digests use the same stable
codec bytes that cross those boundaries.
The CLI includes prepared-component fingerprints in the run-wide checkpoint Debug recording is attempt-scoped and application-owned. A failure to persist
identity alongside resolved configuration, raw input, reference provenance, required debug data is a framework error. State roots, persistence, reason-code
runtime overrides, and LLM-profile fingerprints. Module metadata is not used meanings, resume, and cleanup are intentionally owned by
implicitly for cache identity: components opt in only with stable semantic [Run State Internals](state.md) and [Operations](../operations.md).
values that can change accepted output. Adding or changing a component
fingerprint intentionally produces a cold cache miss. Existing checkpoint
schemas and paths remain unchanged.
Debug instrumentation wraps run, stage, attempt, validator, and structured LLM ## Invariants To Preserve
boundaries. Every executed chunk, extract, merge, and normalize attempt writes
one terminal envelope for acceptance, validator rejection, module or validator
error, or applicable candidate or final serialization error. The envelope
contains its attempt-local warnings, any available candidate and rejection,
and terminal error text; failures before a candidate exists omit that payload.
Only LLM calls made by the module operation belong to the module attempt.
Validator calls retain independent scopes under `validate/` and are not
duplicated into the module envelope. A failed terminal-envelope write is a
non-retryable framework error and is joined with any primary attempt error.
Debug data is never used as a checkpoint source. Typed artifact debug envelopes
are domain-neutral, redact sensitive metadata and bytes through the common
debug policy, and record codec identity plus schema and content digests.
Merge and normalize attempts serialize their in-memory candidate with the - The six fixed stages remain explicit; a pipeline is not a general DAG.
codec's required candidate encoder before typed validation. Serialized - Resolution and preparation reject statically discoverable incompatibility
validators and attempt debug use that candidate representation, which carries before parsing or execution.
the codec media type and schema identity but is never checkpointed or passed - Every typed lane uses one compatible artifact kind, codec, and exact Go type.
downstream. Only a validator-approved value is encoded through the strict final - Generated references come only from one earlier accepted normalized producer
codec and made eligible for a checkpoint or stage output. and carry canonical identity rather than an unverified value.
- Rejections are recorded outcomes; framework errors cancel derived work and
prevent output encoding.
- Public ordering and selected errors are independent of goroutine completion
order.
- Pipeline modules receive collaborators and data, never CLI streams or
physical output, cache, or debug roots.
Checkpoint identity, physical layout, reuse behavior, and debug artifact ## Focused Tests
handling are operator contracts in [Operations](../operations.md). Serialization
and recorder implementation are inventoried in
[Internal Overview](overview.md#run-state-components).
## Results And Failures - **internal/framework/pipeline/profile_test.go** and
**typed_resolution_test.go** cover resolution, defaults, ordered steps,
compatibility, validators, references, and resolved identity.
- **internal/framework/pipeline/preparation_test.go** covers complete
construction before execution and contextual construction failures.
- **internal/framework/pipeline/references_test.go** and **handoff_test.go**
cover external materialization, generated references, provenance, and typed
producer checks.
- **internal/framework/pipeline/runner_concurrency_test.go** covers bounded
execution, ordered steps, stable error selection, rejections, and
cancellation.
- **internal/framework/pipeline/runner_chunk_plan_test.go**,
**runner_typed_checkpoint_test.go**, and
**runner_accepted_checkpoint_test.go** cover state hooks and reuse behavior.
- **internal/framework/pipeline/runner_attempt_debug_test.go** and
**runner_terminal_debug_test.go** cover attempt and terminal debug behavior.
The runner owns manifest assembly and handoff summaries but not the durable JSON Run **go test ./internal/framework/pipeline ./internal/cli** after changing a
schema. It records resolved module and lane provenance, validator chains, pipeline boundary. Use the more focused tests above while iterating.
source/reference identities, selected LLM profiles, normalized and rejected
summaries, status, and timing. Serialized artifact content remains outside the manifest.
Module metadata providers may add non-secret singleton or lane-scoped metadata.
Execution errors include stage, module, lane, or validator context. Once a
manifest exists, a failing run returns it with failed status and completion
time. Successful status reflects whether any result was rejected. The
durable manifest and logical file schemas are defined in the
[JSON output contract](../integrations/json-output.md).
On a framework failure, the runner cancels its derived context, stops submitting
new extract work, drains started tasks, and skips the output encoder. Parent
cancellation takes precedence. Otherwise context-cancellation fallout is
discarded when a substantive error exists, and the primary error is selected by
stage, resolved lane, and source chunk rather than completion time.
## Tests To Inspect
- `internal/core/config/effective_config_test.go`: config-to-resolution boundary.
- `internal/framework/pipeline/profile_test.go`: selection, defaults,
capabilities, validator chains, and digest behavior.
- `internal/framework/pipeline/artifact_codec_registry_test.go`: typed codec
metadata, registration, erasure safety, strict decoding, and cloning.
- `internal/framework/pipeline/typed_resolution_test.go`: heterogeneous typed
lane resolution and preparation, target-specific validators,
incompatibilities, ordering, and schema-sensitive pipeline identity.
- `internal/framework/pipeline/runner_concurrency_test.go`: bounded dispatch and
continuations, reverse completion, stable errors, rejection, cancellation,
retries, and independent provider-call limits.
- `internal/framework/pipeline/preparation_test.go`: option validation,
construction order, dependency failures, and the before-source-work boundary.
- `internal/framework/pipeline/references_test.go`: target resolution and
materialization.
- `internal/cli/run_contract_test.go`: production run transitions, retries,
rejections, warnings, debug hooks, and manifests.
- `internal/cli/production_contract_test.go`: production composition and
configuration-resolution smoke coverage.
- `internal/cli/example_contract_test.go`: maintained example resolution and
execution ownership.
- `internal/modules/integration/*_test.go` and
`internal/modules/seriatim/input/transcript/runner_test.go`: typed runner
composition across concrete module families.
- `internal/framework/checkpoint/*_test.go`: checkpoint serialization and reuse
collaborators.

View File

@@ -2,7 +2,8 @@
This document describes the implementation collaborators behind output, cache, This document describes the implementation collaborators behind output, cache,
and debug state. User-visible fields belong in [Configuration](../config.md), and debug state. User-visible fields belong in [Configuration](../config.md),
and layouts and lifecycle belong in [Operations](../operations.md). and physical layout, retention, recovery, reason codes, and cleanup belong in
[Operations](../operations.md).
## Composition ## Composition
@@ -12,11 +13,22 @@ constructs cache collaborators, writes logical output files, and reports paths.
Pipeline modules receive interfaces and request data, never output, cache, or Pipeline modules receive interfaces and request data, never output, cache, or
debug roots. debug roots.
The CLI creates no chunk-plan store in bypass mode. It creates a checkpoint
recorder only when recording is enabled and a checkpoint loader only for a
resume invocation. It allocates debug state only after a safe run identity has
been generated and only when debug capture was requested. These choices keep
the three state families independently composable.
## Output And Cache ## Output And Cache
The pipeline runner returns logical output files. After validating every The pipeline runner returns logical output files. After validating every
logical name, the CLI exclusively creates the run directory beneath the logical name, the CLI exclusively creates the run directory beneath the
selected output root and performs confined, atomic file writes within it. selected output root and performs confined, atomic file writes within it.
The runner supplies an accepted chunk map as an optional, defensively owned
output-request artifact. The JSON encoder alone decides whether its explicit
option writes the map and optional index descriptor; neither the map payload
nor its annotations are copied into the run manifest. The durable fields are
owned by the [Accepted Chunk Map contract](../integrations/chunk-map.md).
`internal/framework/chunkplan` owns source-addressed plan storage, validation, `internal/framework/chunkplan` owns source-addressed plan storage, validation,
and atomic publication. Its store is constructed only when the selected mode is and atomic publication. Its store is constructed only when the selected mode is
@@ -27,14 +39,60 @@ codecs, loader, and recorder. The CLI constructs a recorder whenever checkpoint
recording is enabled and constructs a loader only for a `--resume` invocation. recording is enabled and constructs a loader only for a `--resume` invocation.
Identity incorporates explicit stable semantic fingerprints collected from Identity incorporates explicit stable semantic fingerprints collected from
prepared modules and validators in addition to configuration, input, prepared modules and validators in addition to configuration, input,
references, runtime overrides, and LLM profiles. references, runtime overrides, observed LLM profiles, and the LLM runtime's
non-secret effective profile-source identity. A profile source change therefore
causes a cold miss even when the configured profile ID remains unchanged.
The serialized The serialized
`workspace_schema_version` identifiers are frozen wire-compatibility fields; `workspace_schema_version` identifiers are frozen wire-compatibility fields;
they do not describe a current public state surface. they do not describe a current public state surface.
Ordered-step lane checkpoints include the step identity in their storage scope.
Accepted step and lane identities are encoded injectively before becoming
filesystem path components, while ordinary safe identifiers retain their
readable paths.
When a later lane consumes a generated artifact, its dependency fingerprints
include the producer's artifact kind, complete schema identity, media type,
canonical content digest, and size. Ordinary resume compares those fingerprints
when progressively loading consumer stage checkpoints, so changed producer
content produces `dependency_invalidated` rather than stale downstream reuse.
Selective recomputation instead requires each unselected producer's accepted
normalized artifact; invalid accepted state records its specific bounded reason
and stops before the dependent. The selected step and its transitive dependents
record `forced_recompute`.
Ordinary resume loads extract, merge, and normalize checkpoints progressively
and may execute later lane stages after an earlier cache miss. Selective
recomputation instead asks the loader for the required producer's accepted
normalize artifact. That lookup reuses the existing normalize files, requires
workspace schema v3 plus an exact non-empty invocation identity, and deliberately
does not require extract or merge checkpoint files or dependency fingerprints.
The runner performs canonical codec and producer-provenance validation before
cloning the artifact into normal step output. Success restores only stored
normalize warnings and emits one normalize decision; failure retains the files,
records the decision, and stops without executing the producer or consumer.
The loader assigns a typed category and reason code at each validation site;
diagnostic prose is not classified after the fact. The runner then applies
forced-execution policy, validates reusable artifact bytes through the prepared
codec once, returns the canonical hydrated value to the stage, and records the
final decision before enforcing a required-predecessor failure. That failure
names only the step, lane, and stable reason code. Decision detail is selected
from code-owned descriptions by reason code and then UTF-8 normalized and
bounded; callers cannot supply arbitrary diagnostic prose. Typed categories and
codes remain intact through pipeline events and become strings only in manifest
and debug-summary JSON.
[Operations](../operations.md#checkpoint-recording-resume-and-recompute) is the
canonical operator-facing reason-code reference.
`internal/core/fileio` provides confined atomic file writes used by state `internal/core/fileio` provides confined atomic file writes used by state
collaborators. The chunk-plan store retains its stronger entry validation. collaborators. The chunk-plan store retains its stronger entry validation.
The CLI constructs selective-recomputation policy from resolved generated
artifact dependencies. It forces the selected step and transitive consumers,
while marking unforced producers as required reusable inputs. The runner owns
the actual hydration and rejection decisions; the [Operations guide](../operations.md#checkpoint-recording-resume-and-recompute)
owns the operator workflow and stable reason-code meanings.
## Debug Bundles ## Debug Bundles
`internal/core/debugbundle` allocates an explicitly requested per-run bundle `internal/core/debugbundle` allocates an explicitly requested per-run bundle
@@ -48,18 +106,40 @@ boundaries redact sensitive metadata and credential-shaped bytes while allowing
application-owned trace material. Debug data is never a checkpoint source or application-owned trace material. Debug data is never a checkpoint source or
cache input. cache input.
Generated reference bytes exist only in cloned operation requests and are not
written as paths into checkpoints, manifests, or debug summaries. Those state
surfaces retain canonical identities and bounded producer provenance so that a
resume decision can be explained without copying generated campaign content.
After allocation, one CLI-owned state value accumulates the known report paths, After allocation, one CLI-owned state value accumulates the known report paths,
pipeline outcome counts, and validation status. A single guarded terminalization pipeline outcome counts, and validation status. A single guarded terminalization
operation writes the success report, or makes one attempt each to write the operation writes the success report, or makes one attempt each to write the
failure report and error log. Terminal persistence failures are reported failure report and error log. Terminal persistence failures are reported
separately and never replace the command's primary error. separately and never replace the command's primary error.
## Invariants To Preserve
- Modules receive state collaborators and request data, never physical roots.
- Output logical paths are validated before a run directory is allocated, and
files are atomically written within that directory.
- Chunk-plan publication occurs only for accepted plans; bypass does not
construct or touch a plan store.
- Checkpoint recording and checkpoint loading remain separate collaborators.
- Debug state is opt-in, is not cache input, and terminal reporting does not
obscure the command's primary failure.
## Tests To Inspect ## Tests To Inspect
- `internal/cli/run_contract_test.go`: command-owned state allocation, - `internal/cli/run_contract_test.go`: command-owned state allocation,
terminalization, and output/report boundaries. terminalization, and output/report boundaries.
- `internal/cli/cache_contract_test.go`: cache-mode precedence, root selection,
and resume collaborator construction.
- `internal/cli/state_hardening_test.go`: independent roots, reuse, failures, - `internal/cli/state_hardening_test.go`: independent roots, reuse, failures,
permissions, cleanup, and redaction. permissions, cleanup, and redaction.
- `internal/cli/recompute_policy_test.go`: forced dependents and required
reusable predecessors for selective recomputation.
- `internal/cli/recompute_execution_contract_test.go`: selective recomputation,
filesystem recovery, deterministic decisions, and failed predecessor state.
- `internal/cli/production_contract_test.go`: production composition and - `internal/cli/production_contract_test.go`: production composition and
configuration validation at the CLI boundary. configuration validation at the CLI boundary.
- `internal/cli/example_contract_test.go`: maintained example ownership. - `internal/cli/example_contract_test.go`: maintained example ownership.

View File

@@ -1,193 +1,301 @@
# Operations # Operations
This is the canonical guide to operating Notarius filesystem state. Command This is the canonical guide for operating Notarius runtime state. The
syntax is in the [CLI reference](cli.md); field definitions and precedence are [CLI reference](cli.md) owns command syntax and exit statuses, while
in [Configuration](config.md). [Configuration](config.md) owns fields, defaults, and precedence. Maintainers
who need implementation mechanics should read [Run State Internals](internal/state.md).
## State Model ## State Surfaces
Notarius uses three independent filesystem surfaces: Each run can use independent roots with different retention and access-control
needs.
- output is durable user data; | Surface | Purpose | Created when | Retention |
- cache is reconstructible chunk-plan and checkpoint state; and | --- | --- | --- | --- |
- debug is explicitly requested inspection data. | Output | Durable user-facing result bundle | A pipeline completes and returns logical output files | Keep until consumers no longer need it. |
| Chunk-plan cache | Reconstructible source-addressed plan | The configured cache mode permits cache I/O | Keep while reuse is useful. |
| Checkpoint cache | Reconstructible execution and recovery state | Checkpoint recording is enabled | Keep only while recovery or reuse is useful. |
| Debug bundle | Explicit diagnostic record | A run requests debug collection | Keep only under an intentional sensitive-data retention policy. |
Choose separate roots and access controls for each surface. A normal run writes Output, cache, and debug roots are never merged or cleaned automatically. Use
durable output, may use the chunk-plan cache, and records checkpoints when separate locations and permissions for operators or services that must not
`cache.checkpoints.enabled` is true. It does not create debug state unless its share application data.
invocation includes `--debug`.
## Output ## Roots And Permissions
Durable logical files are written under: The configured output and debug directories are exact roots. An empty cache
directory selects a per-user root:
```text ~~~
<os.UserCacheDir>/notarius/chunk-plans
<os.UserCacheDir>/notarius/checkpoints
~~~
The field definitions and configuration examples are in [Configuration](config.md).
On supported Unix systems, output directories and files are created with
requested modes **0755** and **0644**. Chunk-plan, checkpoint, and debug
directories and files use **0700** and **0600**. The operating system's umask
may impose stricter output modes. Cache and debug roots may contain sensitive
source-derived data, so provision them for one trusted account or service. An
output bundle can also contain source content when its JSON output enables
evidence publication. Apply an appropriate umask and output-root access policy
before enabling that option; the requested output modes alone may not be
suitable for transcript-bearing bundles.
## PromptKit Profile Deployment
Profile deployment has four distinct layers:
| Layer | Owner | Operational role |
| --- | --- | --- |
| Prompts and schemas | Notarius module families | Embedded request and structured-output definitions. They are not deployment profile files. |
| Fallback profiles | Notarius module families | Embedded application defaults, including D&D's `dnd-extraction` profile. |
| Built-in profiles | PromptKit | Upstream catalog entries available when no higher-precedence source defines an ID. |
| Operator profiles | Deployment filesystem | Complete environment-specific definitions selected by `promptkit.profile_file` or `promptkit.profile_dir`. |
The maintained D&D pipeline uses the workload ID `dnd-extraction`. The
embedded fallback makes that ID usable without an operator file. Production,
development, and local deployments can each install a different complete
definition for the same ID, retaining the pipeline while choosing their own
model, backend, timeout, or reasoning policy. An operator definition wins over
the fallback; it is not merged with it. The configuration field and full
precedence rules are owned by [Configuration](config.md#promptkit-profiles).
Use a profile source owned by the service account, keep it readable only by
the intended operator, and supply provider credentials through the service
environment—not in the Notarius configuration or profile YAML. The maintained
[operator profile](../examples/profiles/dnd-extraction.yml) is secret-free and
can be copied as a format starting point. Validate a deployment without a
provider call or credentials:
~~~sh
notarius config validate --config /etc/notarius/config.yml --pipeline dnd-session
~~~
Profile paths are currently resolved from the process working directory, not
from the configuration file. The complete example's
`./examples/profiles/dnd-extraction.yml` path is valid for a repository-root
invocation only. Use absolute paths such as
`/etc/notarius/profiles/dnd-extraction.yml` for services and containers.
## Run Lifecycle
Use the [run command](cli.md#run) to start a pipeline. A valid invocation loads
and resolves configuration before module preparation and source parsing. It
then performs any permitted cache lookup, executes the pipeline, and publishes
logical output files only after a successful runner result.
On success, the command reports the output bundle path. A warning-bearing run
still succeeds and reports its warning count on standard error. Errors and
their exit classes are defined in the [CLI reference](cli.md#output-streams-and-exit-statuses).
## Output Bundles
Each successful run receives a generated safe run identifier and writes beneath:
~~~
<output-root>/<run-id>/ <output-root>/<run-id>/
``` ~~~
The CLI generates one run ID in the form The [JSON output contract](integrations/json-output.md) owns the logical files
`run-<started-at-unix-nanoseconds>-<32-lowercase-hex-characters>` and uses it and their schemas. Before creating the run directory, Notarius validates every
for output, manifests, and any requested debug bundle. It validates every logical output path. It refuses an existing run directory without changing it.
logical output name before exclusively creating the run directory. If that Files are written atomically; if a later write fails, the newly created partial
directory already exists, the invocation fails without changing it. run directory remains for inspection and is never removed automatically.
Each output file is written atomically. A later file-write failure leaves the Treat an output bundle as durable user data. Do not use cache-cleanup policy to
newly allocated partial run directory in place for inspection; Notarius never remove it. An optional accepted chunk map is also durable output and can carry
automatically removes output. The source- or model-derived annotations; its content and compatibility contract
[JSON output contract](integrations/json-output.md) owns the logical file are defined in [Accepted Chunk Map](integrations/chunk-map.md). An optional
names, schemas, and media types inside a run directory. [evidence context](integrations/evidence-context.md) contains source-unit text
and metadata. It is not a cache or debug artifact: retain it with the output
Remove an output run directory only after its consumer data is no longer bundle only for as long as consumers need it, and apply source-content access
needed. This is data deletion, not cache cleanup. controls to the entire bundle. Its selected source-unit excerpt may include
every source unit once when coverage is broad or its configured window is
large, so do not assume a byte or token reduction or reduced sensitivity.
## Chunk-Plan Cache ## Chunk-Plan Cache
Chunk plans are stored at: Chunk plans live beneath the selected chunk-plan root:
```text ~~~
<chunk-plan-root>/<source-sha256-hex>/plan.json <chunk-plan-root>/<source-sha256-hex>/plan.json
``` ~~~
`auto` reuses a complete valid plan or regenerates missing or invalid state. One validated canonical plan is active for each source digest. The plan stores
`refresh` regenerates and atomically replaces a plan after chunk validation. boundaries and provenance, not a second copy of the entire source. This
`bypass` performs no plan-cache I/O and does not resolve or create the root. source-addressed policy is recorded in [ADR-0005](adr/0005-cache-canonical-chunk-plans-by-source.md).
Plan selection is source-addressed and independent of checkpoint and debug
roots.
When its directory is empty in configuration, the root is The configured cache mode controls one invocation:
`<os.UserCacheDir>/notarius/chunk-plans`. A configured directory is the exact
root; no suffix is appended. Directories and files created by the store use
`0700` and `0600` permissions on supported Unix systems. The configured root
is a trust boundary: do not share it among mutually untrusted users.
Remove an exact digest directory or the configured root only when accepting the - **auto** looks for a valid active plan. Missing or invalid state causes a new
cost of recomputing plans and any chunk-stage work. Cache publication is atomic; plan to be generated; an accepted new plan is atomically published.
there is no history, locking, garbage collection, or rollback facility. - **refresh** skips lookup, generates a plan with the configured chunker, and
atomically replaces the active plan after it is accepted.
- **bypass** performs no chunk-plan cache I/O. It does not resolve or create a
chunk-plan root.
For a Linux service account, provision a dedicated restrictive root such as: A reused plan is still materialized and validated against the current source.
If a prior plan no longer gives acceptable results, use a refresh run rather
than editing cache files. Deleting a plan is recoverable but can repeat costly
chunking work.
```yaml ## Checkpoint Recording, Resume, And Recompute
cache:
chunk_plans:
directory: /var/cache/notarius/chunk-plans
```
## Checkpoint Cache Checkpoint recording is an explicit configuration choice and is disabled by
default. When enabled, each run records stage transitions and the state needed
for compatible recovery. A run records checkpoints even when it does not ask
to reuse them. Checkpoint payloads can contain source-derived and intermediate
application data, so treat the entire root as sensitive.
Checkpoint recording is controlled by `cache.checkpoints.enabled`, which Checkpoint loading is separate: [**--resume**](cli.md#run) asks a run to reuse
defaults to `false`. When enabled, every run records running, succeeded, and compatible recorded work. A resume request fails when checkpoint recording is
failed transitions and reusable validator-approved results. Successful, disabled. Without **--resume**, a recording-enabled run executes normally and
rejected, and failed runs may therefore all leave checkpoint state. The does not load checkpoint state. Compatibility includes the resolved pipeline,
`--resume` flag additionally loads compatible completed work before executing input, selected lanes, runtime overrides, reference provenance, LLM-profile
missing or incompatible stages. Without `--resume`, a recording-enabled run provenance, the effective PromptKit profile-source fingerprint, and
never loads checkpoints. Using `--resume` while recording is disabled is an prepared-component fingerprints. When a local PromptKit backend is configured,
error. compatibility also includes a non-secret fingerprint of its endpoint. Changing
profile content or the local endpoint causes a cold miss; changing only the
local concurrency limit does not. A changed identity produces a cold miss;
Notarius does not migrate, rewrite, or delete older checkpoint directories.
Reasoning-effort inheritance, replacement, and explicit clearing are distinct
runtime identities, so checkpoints created under one state are not reused by
either of the others.
Checkpoints use the selected root and the existing identity hierarchy: Checkpoint state is confined below an identity-specific path:
```text ~~~
<checkpoint-root>/<pipeline-id>/<input-key>-<source-or-input-digest>/<pipeline-digest>/<identity-digest>/... <checkpoint-root>/<pipeline-id>/<input-key>-<source-or-input-digest-prefix>/<pipeline-digest-prefix>/<identity-digest-prefix>/
``` ~~~
The final identity digest includes stable semantic fingerprints explicitly ### Selective Recompute
contributed by prepared modules and validators. Adding or changing one of
these fingerprints intentionally causes a cold cache miss; old checkpoint
directories are left in place and are never migrated or deleted automatically.
An empty configured directory selects [**--recompute-step**](cli.md#run) requires both **--resume** and enabled
`<os.UserCacheDir>/notarius/checkpoints`. The root is exact when configured. checkpoint recording. It forces the selected ordered step and every lane that
Created directories and files use `0700` and `0600` permissions on supported depends on it through generated artifact references. Unrelated lanes remain
Unix systems. eligible for reuse.
Checkpoint payloads can contain source text, intermediate artifacts, metadata, For an earlier producer required by a forced consumer, Notarius requires a
warnings, and content digests. Treat them as sensitive derived application compatible accepted normalized artifact. It validates that artifact before
data. Compatible files from a former checkpoint root remain reusable when hydrating it and does not silently rerun the producer. If that state is
`cache.checkpoints.directory` names that exact existing root. They are not missing, rejected, corrupt, non-canonical, or incompatible, the run stops
moved, migrated, or deleted automatically. The frozen serialized identifier before its dependent starts. Rerun the required producer deliberately instead
`workspace_schema_version` remains part of checkpoint compatibility; it is not of copying or editing checkpoint files.
a configuration setting.
For a Linux service account, independently provision: ## Checkpoint Decisions And Recovery
```yaml Checkpoint events classify work as **executed**, **reused**,
cache: **forced_recompute**, or **dependency_invalidated**. Their stable reason codes
checkpoints: are written to run diagnostics and provenance. Use the code, not a copied
enabled: true error message, to decide what to repair.
directory: /var/cache/notarius/checkpoints
```
Remove an exact checkpoint identity directory or the configured root only when | Reason code | Recovery meaning |
recomputation is acceptable. | --- | --- |
| **loading_disabled** | This invocation did not permit checkpoint loading. |
| **checkpoint_missing**, **checkpoint_path_invalid**, **checkpoint_read_failed**, **checkpoint_decode_failed** | The stored checkpoint could not be located or read safely; normal resume work can execute again. |
| **workspace_schema_incompatible**, **identity_mismatch**, **stage_mismatch**, **step_mismatch**, **lane_mismatch**, **module_mismatch** | Stored state belongs to a different compatible scope or identity; allow a fresh run to create new state. |
| **status_not_reusable** | The recorded operation did not end in reusable state. |
| **dependency_mismatch** | A dependency changed; dependent work is invalidated rather than reused. |
| **artifact_payload_invalid**, **artifact_digest_mismatch**, **artifact_codec_incompatible**, **artifact_not_canonical** | A stored artifact cannot safely be hydrated; rerun the producer instead of modifying the cache. |
| **checkpoint_reused** | A normal checkpoint passed compatibility checks. |
| **accepted_artifact_reused** | A required predecessor's accepted normalized artifact was safely hydrated. |
| **recompute_step** | Selective recomputation deliberately forced this work. |
Reason detail is bounded code-owned text. It is diagnostic information, not a
path-discovery or data-recovery mechanism, and does not contain checkpoint,
source, reference, credential, or environment content.
## Debug Bundles ## Debug Bundles
Only `notarius run --debug` enables debug collection. The selected root contains Only a [debug-enabled run](cli.md#run) creates a bundle:
one retained bundle per invocation:
```text ~~~
<debug-root>/<run-id>/ <debug-root>/<run-id>/
summary/ summary/
trace/ trace/
``` ~~~
`summary/` contains redacted invocation, effective-configuration, resolved The summary contains redacted invocation and resolution information plus run,
pipeline and reference provenance, checkpoint and chunk-plan decisions, run warning, checkpoint, chunk-plan, and terminal reporting artifacts. The trace
manifest, warnings, report, and any available error text. It excludes raw contains allowlisted application diagnostic records and can include source or
source, references, annotations, prompts, model responses, credentials, and derived application data. Neither surface is a cache input. Do not treat a
malformed cache bytes. debug bundle as safe to share merely because its configuration summary is
redacted. Invocation metadata omits reasoning effort when it is inherited,
records the replacement value when one is supplied, and records an empty value
when inherited reasoning was explicitly cleared.
`trace/` contains application-owned execution detail, including source and Notarius never creates debug state without an explicit request and never
stage material, plans, chunks, validator attempts, prompts, model responses, automatically deletes a requested bundle. If allocation succeeds, the command
timing, and serialized artifacts. It may retain application data omitted from reports its path on both success and later failure. A summary, trace, or
output. Credentials, credential-shaped values, sensitive metadata, unrelated terminal-report persistence failure fails the command while preserving any
environment values, and unrelated filesystem content are not captured. already-written diagnostic data for inspection.
Bundles inherit the sensitivity of the application data they capture. Their
additional risk comes from copying and aggregating that data, so restrict
access, avoid shared roots between untrusted users, and define retention outside
Notarius. Created bundle directories use `0700` and files use `0600` on
supported Unix systems.
Notarius never automatically deletes a requested bundle. If allocation
succeeds, its path is reported on success and failure. A requested summary or
trace write failure makes the command fail, preserving whatever bundle data was
already written for inspection. Every allocated bundle makes one best-effort
attempt to record a terminal `run-report.json`.
## Failures And Warnings
Failures before debug allocation are reported on stderr without a bundle.
Failures after allocation report the bundle path on stderr and make independent
attempts to write a failure `run-report.json` and `error.log`. The report retains
the paths and pipeline outcome fields known at the failure point. If either
terminal write fails, the original command error remains first on stderr,
followed by the persistence error and bundle path. An output-write failure
leaves the allocated bundle in place. A successful run with warnings exits `0`,
reports a warning count on stderr, and records warnings in durable output and
any requested debug summary.
## Cleanup ## Cleanup
Use exact paths for manual cleanup. Examples: Cleanup is manual and destructive. First inspect the exact leaf directory,
then remove only that leaf; do not use a glob or a parent root as the target.
```sh ~~~
rm -rf ./notarius-output/run-1721300000000000000-0123456789abcdef0123456789abcdef rm -rf -- /srv/notarius/output/run-1721300000000000000-0123456789abcdef0123456789abcdef
rm -rf /var/cache/notarius/chunk-plans/0123abcd rm -rf -- /srv/notarius/chunk-plans/0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef
rm -rf /var/cache/notarius/checkpoints/pipeline/input-0123/pipeline-4567/identity-89ab rm -rf -- /srv/notarius/checkpoints/example/seriatim-0123456789abcdef/0123456789abcdef/0123456789abcdef
rm -rf ./notarius-debug/run-1721300000000000000-0123456789abcdef0123456789abcdef rm -rf -- /srv/notarius/debug/run-1721300000000000000-0123456789abcdef0123456789abcdef
``` ~~~
Avoid broad recursive cleanup against a parent root unless it is an explicit Deleting output permanently removes user data. Deleting chunk plans or
operator policy. Output deletion is permanent user-data loss. Cache deletion is checkpoints is recoverable but may repeat expensive provider or pipeline work.
recoverable but can repeat expensive work. Debug deletion removes troubleshooting Deleting a debug bundle removes troubleshooting evidence and a retained copy of
evidence and any retained application-data copy. application data. Notarius has no cache garbage collector, rollback operation,
or automatic cleanup command.
## Operational Limits ## Operational Limits
Provider retries and timeouts are handled by Scriptorium according to the Provider execution settings and the generation timeout come from the selected
selected execution profile. Pipeline module retry settings are defined in PromptKit profile. The invocation-only **--reasoning-effort** and
[Configuration](config.md#module-bindings). Extract worker concurrency and **--clear-reasoning-effort** controls may replace or clear that profile setting
actual provider-call concurrency are separate limits; their fields and for all LLM-backed calls in one run without changing the profile. PromptKit
validation are defined in [Configuration](config.md#concurrency). Notarius v0.5.0 does not add a provider retry loop. Notarius binding retries rerun the
writes local files only; remote storage and archive management are outside the complete module operation and validation chain as defined by
implemented CLI. [module bindings](config.md#module-bindings-and-validators).
Timeouts are layered. Caller cancellation is the outer authority. A positive
effective generation timeout adds an inner request deadline, while zero
disables only that generation deadline. The HTTP client timeout remains a
transport-wide cap. Notarius does not add another timeout around PromptKit.
The pinned upstream boundary and profile-format links are in
[PromptKit Integration](integrations/pkg-promptkit.md).
Concurrency has two independent layers. Notarius **total_llm** defaults to 16
and is the application-wide provider-call limit shared by all backends,
modules, retries, and validators. PromptKit may impose a narrower admission
limit for the selected backend. The effective active-generation bound is the
intersection of the Notarius limit, any PromptKit backend limit, and work made
available by the pipeline. Built-in OpenRouter profiles use PromptKit's
upstream backend limit; endpoint-only profiles have no PromptKit backend limit
and remain bounded by Notarius. For the configured local backend, a zero
**concurrency_limit** leaves only the Notarius scheduler as a call limit. A
positive value makes the effective active local-generation bound the smaller
of **total_llm** and that local limit, so a local limit of four permits no more
than four active local generations.
For a positive local limit, PromptKit owns its default waiting capacity and
admission behavior. When a PromptKit backend has admitted all active and queued
work, a new call fails as capacity exhaustion before generation. The adapter
maps that failure to Notarius's existing provider-neutral capacity error and
does not retry it. The calling stage's configured retry policy applies
normally, and the run fails if those attempts are exhausted. Caller
cancellation remains authoritative. Configuration contracts are documented
under [PromptKit profiles](config.md#promptkit-profiles) and
[concurrency](config.md#concurrency-output-cache-and-debug). Extract-worker
limits and actual provider-call limits are independent. Notarius writes local
filesystem state only; remote storage, archival, and retention automation are
outside the implemented CLI.
Every run has an effective prompt session used for provider routing and run
provenance. The generated default is stable for the same input module and raw
input bytes; use [**--session-id**](cli.md#run) only when intentionally grouping
different invocations. Both generated and explicit values can be visible to
providers, manifests, checkpoints, and requested debug bundles. Do not put
credentials or other secrets in an explicit session identifier; command-line
values are not a credential mechanism.

Some files were not shown because too many files have changed in this diff Show More