notarius/docs/internal/overview.md

Internal Overview

This document inventories the implemented Notarius components. Normative boundaries and dependency direction belong in Architecture; external behavior belongs in the CLI, Configuration, Operations, and integration contracts.

Execution Path

cmd/notarius delegates to internal/cli, the production composition root. The CLI loads configuration, builds the production catalogs and runtime collaborators, invokes internal/framework/pipeline, and places the logical output files returned by the runner. Diagnostics, checkpoints, and debug recorders are optional side-channel collaborators supplied at this boundary.

Pipeline execution is serial. Resolution produces a fixed ordered workflow and a sorted set of artifact lanes before the runner constructs any stage module.

Application Boundary

Package	Implemented responsibility
cmd/notarius	Executable entry point and process exit delegation.
internal/cli	Command parsing, config discovery, package-family registrar invocation, LLM client construction, reference materialization, workspace collaborator setup, durable writes, and user-facing results.

Core Packages

Package	Implemented responsibility
internal/core/artifacts	Run-manifest and provenance models.
internal/core/config	Defaults, YAML parsing, environment overrides, validation, redaction, and effective pipeline resolution.
internal/core/diagnostics	Scoped run directories, diagnostics writers, atomic writes, and retention decisions.
internal/core/source	Generic source documents, units, references, lookup, and validation.
internal/core/workspace	Effective workspace settings, confined paths and writes, checkpoint identity, and checkpoint manifest models.

Framework Packages

Package	Implemented responsibility
internal/framework/contracts	Stage, validator, reference, output, and structured-completion interfaces and data types.
internal/framework/pipeline	Registries, profile resolution, capability checks, reference materialization, validator-chain resolution, retries, orchestration, warnings, and manifest population.
internal/framework/validate	Shared validator decision and cardinality helpers.
internal/framework/llm	Scriptorium-backed structured completions, prompt/schema registration, scheduling, profile recording, and secret redaction.
internal/framework/checkpoint	Workspace-backed checkpoint loading, recording, and payload serialization.
internal/framework/debug	Workspace-backed framework and LLM debug recording.

Framework contracts carry raw stage results between implementations. The runner owns handoff provenance, validation sequencing, rejection handling, checkpoint and debug boundaries, and final manifest assembly.

Production Extensions

The canonical catalogs of user-selectable module and validator keys are in Configuration. The implemented module packages are:

Package	Implemented responsibility
internal/modules/input/seriatim	Parses the supported Seriatim transcript format into the generic source model.
internal/modules/chunk/generic	Splits ordered source units by unit count and overlap.
internal/modules/chunk/dnd/scenes	Produces contiguous D&D scene chunks from structured model output.
internal/modules/extract/dnd/spells	Produces source-grounded D&D spell-cast raw output.
internal/modules/merge/appendorder	Combines accepted extraction results in chunk order.
internal/modules/normalize/noop	Preserves accepted merged output.
internal/modules/output/json	Encodes manifests, lane payloads, warnings, and rejections as logical JSON files.

internal/modules/sharedassets composes shared prompt filesystems. internal/modules/sharedassets/dnd owns reusable D&D prompt fragments, reference declarations, prompt input assembly, and source-unit reference helpers.

Concrete validators live under internal/validators. Generic packages provide unconditional test decisions, JSON syntax validation, and JSON Schema validation. D&D spell packages provide shape, source-reference, and source-relatedness decisions, with spellpayload holding their shared parser and lookup helpers.

Production composition is grouped behind package-family registrars while the implementations remain in their current stage-oriented packages:

Package	Implemented responsibility
internal/modules/generic/register	Registers domain-neutral chunk, merge, normalize, output, and validator implementations.
internal/modules/seriatim/register	Registers the Seriatim input adapter.
internal/modules/dnd/register	Registers D&D modules, validators, default validator policy, and prompt/schema assets.

The CLI allocates the framework registries and asset registry, then invokes these registrars in generic, Seriatim, and D&D order.

Implementation details for all production extensions are in Module Internals.

Run-State Components

Surface	Implemented owners	Internal purpose
Durable output	Output module, pipeline runner, and CLI writer	Return logical consumer files and place them for a run.
Diagnostics	internal/core/diagnostics and internal/cli	Record redacted invocation, resolution, result, and failure inspection data.
Checkpoints	internal/framework/checkpoint and internal/core/workspace	Validate and serialize reusable stage outcomes.
Debug artifacts	internal/framework/debug and pipeline instrumentation	Capture sensitive framework-boundary and LLM-call material.

Physical layout, retention, recovery, and sensitive-data handling are defined in Operations. Concrete stage modules receive recorder interfaces and request data, not workspace paths.

Focused Documentation

• Pipeline Internals: resolution, execution, validation, retries, checkpoint/debug hooks, and result assembly.
• Module Internals: production modules, validators, assets, registration, and the contributor recipe for adding an extension.
• LLM Runtime: structured completion contracts, Scriptorium adapter, assets, scheduling, profile recording, and redaction.
• Diagnostics Internals: scoped writers, retention coordination, CLI failure flow, and path safety.