Add a staged implementation plan for MVP
This commit is contained in:
@@ -105,8 +105,8 @@ artifacts:
|
|||||||
|
|
||||||
## Limits
|
## Limits
|
||||||
|
|
||||||
This checkpoint implements only D&D spell-cast extraction. It does not
|
The current implementation covers only D&D spell-cast extraction. It does not
|
||||||
implement:
|
yet implement:
|
||||||
|
|
||||||
- item extraction;
|
- item extraction;
|
||||||
- NPC extraction;
|
- NPC extraction;
|
||||||
|
|||||||
@@ -1,70 +0,0 @@
|
|||||||
# Checkpoint 1: Core Contracts And Skeleton
|
|
||||||
|
|
||||||
## Status
|
|
||||||
|
|
||||||
This document describes planned work, not implemented behavior.
|
|
||||||
|
|
||||||
## Goal
|
|
||||||
|
|
||||||
Define the stable vocabulary and core interfaces that adapters, extractors,
|
|
||||||
validators, and runners will build against.
|
|
||||||
|
|
||||||
This checkpoint should produce a compileable Go repository with a minimal CLI
|
|
||||||
shell and contract-level tests. It does not need to process real input or
|
|
||||||
produce useful artifacts.
|
|
||||||
|
|
||||||
## Scope
|
|
||||||
|
|
||||||
In scope:
|
|
||||||
|
|
||||||
- Go module bootstrap;
|
|
||||||
- executable entrypoint;
|
|
||||||
- minimal CLI package;
|
|
||||||
- core source, artifact, manifest, and contract types;
|
|
||||||
- fake implementation tests proving the interfaces are usable.
|
|
||||||
|
|
||||||
Out of scope:
|
|
||||||
|
|
||||||
- real input modules;
|
|
||||||
- real extract modules;
|
|
||||||
- LLM provider calls;
|
|
||||||
- prompt or response-schema assets;
|
|
||||||
- diagnostics run directory;
|
|
||||||
- production config loading.
|
|
||||||
|
|
||||||
## Target End State
|
|
||||||
|
|
||||||
The repository should contain a compileable Go application shell and stable core
|
|
||||||
contract packages:
|
|
||||||
|
|
||||||
- `cmd/notarius` provides the executable entrypoint.
|
|
||||||
- `internal/cli` provides a minimal CLI shell.
|
|
||||||
- `internal/core/source` defines generic source documents, source units, source
|
|
||||||
references, and source validation helpers.
|
|
||||||
- `internal/core/artifacts` defines extractor-neutral artifact candidate,
|
|
||||||
approved artifact, rejected artifact, and run manifest types.
|
|
||||||
- `internal/framework/contracts` defines the adapter, extractor, validator, and
|
|
||||||
structured LLM interfaces used by later checkpoints.
|
|
||||||
|
|
||||||
The contracts should be proven with fake implementations in tests. Those tests
|
|
||||||
should demonstrate composition without real input modules, real extract modules,
|
|
||||||
LLM provider calls, prompt assets, or diagnostics infrastructure.
|
|
||||||
|
|
||||||
Implementation staging belongs in
|
|
||||||
[`implementation.md`](implementation.md).
|
|
||||||
|
|
||||||
## Done Criteria
|
|
||||||
|
|
||||||
- `go test ./...` passes.
|
|
||||||
- `go build ./cmd/notarius` passes.
|
|
||||||
- Core types and contracts exist in stable package locations.
|
|
||||||
- Tests prove fake implementations can compose at the type-contract level.
|
|
||||||
- No real Seriatim, D&D, LLM, or Audita-specific behavior has been added yet.
|
|
||||||
|
|
||||||
## Review Questions
|
|
||||||
|
|
||||||
- Are the interfaces small enough?
|
|
||||||
- Are source-format details absent from core packages?
|
|
||||||
- Are D&D concepts absent from framework and core packages?
|
|
||||||
- Is the shell compileable without placeholder behavior that will be hard to
|
|
||||||
unwind?
|
|
||||||
@@ -1,75 +0,0 @@
|
|||||||
# Checkpoint 2: Framework Composition
|
|
||||||
|
|
||||||
## Status
|
|
||||||
|
|
||||||
This document describes planned work, not implemented behavior.
|
|
||||||
|
|
||||||
## Goal
|
|
||||||
|
|
||||||
Prove the core contracts compose before adding real stage modules or portable
|
|
||||||
Audita infrastructure.
|
|
||||||
|
|
||||||
This checkpoint should produce a minimal runner that can execute fake registered
|
|
||||||
components from source input to artifact output in tests.
|
|
||||||
|
|
||||||
## Scope
|
|
||||||
|
|
||||||
In scope:
|
|
||||||
|
|
||||||
- input adapter registry;
|
|
||||||
- extractor registry;
|
|
||||||
- validator decision model;
|
|
||||||
- decision-cardinality checks;
|
|
||||||
- minimal runner;
|
|
||||||
- fake-component runner tests.
|
|
||||||
|
|
||||||
Out of scope:
|
|
||||||
|
|
||||||
- real input parsing;
|
|
||||||
- real input modules;
|
|
||||||
- real extract modules;
|
|
||||||
- pipeline-profile config loading;
|
|
||||||
- module capability validation;
|
|
||||||
- real LLM calls;
|
|
||||||
- prompt assets;
|
|
||||||
- response schema assets;
|
|
||||||
- diagnostics run directory;
|
|
||||||
- real D&D artifact schemas.
|
|
||||||
|
|
||||||
## Target End State
|
|
||||||
|
|
||||||
The repository should contain a minimal framework composition layer:
|
|
||||||
|
|
||||||
- `internal/framework/pipeline` registers and builds input adapter constructors
|
|
||||||
and extractor constructors by stable key.
|
|
||||||
- `internal/framework/validate` provides shared validator decision helpers and
|
|
||||||
cardinality checks.
|
|
||||||
- `internal/framework/pipeline` executes configured extractors against a
|
|
||||||
`SourceDocument`, applies validator chains, and returns approved and rejected
|
|
||||||
artifacts.
|
|
||||||
|
|
||||||
The runner should operate on already parsed source documents in this checkpoint.
|
|
||||||
Raw input parsing and concrete input module behavior remain deferred to the
|
|
||||||
Seriatim input module checkpoint.
|
|
||||||
|
|
||||||
Pipeline-profile resolution, module metadata, and capability validation are
|
|
||||||
deferred to checkpoint 3. This checkpoint only needs constructor registries and
|
|
||||||
minimal runner composition.
|
|
||||||
|
|
||||||
Implementation staging belongs in
|
|
||||||
[`implementation.md`](implementation.md).
|
|
||||||
|
|
||||||
## Done Criteria
|
|
||||||
|
|
||||||
- `go test ./...` passes.
|
|
||||||
- Fake adapter/extractor/validator registrations work in tests.
|
|
||||||
- The runner operates on `SourceDocument`, not transcript-specific structures.
|
|
||||||
- The runner does not import concrete D&D extract module packages.
|
|
||||||
|
|
||||||
## Review Questions
|
|
||||||
|
|
||||||
- Does the runner know only about sources, extractors, validators, and artifacts?
|
|
||||||
- Are registries simple enough to evolve?
|
|
||||||
- Are validation decisions expressive enough for deterministic and LLM-backed
|
|
||||||
validators?
|
|
||||||
- Is any domain-specific behavior creeping into framework packages?
|
|
||||||
@@ -1,181 +0,0 @@
|
|||||||
# Checkpoint 3: Pipeline Stages, Chunking, Merge, And Normalize
|
|
||||||
|
|
||||||
## Status
|
|
||||||
|
|
||||||
This document describes planned work, not implemented behavior.
|
|
||||||
|
|
||||||
## Goal
|
|
||||||
|
|
||||||
Make Notarius's application workflow first-class before adding real input
|
|
||||||
modules or extract modules.
|
|
||||||
|
|
||||||
The workflow should be:
|
|
||||||
|
|
||||||
```text
|
|
||||||
input -> chunk -> extract -> merge -> normalize -> output
|
|
||||||
```
|
|
||||||
|
|
||||||
Checkpoint 3 should define the contracts and minimal fake-tested framework
|
|
||||||
behavior for chunking, per-chunk extraction, merging, and normalization. It
|
|
||||||
should also introduce a fixture-driven walking skeleton that exercises the full
|
|
||||||
stage sequence with fake modules and a fake LLM client. It should not add real
|
|
||||||
input modules, real domain extract modules, LLM provider code, or production
|
|
||||||
output modules.
|
|
||||||
|
|
||||||
## Scope
|
|
||||||
|
|
||||||
In scope:
|
|
||||||
|
|
||||||
- source chunk model;
|
|
||||||
- chunker contract;
|
|
||||||
- extract-stage contract for extractors operating on chunks;
|
|
||||||
- merge-stage contract;
|
|
||||||
- normalize-stage contract;
|
|
||||||
- output-stage contract and fake output encoder for pipeline completeness;
|
|
||||||
- resolved pipeline definition types for a fixed-shape pipeline template;
|
|
||||||
- module binding and module metadata types, including flat capability strings;
|
|
||||||
- default application for `chunk`, lane `merge`, lane `normalize`, `output`,
|
|
||||||
and `llm_profile`;
|
|
||||||
- lane selection behavior equivalent to future `--only`;
|
|
||||||
- runner/pipeline updates that exercise these stages with fake components;
|
|
||||||
- generic append/chronological merge behavior for artifact candidates when
|
|
||||||
appropriate;
|
|
||||||
- fixture-driven walking skeleton test for
|
|
||||||
`input -> chunk -> extract -> merge -> normalize -> output`;
|
|
||||||
- fake `StructuredLLMClient` wired through a trivial extractor.
|
|
||||||
|
|
||||||
Out of scope:
|
|
||||||
|
|
||||||
- Seriatim parsing;
|
|
||||||
- D&D spell extraction;
|
|
||||||
- LLM provider calls;
|
|
||||||
- prompt assets;
|
|
||||||
- response schema assets;
|
|
||||||
- diagnostics run directory;
|
|
||||||
- production config file loading;
|
|
||||||
- real CLI command behavior;
|
|
||||||
- production output serialization or durable output writing.
|
|
||||||
|
|
||||||
## Target End State
|
|
||||||
|
|
||||||
The repository should contain explicit pipeline-stage contracts:
|
|
||||||
|
|
||||||
- `InputAdapter`: external source input to `SourceDocument`.
|
|
||||||
- `Chunker`: `SourceDocument` to ordered `SourceChunk` values.
|
|
||||||
- `Extractor`: `SourceChunk` to artifact candidates.
|
|
||||||
- `Merger`: per-chunk candidates to merged candidates.
|
|
||||||
- `Normalizer`: merged candidates to normalized candidates.
|
|
||||||
- `OutputEncoder`: final artifact bundle to bytes.
|
|
||||||
|
|
||||||
The repository should also contain a resolved pipeline model that represents:
|
|
||||||
|
|
||||||
- `pipeline_id`;
|
|
||||||
- shared input binding;
|
|
||||||
- shared chunk binding;
|
|
||||||
- selected artifact lanes;
|
|
||||||
- lane extract, merge, normalize, and validator bindings;
|
|
||||||
- output binding;
|
|
||||||
- resolved defaults;
|
|
||||||
- resolved pipeline digest input.
|
|
||||||
|
|
||||||
The runner should orchestrate fake implementations through chunk, extract,
|
|
||||||
merge, normalize, and approval/validation behavior in tests.
|
|
||||||
|
|
||||||
The checkpoint should include a fixture-driven walking skeleton that starts from
|
|
||||||
fixture input bytes and ends at encoded output bytes. The walking skeleton should
|
|
||||||
exercise the stage contracts, resolved pipeline model, module metadata,
|
|
||||||
capability validation, defaults, lane selection, and fake LLM client wiring. It
|
|
||||||
is contract coverage, not useful user-facing behavior.
|
|
||||||
|
|
||||||
## Design Intent
|
|
||||||
|
|
||||||
Chunking is a core application concern because many source documents, especially
|
|
||||||
transcripts, will be too large for a single LLM extraction pass.
|
|
||||||
|
|
||||||
Chunk processing may be serial or parallel depending on extractor needs. The
|
|
||||||
architecture should support both, while checkpoint 3 may execute
|
|
||||||
deterministically in series until a later checkpoint introduces concurrency.
|
|
||||||
|
|
||||||
Merge and normalize are separate stages:
|
|
||||||
|
|
||||||
- merge combines per-chunk extracted candidates into one stream or collection;
|
|
||||||
- normalize reconciles the merged output by checking duplicates, consistency,
|
|
||||||
ordering, identity resolution, or other cross-chunk concerns.
|
|
||||||
|
|
||||||
For some artifact types, merge may be generic append-in-source-order behavior.
|
|
||||||
For other artifact types, merge may be domain-specific. Normalization is where
|
|
||||||
deduplication and consistency checks should live.
|
|
||||||
|
|
||||||
## Processing Modes
|
|
||||||
|
|
||||||
The architecture should leave room for extractor-level processing modes:
|
|
||||||
|
|
||||||
- whole-document processing;
|
|
||||||
- serial chunk processing;
|
|
||||||
- parallel chunk processing.
|
|
||||||
|
|
||||||
Checkpoint 3 may execute chunks serially for deterministic behavior.
|
|
||||||
The contracts should not bake in a single-pass assumption or prevent later
|
|
||||||
parallel execution.
|
|
||||||
|
|
||||||
## Generic Merge Behavior
|
|
||||||
|
|
||||||
A generic merger should be able to concatenate candidates in deterministic
|
|
||||||
chunk order and candidate order. This is likely sufficient for early spell-cast
|
|
||||||
extraction, where chronological serialization is useful.
|
|
||||||
|
|
||||||
Domain-specific mergers may be added later when generic ordering is not enough.
|
|
||||||
|
|
||||||
## Generic Normalize Behavior
|
|
||||||
|
|
||||||
A no-op normalizer should be available as the default.
|
|
||||||
|
|
||||||
Domain-specific normalizers may later:
|
|
||||||
|
|
||||||
- deduplicate repeated extracted facts;
|
|
||||||
- resolve aliases;
|
|
||||||
- reconcile conflicting candidate fields;
|
|
||||||
- enforce chronological or source-reference consistency;
|
|
||||||
- attach normalization warnings.
|
|
||||||
|
|
||||||
## Walking Skeleton
|
|
||||||
|
|
||||||
The fixture-driven skeleton should prove the staged architecture continuously as
|
|
||||||
new contracts are added. It should remain deliberately small and use fake modules
|
|
||||||
only. It should validate module keys and flat capability requirements before
|
|
||||||
execution, using registry metadata rather than constructing modules. Capability
|
|
||||||
values should remain simple strings.
|
|
||||||
|
|
||||||
Implementation staging for the walking skeleton belongs in
|
|
||||||
[`implementation.md`](implementation.md).
|
|
||||||
|
|
||||||
## Done Criteria
|
|
||||||
|
|
||||||
- `go test ./...` passes.
|
|
||||||
- Pipeline-stage contracts are explicit and source/domain agnostic.
|
|
||||||
- A resolved pipeline profile model exists for fixed-shape pipeline templates.
|
|
||||||
- Pipeline defaults and lane selection are covered by fake tests.
|
|
||||||
- Module capability validation is covered by fake tests.
|
|
||||||
- Fake tests prove input source documents can be chunked, extracted, merged, and
|
|
||||||
normalized.
|
|
||||||
- A fixture-driven walking skeleton proves fake input, chunk, extract, merge,
|
|
||||||
normalize, and output modules compose end to end with a fake LLM client.
|
|
||||||
- Merge and normalize are distinct concepts in code and tests.
|
|
||||||
- The runner no longer implies whole-document-only extraction as the core
|
|
||||||
application model.
|
|
||||||
- No concrete input module, domain extract module, LLM provider, prompt,
|
|
||||||
response schema, diagnostics, config, or D&D artifact code is added.
|
|
||||||
|
|
||||||
## Review Questions
|
|
||||||
|
|
||||||
- Is the workflow clearly represented as input, chunk, extract, merge,
|
|
||||||
normalize, and output?
|
|
||||||
- Are merge and normalize cleanly separated?
|
|
||||||
- Does the resolved pipeline model avoid becoming a general-purpose workflow
|
|
||||||
engine?
|
|
||||||
- Are module capabilities simple flat strings?
|
|
||||||
- Does lane selection avoid creating ad hoc pipelines?
|
|
||||||
- Can a generic merger handle simple chronological artifact streams?
|
|
||||||
- Can a later domain-specific normalizer handle duplicates and consistency
|
|
||||||
without changing core runner contracts?
|
|
||||||
- Does the design allow serial and parallel chunk processing later?
|
|
||||||
@@ -1,114 +0,0 @@
|
|||||||
# Checkpoint 4: Portable Audita Infrastructure
|
|
||||||
|
|
||||||
## Status
|
|
||||||
|
|
||||||
This document describes planned work, not implemented behavior.
|
|
||||||
|
|
||||||
## Goal
|
|
||||||
|
|
||||||
Port or adapt reusable Audita infrastructure that directly supports Notarius
|
|
||||||
contracts while avoiding Audita's transcript-correction model.
|
|
||||||
|
|
||||||
This checkpoint should add reusable runtime plumbing, not real extraction
|
|
||||||
behavior.
|
|
||||||
|
|
||||||
## Scope
|
|
||||||
|
|
||||||
In scope:
|
|
||||||
|
|
||||||
- structured LLM client interface implementation;
|
|
||||||
- LLM scheduler;
|
|
||||||
- prompt registry pattern;
|
|
||||||
- response-schema registry pattern;
|
|
||||||
- diagnostics run directory pattern;
|
|
||||||
- config loading and validation for named pipeline profiles and implemented
|
|
||||||
runtime pieces.
|
|
||||||
|
|
||||||
Out of scope:
|
|
||||||
|
|
||||||
- correction proposals;
|
|
||||||
- replacement policies;
|
|
||||||
- transcript mutation;
|
|
||||||
- correction ledger terminology;
|
|
||||||
- Audita module or validator behavior;
|
|
||||||
- real D&D prompts or schemas;
|
|
||||||
- domain-specific prompt or schema assets;
|
|
||||||
- embedded built-in pipeline profiles.
|
|
||||||
|
|
||||||
## Target End State
|
|
||||||
|
|
||||||
The repository should contain reusable runtime infrastructure adapted from
|
|
||||||
Audita where it directly supports Notarius contracts:
|
|
||||||
|
|
||||||
- an OpenAI-compatible structured-output LLM client behind the existing
|
|
||||||
`StructuredLLMClient` interface;
|
|
||||||
- an LLM scheduler for bounded concurrency;
|
|
||||||
- an embedded response-schema registry pattern;
|
|
||||||
- an embedded prompt registry pattern;
|
|
||||||
- a diagnostics run directory pattern using extraction-oriented artifact names;
|
|
||||||
- config structs, loading, defaults, redaction, and validation for named
|
|
||||||
pipeline profiles.
|
|
||||||
|
|
||||||
Framework code should remain source-agnostic and domain-agnostic. Provider HTTP
|
|
||||||
details should stay inside the LLM runtime package. Prompt and schema registries
|
|
||||||
should use placeholder/test assets until real extractor prompts and schemas are
|
|
||||||
introduced by later checkpoints.
|
|
||||||
|
|
||||||
Config should support:
|
|
||||||
|
|
||||||
- `llm_profiles`;
|
|
||||||
- `pipelines.<pipeline_id>.input`;
|
|
||||||
- `pipelines.<pipeline_id>.chunk`;
|
|
||||||
- `pipelines.<pipeline_id>.artifacts.<lane>.extract`;
|
|
||||||
- lane `merge`, `normalize`, and validator settings;
|
|
||||||
- output module selection;
|
|
||||||
- inline module-binding object form and string shorthand;
|
|
||||||
- default `chunk`, `merge`, `normalize`, `output`, and `llm_profile`;
|
|
||||||
- selected pipeline ID and lane filtering for runtime use;
|
|
||||||
- concurrency;
|
|
||||||
- work directory;
|
|
||||||
- diagnostics retention.
|
|
||||||
|
|
||||||
Config loading should support the standard precedence model:
|
|
||||||
|
|
||||||
1. built-in defaults
|
|
||||||
2. configuration file
|
|
||||||
3. environment variables
|
|
||||||
4. CLI flags
|
|
||||||
|
|
||||||
Structural module selection should come from pipeline config. CLI flags may
|
|
||||||
override operational settings and artifact lane selection, but should not offer
|
|
||||||
ad hoc `--extractor` or `--chunker` wiring.
|
|
||||||
|
|
||||||
Config validation should fail fast for unknown pipeline IDs, unknown module
|
|
||||||
keys, missing required slots, missing capabilities, unknown LLM profiles, empty
|
|
||||||
artifact-lane sets, and invalid lane selections.
|
|
||||||
|
|
||||||
If the CLI shell is ready, the checkpoint should expose discovery/validation
|
|
||||||
commands for config and pipeline profiles:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
notarius config validate
|
|
||||||
notarius pipelines list
|
|
||||||
```
|
|
||||||
|
|
||||||
Implementation staging belongs in [`implementation.md`](implementation.md).
|
|
||||||
|
|
||||||
## Done Criteria
|
|
||||||
|
|
||||||
- `go test ./...` passes.
|
|
||||||
- Audita runtime pieces are adapted to Notarius package names and contracts.
|
|
||||||
- No correction proposal, replacement policy, transcript mutation, or correction
|
|
||||||
ledger code has been copied.
|
|
||||||
- Runtime tests cover secret redaction, schema registry lookup, prompt metadata,
|
|
||||||
and scheduler behavior where applicable.
|
|
||||||
- Config tests cover named pipeline profiles, defaults, lane selection,
|
|
||||||
capability validation, and resolved pipeline digesting.
|
|
||||||
|
|
||||||
## Review Questions
|
|
||||||
|
|
||||||
- Did we copy only reusable infrastructure?
|
|
||||||
- Do provider-specific types stay behind adapter/runtime boundaries?
|
|
||||||
- Are diagnostics names and report concepts extraction-oriented?
|
|
||||||
- Is config limited to named pipeline profiles and implemented behavior?
|
|
||||||
- Are structural pipeline changes kept out of ad hoc CLI flags?
|
|
||||||
@@ -1,107 +0,0 @@
|
|||||||
# Checkpoint 5: Seriatim Input Module
|
|
||||||
|
|
||||||
## Status
|
|
||||||
|
|
||||||
This document describes planned work, not implemented behavior.
|
|
||||||
|
|
||||||
## Goal
|
|
||||||
|
|
||||||
Add the first real input source while keeping transcript-specific behavior
|
|
||||||
isolated inside an input-stage module.
|
|
||||||
|
|
||||||
This checkpoint should allow Seriatim minimal transcript JSON to become a
|
|
||||||
generic `SourceDocument`.
|
|
||||||
|
|
||||||
## Scope
|
|
||||||
|
|
||||||
In scope:
|
|
||||||
|
|
||||||
- `internal/modules/input/seriatim`;
|
|
||||||
- parser for Seriatim minimal output JSON;
|
|
||||||
- mapping into `SourceDocument` and `SourceUnit`;
|
|
||||||
- source-document validation;
|
|
||||||
- input adapter registry wiring;
|
|
||||||
- module metadata/capabilities for pipeline validation;
|
|
||||||
- fixtures and tests;
|
|
||||||
- config compatibility through named pipeline profiles.
|
|
||||||
|
|
||||||
Out of scope:
|
|
||||||
|
|
||||||
- D&D extraction;
|
|
||||||
- LLM extraction calls;
|
|
||||||
- transcript-specific behavior in runner/core packages;
|
|
||||||
- support for every possible Seriatim schema variant.
|
|
||||||
|
|
||||||
## Target End State
|
|
||||||
|
|
||||||
The repository should contain a real Seriatim input-stage module that translates
|
|
||||||
Seriatim minimal transcript JSON into the generic source model.
|
|
||||||
|
|
||||||
The Seriatim module should be registered under the stable input adapter key
|
|
||||||
`seriatim`. It should be selectable through the existing input adapter registry
|
|
||||||
and through pipeline-profile resolution when a profile binds `input: seriatim`.
|
|
||||||
|
|
||||||
The module should accept the Seriatim minimal output shape:
|
|
||||||
|
|
||||||
- top-level `metadata`;
|
|
||||||
- top-level `segments`;
|
|
||||||
- segment `id`;
|
|
||||||
- segment `start`;
|
|
||||||
- segment `end`;
|
|
||||||
- segment `speaker`;
|
|
||||||
- segment `text`.
|
|
||||||
|
|
||||||
The module should map Seriatim data into generic source values:
|
|
||||||
|
|
||||||
- segment `id` becomes `SourceUnit.ID`;
|
|
||||||
- segment `text` becomes `SourceUnit.Text`;
|
|
||||||
- the document and unit kind strings identify transcript-like source material
|
|
||||||
without adding transcript-specific fields or types to core packages;
|
|
||||||
- `speaker`, `start`, and `end` become source-unit metadata;
|
|
||||||
- top-level Seriatim metadata becomes source-document metadata;
|
|
||||||
- the resulting source document passes core source validation.
|
|
||||||
|
|
||||||
The module should reject invalid Seriatim input with clear module-specific
|
|
||||||
errors. Validation should cover:
|
|
||||||
|
|
||||||
- valid JSON;
|
|
||||||
- required top-level metadata and segments;
|
|
||||||
- required segment fields;
|
|
||||||
- unique segment IDs;
|
|
||||||
- non-empty segment text;
|
|
||||||
- valid start and end values.
|
|
||||||
|
|
||||||
The module should declare flat capabilities for pipeline validation. Initial
|
|
||||||
capabilities should describe transcript-oriented source properties preserved by
|
|
||||||
the adapter, including speaker and timestamp metadata.
|
|
||||||
|
|
||||||
Implementation staging belongs in
|
|
||||||
[`implementation.md`](implementation.md).
|
|
||||||
|
|
||||||
## Fixtures And Tests
|
|
||||||
|
|
||||||
The checkpoint should add synthetic fixtures and focused tests for:
|
|
||||||
|
|
||||||
- valid Seriatim minimal transcript;
|
|
||||||
- malformed JSON;
|
|
||||||
- missing metadata;
|
|
||||||
- missing or duplicate segment IDs;
|
|
||||||
- empty segment text;
|
|
||||||
- source-reference compatibility with generated unit IDs.
|
|
||||||
|
|
||||||
## Done Criteria
|
|
||||||
|
|
||||||
- `go test ./...` passes.
|
|
||||||
- Seriatim minimal transcript JSON maps into `SourceDocument`.
|
|
||||||
- Transcript fields do not appear in core runner contracts.
|
|
||||||
- The input module is selectable through the registry and pipeline-profile
|
|
||||||
configuration when config support exists.
|
|
||||||
- The input module declares capabilities needed for pipeline validation.
|
|
||||||
- Tests prove transcript-specific assumptions are isolated to the input module.
|
|
||||||
|
|
||||||
## Review Questions
|
|
||||||
|
|
||||||
- Are segment, speaker, and timestamp assumptions contained inside the input module?
|
|
||||||
- Are unit IDs stable and suitable for source references?
|
|
||||||
- Does the input module preserve enough metadata for transcript-oriented output later?
|
|
||||||
- Should the input module accept only Seriatim minimal output for now?
|
|
||||||
@@ -1,120 +0,0 @@
|
|||||||
# Checkpoint 6: D&D Spells Extractor
|
|
||||||
|
|
||||||
## Status
|
|
||||||
|
|
||||||
This document records the target scope for checkpoint 6. The implemented
|
|
||||||
integration contract is documented in
|
|
||||||
[`docs/integrations/dnd-spells.md`](../integrations/dnd-spells.md).
|
|
||||||
|
|
||||||
## Goal
|
|
||||||
|
|
||||||
Implement the first useful extract-stage module: D&D spell casts from a
|
|
||||||
Seriatim transcript source document.
|
|
||||||
|
|
||||||
This checkpoint should produce the first meaningful vertical slice from real
|
|
||||||
source input to validated artifact output.
|
|
||||||
|
|
||||||
## Scope
|
|
||||||
|
|
||||||
In scope:
|
|
||||||
|
|
||||||
- D&D spell artifact schema and Go structs;
|
|
||||||
- structured response schema asset;
|
|
||||||
- prompt assets;
|
|
||||||
- `internal/modules/extract/dnd/spells`;
|
|
||||||
- module metadata/capability requirements for pipeline validation;
|
|
||||||
- source-reference and schema validators in the extractor chain;
|
|
||||||
- fake LLM tests;
|
|
||||||
- runner-level integration tests with fake infrastructure.
|
|
||||||
|
|
||||||
Out of scope:
|
|
||||||
|
|
||||||
- D&D item extraction;
|
|
||||||
- NPC extraction;
|
|
||||||
- combat extraction;
|
|
||||||
- cross-slice deduplication beyond simple deterministic merging;
|
|
||||||
- broad D&D rules validation;
|
|
||||||
- a `notarius run` command.
|
|
||||||
|
|
||||||
## Target End State
|
|
||||||
|
|
||||||
The repository should contain a real D&D spells extract-stage module at
|
|
||||||
`internal/modules/extract/dnd/spells`.
|
|
||||||
|
|
||||||
The spells module should be registered under the stable extractor key
|
|
||||||
`dnd/spells`. It should be selectable as a named artifact lane in pipeline
|
|
||||||
configuration, for example a lane named `spells` whose extractor module is
|
|
||||||
`dnd/spells`.
|
|
||||||
|
|
||||||
The module should translate generic source chunks into spell-cast artifact
|
|
||||||
candidates with this spell payload:
|
|
||||||
|
|
||||||
- `caster`;
|
|
||||||
- `spell`;
|
|
||||||
- `effect`;
|
|
||||||
- `narrative_description`.
|
|
||||||
|
|
||||||
The LLM structured response should also include source references for each
|
|
||||||
spell cast. The extractor should copy those references into the generic
|
|
||||||
artifact envelope rather than duplicating source references inside the durable
|
|
||||||
spell payload.
|
|
||||||
|
|
||||||
The caster is the in-world character or creature casting the spell, not the
|
|
||||||
table speaker. Speaker metadata from transcript source units may be used as
|
|
||||||
optional prompt context when present, but it should not be a required or durable
|
|
||||||
field in the spell payload.
|
|
||||||
|
|
||||||
Prompts should:
|
|
||||||
|
|
||||||
- describe the generic source-unit input format;
|
|
||||||
- explain that source references must use source-unit IDs exactly;
|
|
||||||
- avoid relying on transcript-specific fields except as optional metadata;
|
|
||||||
- request only D&D spell-cast artifacts.
|
|
||||||
|
|
||||||
The extractor should attach deterministic validators by default. Validation
|
|
||||||
should cover:
|
|
||||||
|
|
||||||
- spell payload shape;
|
|
||||||
- non-empty required spell fields;
|
|
||||||
- at least one source reference per spell cast;
|
|
||||||
- source references that validate against the source document.
|
|
||||||
|
|
||||||
The module should declare flat capabilities for pipeline validation. Initial
|
|
||||||
capabilities should require chunked transcript source material and provide a
|
|
||||||
D&D spell-cast artifact capability.
|
|
||||||
|
|
||||||
Implementation staging belongs in
|
|
||||||
[`implementation.md`](implementation.md).
|
|
||||||
|
|
||||||
## Fixtures And Tests
|
|
||||||
|
|
||||||
The checkpoint should add synthetic fixtures and focused tests for:
|
|
||||||
|
|
||||||
- successful spell extraction with a fake structured LLM client;
|
|
||||||
- empty spell-cast results;
|
|
||||||
- malformed structured output handling;
|
|
||||||
- invalid source-reference rejection;
|
|
||||||
- stable output ordering;
|
|
||||||
- pipeline-profile selection of a `spells` artifact lane;
|
|
||||||
- runner integration from Seriatim input through the spells extractor using
|
|
||||||
fake chunk and output modules.
|
|
||||||
|
|
||||||
## Done Criteria
|
|
||||||
|
|
||||||
- `go test ./...` passes.
|
|
||||||
- Seriatim input can flow through the runner into the D&D spells extractor.
|
|
||||||
- Spell artifacts include valid source references.
|
|
||||||
- D&D concepts are contained in extract module/artifact packages and docs.
|
|
||||||
- The spells module can be selected as a named artifact lane in pipeline
|
|
||||||
configuration.
|
|
||||||
- The first meaningful vertical slice is available through tests.
|
|
||||||
- No CLI `run` behavior is documented or implemented until the CLI path exists.
|
|
||||||
|
|
||||||
## Review Questions
|
|
||||||
|
|
||||||
- Is the spell extract module domain-specific without making the framework
|
|
||||||
D&D-specific?
|
|
||||||
- Are source references valid and useful for downstream validation?
|
|
||||||
- Is prompt/schema ownership clear?
|
|
||||||
- Does this vertical slice reveal contract changes needed before adding items,
|
|
||||||
NPCs, or combat?
|
|
||||||
File diff suppressed because it is too large
Load Diff
@@ -546,10 +546,10 @@ Reuse these architectural patterns:
|
|||||||
- validator decision cardinality and deterministic validator ordering;
|
- validator decision cardinality and deterministic validator ordering;
|
||||||
- CLI tests and fixture-driven integration tests.
|
- CLI tests and fixture-driven integration tests.
|
||||||
|
|
||||||
The fixture-driven integration-test pattern should begin at checkpoint 3 with a
|
The fixture-driven integration-test pattern should remain part of the codebase:
|
||||||
walking skeleton over fake modules and a fake LLM client. Later checkpoints
|
walking skeleton tests over fake modules and fake LLM clients should be
|
||||||
should replace fake pieces with real Seriatim, runtime, and D&D modules without
|
preserved as real Seriatim, runtime, and D&D modules are added, so the
|
||||||
losing that end-to-end contract coverage.
|
end-to-end contract coverage is not lost.
|
||||||
|
|
||||||
Avoid copying these Audita concepts directly:
|
Avoid copying these Audita concepts directly:
|
||||||
|
|
||||||
@@ -565,23 +565,13 @@ extraction-report concepts.
|
|||||||
|
|
||||||
## Checkpoint Roadmap
|
## Checkpoint Roadmap
|
||||||
|
|
||||||
The initial implementation should proceed through six coherent checkpoints.
|
The initial six checkpoint roadmap has been implemented and retired. The
|
||||||
Each checkpoint should leave the repository in a reviewable state, with the code
|
checkpoint files have been removed from `docs/roadmap/` so the active roadmap
|
||||||
compiling and targeted tests covering the newly introduced contracts or behavior.
|
does not compete with completed implementation history.
|
||||||
|
|
||||||
1. [Core Contracts And Skeleton](1-core-contracts-and-skeleton.md)
|
The remaining work needed to reach the first functional MVP is tracked in
|
||||||
2. [Framework Composition](2-framework-composition.md)
|
[`mvp.md`](mvp.md). Future staged implementation plans should be written to
|
||||||
3. [Pipeline Stages, Chunking, Merge, And Normalize](3-pipeline-stages-chunking-merge-normalize.md)
|
[`implementation.md`](implementation.md) from that active MVP roadmap.
|
||||||
4. [Portable Audita Infrastructure](4-portable-audita-infrastructure.md)
|
|
||||||
5. [Seriatim Input Module](5-seriatim-input-module.md)
|
|
||||||
6. [D&D Spells Extractor](6-dnd-spells-extractor.md)
|
|
||||||
|
|
||||||
The first contract-level walking skeleton should arrive at checkpoint 3: fixture
|
|
||||||
input through fake input, chunk, extract, merge, normalize, and output modules
|
|
||||||
with a fake LLM client. The first useful vertical slice should arrive at
|
|
||||||
checkpoint 6: Seriatim transcript input to validated D&D spell artifact output.
|
|
||||||
Earlier checkpoints remain contract-first and may not produce useful user output
|
|
||||||
yet.
|
|
||||||
|
|
||||||
## Architecture Decisions
|
## Architecture Decisions
|
||||||
|
|
||||||
|
|||||||
249
docs/roadmap/mvp.md
Normal file
249
docs/roadmap/mvp.md
Normal file
@@ -0,0 +1,249 @@
|
|||||||
|
# MVP Roadmap
|
||||||
|
|
||||||
|
## Status
|
||||||
|
|
||||||
|
This is the active roadmap for reaching the first functional Notarius MVP.
|
||||||
|
|
||||||
|
The previous numbered checkpoint roadmaps have been implemented and retired.
|
||||||
|
This document captures the remaining work needed to turn the implemented
|
||||||
|
architecture into a usable MVP, with the current architectural review findings
|
||||||
|
folded in as first-class work.
|
||||||
|
|
||||||
|
Implementation staging belongs in [`implementation.md`](implementation.md).
|
||||||
|
|
||||||
|
## MVP Goal
|
||||||
|
|
||||||
|
The MVP should let a user run Notarius against a Seriatim minimal transcript
|
||||||
|
JSON file, select a configured pipeline profile, extract D&D spell-cast
|
||||||
|
artifacts with an LLM-backed extractor, validate those artifacts, and write
|
||||||
|
durable JSON output plus diagnostics.
|
||||||
|
|
||||||
|
The intended command shape is:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
notarius run dnd-session --input session-014.json
|
||||||
|
notarius run dnd-session --input session-014.json --only spells
|
||||||
|
```
|
||||||
|
|
||||||
|
The MVP remains intentionally narrow:
|
||||||
|
|
||||||
|
- one production input module: `seriatim`;
|
||||||
|
- one production extract module: `dnd/spells`;
|
||||||
|
- one generic chunk module sufficient for transcript-scale processing;
|
||||||
|
- generic append-order merge;
|
||||||
|
- generic noop normalization;
|
||||||
|
- generic JSON output;
|
||||||
|
- config-driven pipeline profiles;
|
||||||
|
- OpenAI-compatible structured LLM execution through the existing LLM client.
|
||||||
|
|
||||||
|
## MVP Work Areas
|
||||||
|
|
||||||
|
### Framework/Domain Asset Boundaries
|
||||||
|
|
||||||
|
This is the highest-priority remaining architecture correction.
|
||||||
|
|
||||||
|
Framework packages must remain source-agnostic and domain-agnostic. D&D spell
|
||||||
|
prompt assets, response schema assets, prompt IDs, response schema keys, and
|
||||||
|
domain-specific prompt/schema tests should not live in `internal/framework/llm`
|
||||||
|
or `internal/framework/prompt`.
|
||||||
|
|
||||||
|
Target state:
|
||||||
|
|
||||||
|
- `internal/framework/llm` provides generic structured-output client,
|
||||||
|
scheduler, schema metadata, schema loading, and schema lookup/registration
|
||||||
|
primitives.
|
||||||
|
- `internal/framework/prompt` provides generic prompt metadata, prompt loading,
|
||||||
|
rendering, hardening, and lookup/registration primitives.
|
||||||
|
- `internal/modules/extract/dnd/spells` owns the D&D spell prompt assets,
|
||||||
|
response schema assets, stable prompt ID, stable response schema key, and
|
||||||
|
module-specific prompt/schema tests.
|
||||||
|
- Framework tests use placeholder/test assets only.
|
||||||
|
- The D&D spells extractor depends on generic framework APIs, not
|
||||||
|
framework-owned D&D constants.
|
||||||
|
|
||||||
|
This work should not change the external artifact shape or module key. It is an
|
||||||
|
ownership and package-boundary correction.
|
||||||
|
|
||||||
|
### Production Application Catalog Wiring
|
||||||
|
|
||||||
|
The implemented modules and registries are currently exercised mostly through
|
||||||
|
tests that inject catalogs. The MVP needs a production assembly point that
|
||||||
|
builds the catalog and stage registries used by real CLI commands.
|
||||||
|
|
||||||
|
Target state:
|
||||||
|
|
||||||
|
- a small app-level package or CLI wiring function constructs the production
|
||||||
|
`pipeline.ModuleCatalog`;
|
||||||
|
- production wiring registers `seriatim`;
|
||||||
|
- production wiring registers `dnd/spells`;
|
||||||
|
- production wiring registers the default `generic`, `appendorder`, `noop`, and
|
||||||
|
`json` modules;
|
||||||
|
- `notarius config validate --pipeline ...` validates real configured
|
||||||
|
pipelines without test-only catalog injection;
|
||||||
|
- `notarius pipelines list` reports production-registered modules where useful
|
||||||
|
for discoverability.
|
||||||
|
|
||||||
|
The production wiring should not move domain behavior into the CLI. The CLI may
|
||||||
|
compose modules, but module packages should continue to own their own behavior
|
||||||
|
and metadata.
|
||||||
|
|
||||||
|
### Default Production Stage Modules
|
||||||
|
|
||||||
|
Pipeline defaults are already part of the architecture:
|
||||||
|
|
||||||
|
- `chunk: generic`;
|
||||||
|
- `merge: appendorder`;
|
||||||
|
- `normalize: noop`;
|
||||||
|
- `output: json`.
|
||||||
|
|
||||||
|
The MVP should make those defaults real production modules rather than
|
||||||
|
test-only conveniences.
|
||||||
|
|
||||||
|
Target state:
|
||||||
|
|
||||||
|
- `generic` chunking creates ordered chunks over generic source units and is
|
||||||
|
configurable enough for transcript MVP use;
|
||||||
|
- `appendorder` merge serializes artifact candidates in deterministic source
|
||||||
|
and chunk order;
|
||||||
|
- `noop` normalize passes merged artifacts through unchanged while preserving
|
||||||
|
diagnostics;
|
||||||
|
- `json` output encodes approved artifacts, rejected artifacts, warnings,
|
||||||
|
manifest data, and relevant run metadata in a durable JSON shape;
|
||||||
|
- each default module declares module specs and capabilities compatible with
|
||||||
|
pipeline validation;
|
||||||
|
- default modules are registered by production app wiring.
|
||||||
|
|
||||||
|
If a default module remains implemented in `internal/framework/pipeline`, its
|
||||||
|
production registration still needs to be explicit and discoverable. If its
|
||||||
|
logic grows beyond a small generic helper, move it under `internal/modules`.
|
||||||
|
|
||||||
|
### `notarius run`
|
||||||
|
|
||||||
|
The MVP needs a functional run command that drives the already-implemented
|
||||||
|
pipeline runner.
|
||||||
|
|
||||||
|
Target state:
|
||||||
|
|
||||||
|
- command shape:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
notarius run <pipeline-id> --input path/to/source.json
|
||||||
|
notarius run <pipeline-id> --input path/to/source.json --only spells
|
||||||
|
```
|
||||||
|
|
||||||
|
- required flags and arguments produce clear usage errors;
|
||||||
|
- `--config` selects the config file;
|
||||||
|
- `--only` filters artifact lanes without changing structural pipeline config;
|
||||||
|
- operational overrides may cover output directory, work directory,
|
||||||
|
concurrency, and LLM profile/model settings where already supported by config;
|
||||||
|
- structural stage selection remains config-driven;
|
||||||
|
- the command parses input through the configured input adapter;
|
||||||
|
- the command constructs the configured LLM client and scheduler;
|
||||||
|
- the command invokes the pipeline runner;
|
||||||
|
- the command writes durable output and diagnostics;
|
||||||
|
- failures return stable non-zero exit codes and useful error messages.
|
||||||
|
|
||||||
|
The command should be covered by fixture-driven CLI tests with fake LLM behavior
|
||||||
|
where network calls would otherwise be required.
|
||||||
|
|
||||||
|
### MVP Output And Diagnostics Behavior
|
||||||
|
|
||||||
|
The MVP should produce inspectable files that are stable enough for downstream
|
||||||
|
experiments, without pretending to be a final public artifact contract.
|
||||||
|
|
||||||
|
Target state:
|
||||||
|
|
||||||
|
- output path behavior is deterministic and documented in code/tests;
|
||||||
|
- JSON output includes approved artifacts grouped or ordered predictably;
|
||||||
|
- each artifact includes its generic source references;
|
||||||
|
- rejected artifacts and validation decisions remain inspectable;
|
||||||
|
- output-stage warnings remain out-of-band from the durable artifact payload but
|
||||||
|
are captured for CLI reporting and diagnostics;
|
||||||
|
- run manifest data includes source digest, resolved pipeline digest, relevant
|
||||||
|
model/profile information, prompt/schema identifiers, and validation status;
|
||||||
|
- diagnostics redact secrets and include the resolved effective configuration
|
||||||
|
needed to debug a run.
|
||||||
|
|
||||||
|
### MVP Fixtures And Acceptance Tests
|
||||||
|
|
||||||
|
The MVP should be continuously testable without external services.
|
||||||
|
|
||||||
|
Target state:
|
||||||
|
|
||||||
|
- maintained Seriatim transcript fixture for the D&D spells MVP;
|
||||||
|
- maintained minimal config fixture for the MVP pipeline;
|
||||||
|
- fake LLM path for deterministic CLI and runner tests;
|
||||||
|
- config validation tests using the production catalog;
|
||||||
|
- `notarius run` fixture test from input file to output JSON;
|
||||||
|
- failure tests for missing config, unknown pipeline, invalid input,
|
||||||
|
invalid lane selection, LLM failure, and validation rejection;
|
||||||
|
- `go test ./...` is sufficient to exercise the MVP path without network
|
||||||
|
access.
|
||||||
|
|
||||||
|
### Documentation Pass Preparation
|
||||||
|
|
||||||
|
The full documentation pass is intentionally deferred until MVP functionality
|
||||||
|
exists. It should happen before tagging alpha `0.1.0`.
|
||||||
|
|
||||||
|
The MVP implementation should still leave clear hooks for the documentation
|
||||||
|
rewrite:
|
||||||
|
|
||||||
|
- command behavior should be stable enough to document in `docs/cli.md`;
|
||||||
|
- config behavior should be stable enough to document in `docs/config.md`;
|
||||||
|
- output behavior should be stable enough to document in integration docs;
|
||||||
|
- examples should be generated from or validated against maintained fixtures
|
||||||
|
where practical.
|
||||||
|
|
||||||
|
## Out Of Scope For MVP
|
||||||
|
|
||||||
|
- D&D item extraction;
|
||||||
|
- NPC extraction;
|
||||||
|
- combat extraction;
|
||||||
|
- D&D rules validation beyond the spell extractor's deterministic checks;
|
||||||
|
- Markdown or Obsidian input;
|
||||||
|
- cross-lane entity normalization;
|
||||||
|
- cross-chunk semantic deduplication beyond whatever a simple normalizer can
|
||||||
|
safely support;
|
||||||
|
- a general DAG or workflow engine;
|
||||||
|
- ad hoc CLI flags for structural module selection;
|
||||||
|
- release-quality documentation before the MVP behavior is implemented.
|
||||||
|
|
||||||
|
## MVP Done Criteria
|
||||||
|
|
||||||
|
- D&D prompt and response schema assets are owned by the D&D spells module, not
|
||||||
|
by framework packages.
|
||||||
|
- Production CLI commands use a real app catalog rather than test-injected
|
||||||
|
module catalogs.
|
||||||
|
- A config profile can bind `input: seriatim` and an artifact lane with
|
||||||
|
`extract: dnd/spells`.
|
||||||
|
- Default `generic`, `appendorder`, `noop`, and `json` modules resolve through
|
||||||
|
production wiring.
|
||||||
|
- `notarius config validate --config <file> --pipeline <id>` works with the
|
||||||
|
MVP config.
|
||||||
|
- `notarius pipelines list --config <file>` works with the MVP config.
|
||||||
|
- `notarius run <pipeline-id> --input <file>` reads a Seriatim transcript,
|
||||||
|
extracts D&D spell artifacts, validates them, and writes JSON output.
|
||||||
|
- `notarius run <pipeline-id> --input <file> --only spells` runs only the
|
||||||
|
selected artifact lane.
|
||||||
|
- The run manifest records source digest, resolved pipeline digest, LLM profile
|
||||||
|
and model, prompt/schema identifiers, and validation status.
|
||||||
|
- Output warnings are available to CLI/diagnostics without becoming artifact
|
||||||
|
payload fields.
|
||||||
|
- MVP fixture tests cover the full path without network access.
|
||||||
|
- `go test ./...`, `go vet ./...`, and `go build ./cmd/notarius` pass.
|
||||||
|
|
||||||
|
## Deferred Documentation Pass
|
||||||
|
|
||||||
|
After MVP behavior is implemented and before alpha `0.1.0`, complete a full
|
||||||
|
documentation pass/rewrite. That pass should move implemented behavior out of
|
||||||
|
roadmap documents and into canonical docs required by
|
||||||
|
[`../policy/documentation.md`](../policy/documentation.md), including at least:
|
||||||
|
|
||||||
|
- `README.md`;
|
||||||
|
- `docs/cli.md`;
|
||||||
|
- `docs/config.md`;
|
||||||
|
- `docs/operations.md`, if diagnostics/run recovery behavior warrants it;
|
||||||
|
- `docs/internal/` architecture and package-boundary docs;
|
||||||
|
- `docs/integrations/` updates for Seriatim input, D&D spell artifacts, and
|
||||||
|
JSON output;
|
||||||
|
- maintained `examples/` files.
|
||||||
Reference in New Issue
Block a user