Add a staged implementation plan for MVP
This commit is contained in:
@@ -1,70 +0,0 @@
|
||||
# Checkpoint 1: Core Contracts And Skeleton
|
||||
|
||||
## Status
|
||||
|
||||
This document describes planned work, not implemented behavior.
|
||||
|
||||
## Goal
|
||||
|
||||
Define the stable vocabulary and core interfaces that adapters, extractors,
|
||||
validators, and runners will build against.
|
||||
|
||||
This checkpoint should produce a compileable Go repository with a minimal CLI
|
||||
shell and contract-level tests. It does not need to process real input or
|
||||
produce useful artifacts.
|
||||
|
||||
## Scope
|
||||
|
||||
In scope:
|
||||
|
||||
- Go module bootstrap;
|
||||
- executable entrypoint;
|
||||
- minimal CLI package;
|
||||
- core source, artifact, manifest, and contract types;
|
||||
- fake implementation tests proving the interfaces are usable.
|
||||
|
||||
Out of scope:
|
||||
|
||||
- real input modules;
|
||||
- real extract modules;
|
||||
- LLM provider calls;
|
||||
- prompt or response-schema assets;
|
||||
- diagnostics run directory;
|
||||
- production config loading.
|
||||
|
||||
## Target End State
|
||||
|
||||
The repository should contain a compileable Go application shell and stable core
|
||||
contract packages:
|
||||
|
||||
- `cmd/notarius` provides the executable entrypoint.
|
||||
- `internal/cli` provides a minimal CLI shell.
|
||||
- `internal/core/source` defines generic source documents, source units, source
|
||||
references, and source validation helpers.
|
||||
- `internal/core/artifacts` defines extractor-neutral artifact candidate,
|
||||
approved artifact, rejected artifact, and run manifest types.
|
||||
- `internal/framework/contracts` defines the adapter, extractor, validator, and
|
||||
structured LLM interfaces used by later checkpoints.
|
||||
|
||||
The contracts should be proven with fake implementations in tests. Those tests
|
||||
should demonstrate composition without real input modules, real extract modules,
|
||||
LLM provider calls, prompt assets, or diagnostics infrastructure.
|
||||
|
||||
Implementation staging belongs in
|
||||
[`implementation.md`](implementation.md).
|
||||
|
||||
## Done Criteria
|
||||
|
||||
- `go test ./...` passes.
|
||||
- `go build ./cmd/notarius` passes.
|
||||
- Core types and contracts exist in stable package locations.
|
||||
- Tests prove fake implementations can compose at the type-contract level.
|
||||
- No real Seriatim, D&D, LLM, or Audita-specific behavior has been added yet.
|
||||
|
||||
## Review Questions
|
||||
|
||||
- Are the interfaces small enough?
|
||||
- Are source-format details absent from core packages?
|
||||
- Are D&D concepts absent from framework and core packages?
|
||||
- Is the shell compileable without placeholder behavior that will be hard to
|
||||
unwind?
|
||||
@@ -1,75 +0,0 @@
|
||||
# Checkpoint 2: Framework Composition
|
||||
|
||||
## Status
|
||||
|
||||
This document describes planned work, not implemented behavior.
|
||||
|
||||
## Goal
|
||||
|
||||
Prove the core contracts compose before adding real stage modules or portable
|
||||
Audita infrastructure.
|
||||
|
||||
This checkpoint should produce a minimal runner that can execute fake registered
|
||||
components from source input to artifact output in tests.
|
||||
|
||||
## Scope
|
||||
|
||||
In scope:
|
||||
|
||||
- input adapter registry;
|
||||
- extractor registry;
|
||||
- validator decision model;
|
||||
- decision-cardinality checks;
|
||||
- minimal runner;
|
||||
- fake-component runner tests.
|
||||
|
||||
Out of scope:
|
||||
|
||||
- real input parsing;
|
||||
- real input modules;
|
||||
- real extract modules;
|
||||
- pipeline-profile config loading;
|
||||
- module capability validation;
|
||||
- real LLM calls;
|
||||
- prompt assets;
|
||||
- response schema assets;
|
||||
- diagnostics run directory;
|
||||
- real D&D artifact schemas.
|
||||
|
||||
## Target End State
|
||||
|
||||
The repository should contain a minimal framework composition layer:
|
||||
|
||||
- `internal/framework/pipeline` registers and builds input adapter constructors
|
||||
and extractor constructors by stable key.
|
||||
- `internal/framework/validate` provides shared validator decision helpers and
|
||||
cardinality checks.
|
||||
- `internal/framework/pipeline` executes configured extractors against a
|
||||
`SourceDocument`, applies validator chains, and returns approved and rejected
|
||||
artifacts.
|
||||
|
||||
The runner should operate on already parsed source documents in this checkpoint.
|
||||
Raw input parsing and concrete input module behavior remain deferred to the
|
||||
Seriatim input module checkpoint.
|
||||
|
||||
Pipeline-profile resolution, module metadata, and capability validation are
|
||||
deferred to checkpoint 3. This checkpoint only needs constructor registries and
|
||||
minimal runner composition.
|
||||
|
||||
Implementation staging belongs in
|
||||
[`implementation.md`](implementation.md).
|
||||
|
||||
## Done Criteria
|
||||
|
||||
- `go test ./...` passes.
|
||||
- Fake adapter/extractor/validator registrations work in tests.
|
||||
- The runner operates on `SourceDocument`, not transcript-specific structures.
|
||||
- The runner does not import concrete D&D extract module packages.
|
||||
|
||||
## Review Questions
|
||||
|
||||
- Does the runner know only about sources, extractors, validators, and artifacts?
|
||||
- Are registries simple enough to evolve?
|
||||
- Are validation decisions expressive enough for deterministic and LLM-backed
|
||||
validators?
|
||||
- Is any domain-specific behavior creeping into framework packages?
|
||||
@@ -1,181 +0,0 @@
|
||||
# Checkpoint 3: Pipeline Stages, Chunking, Merge, And Normalize
|
||||
|
||||
## Status
|
||||
|
||||
This document describes planned work, not implemented behavior.
|
||||
|
||||
## Goal
|
||||
|
||||
Make Notarius's application workflow first-class before adding real input
|
||||
modules or extract modules.
|
||||
|
||||
The workflow should be:
|
||||
|
||||
```text
|
||||
input -> chunk -> extract -> merge -> normalize -> output
|
||||
```
|
||||
|
||||
Checkpoint 3 should define the contracts and minimal fake-tested framework
|
||||
behavior for chunking, per-chunk extraction, merging, and normalization. It
|
||||
should also introduce a fixture-driven walking skeleton that exercises the full
|
||||
stage sequence with fake modules and a fake LLM client. It should not add real
|
||||
input modules, real domain extract modules, LLM provider code, or production
|
||||
output modules.
|
||||
|
||||
## Scope
|
||||
|
||||
In scope:
|
||||
|
||||
- source chunk model;
|
||||
- chunker contract;
|
||||
- extract-stage contract for extractors operating on chunks;
|
||||
- merge-stage contract;
|
||||
- normalize-stage contract;
|
||||
- output-stage contract and fake output encoder for pipeline completeness;
|
||||
- resolved pipeline definition types for a fixed-shape pipeline template;
|
||||
- module binding and module metadata types, including flat capability strings;
|
||||
- default application for `chunk`, lane `merge`, lane `normalize`, `output`,
|
||||
and `llm_profile`;
|
||||
- lane selection behavior equivalent to future `--only`;
|
||||
- runner/pipeline updates that exercise these stages with fake components;
|
||||
- generic append/chronological merge behavior for artifact candidates when
|
||||
appropriate;
|
||||
- fixture-driven walking skeleton test for
|
||||
`input -> chunk -> extract -> merge -> normalize -> output`;
|
||||
- fake `StructuredLLMClient` wired through a trivial extractor.
|
||||
|
||||
Out of scope:
|
||||
|
||||
- Seriatim parsing;
|
||||
- D&D spell extraction;
|
||||
- LLM provider calls;
|
||||
- prompt assets;
|
||||
- response schema assets;
|
||||
- diagnostics run directory;
|
||||
- production config file loading;
|
||||
- real CLI command behavior;
|
||||
- production output serialization or durable output writing.
|
||||
|
||||
## Target End State
|
||||
|
||||
The repository should contain explicit pipeline-stage contracts:
|
||||
|
||||
- `InputAdapter`: external source input to `SourceDocument`.
|
||||
- `Chunker`: `SourceDocument` to ordered `SourceChunk` values.
|
||||
- `Extractor`: `SourceChunk` to artifact candidates.
|
||||
- `Merger`: per-chunk candidates to merged candidates.
|
||||
- `Normalizer`: merged candidates to normalized candidates.
|
||||
- `OutputEncoder`: final artifact bundle to bytes.
|
||||
|
||||
The repository should also contain a resolved pipeline model that represents:
|
||||
|
||||
- `pipeline_id`;
|
||||
- shared input binding;
|
||||
- shared chunk binding;
|
||||
- selected artifact lanes;
|
||||
- lane extract, merge, normalize, and validator bindings;
|
||||
- output binding;
|
||||
- resolved defaults;
|
||||
- resolved pipeline digest input.
|
||||
|
||||
The runner should orchestrate fake implementations through chunk, extract,
|
||||
merge, normalize, and approval/validation behavior in tests.
|
||||
|
||||
The checkpoint should include a fixture-driven walking skeleton that starts from
|
||||
fixture input bytes and ends at encoded output bytes. The walking skeleton should
|
||||
exercise the stage contracts, resolved pipeline model, module metadata,
|
||||
capability validation, defaults, lane selection, and fake LLM client wiring. It
|
||||
is contract coverage, not useful user-facing behavior.
|
||||
|
||||
## Design Intent
|
||||
|
||||
Chunking is a core application concern because many source documents, especially
|
||||
transcripts, will be too large for a single LLM extraction pass.
|
||||
|
||||
Chunk processing may be serial or parallel depending on extractor needs. The
|
||||
architecture should support both, while checkpoint 3 may execute
|
||||
deterministically in series until a later checkpoint introduces concurrency.
|
||||
|
||||
Merge and normalize are separate stages:
|
||||
|
||||
- merge combines per-chunk extracted candidates into one stream or collection;
|
||||
- normalize reconciles the merged output by checking duplicates, consistency,
|
||||
ordering, identity resolution, or other cross-chunk concerns.
|
||||
|
||||
For some artifact types, merge may be generic append-in-source-order behavior.
|
||||
For other artifact types, merge may be domain-specific. Normalization is where
|
||||
deduplication and consistency checks should live.
|
||||
|
||||
## Processing Modes
|
||||
|
||||
The architecture should leave room for extractor-level processing modes:
|
||||
|
||||
- whole-document processing;
|
||||
- serial chunk processing;
|
||||
- parallel chunk processing.
|
||||
|
||||
Checkpoint 3 may execute chunks serially for deterministic behavior.
|
||||
The contracts should not bake in a single-pass assumption or prevent later
|
||||
parallel execution.
|
||||
|
||||
## Generic Merge Behavior
|
||||
|
||||
A generic merger should be able to concatenate candidates in deterministic
|
||||
chunk order and candidate order. This is likely sufficient for early spell-cast
|
||||
extraction, where chronological serialization is useful.
|
||||
|
||||
Domain-specific mergers may be added later when generic ordering is not enough.
|
||||
|
||||
## Generic Normalize Behavior
|
||||
|
||||
A no-op normalizer should be available as the default.
|
||||
|
||||
Domain-specific normalizers may later:
|
||||
|
||||
- deduplicate repeated extracted facts;
|
||||
- resolve aliases;
|
||||
- reconcile conflicting candidate fields;
|
||||
- enforce chronological or source-reference consistency;
|
||||
- attach normalization warnings.
|
||||
|
||||
## Walking Skeleton
|
||||
|
||||
The fixture-driven skeleton should prove the staged architecture continuously as
|
||||
new contracts are added. It should remain deliberately small and use fake modules
|
||||
only. It should validate module keys and flat capability requirements before
|
||||
execution, using registry metadata rather than constructing modules. Capability
|
||||
values should remain simple strings.
|
||||
|
||||
Implementation staging for the walking skeleton belongs in
|
||||
[`implementation.md`](implementation.md).
|
||||
|
||||
## Done Criteria
|
||||
|
||||
- `go test ./...` passes.
|
||||
- Pipeline-stage contracts are explicit and source/domain agnostic.
|
||||
- A resolved pipeline profile model exists for fixed-shape pipeline templates.
|
||||
- Pipeline defaults and lane selection are covered by fake tests.
|
||||
- Module capability validation is covered by fake tests.
|
||||
- Fake tests prove input source documents can be chunked, extracted, merged, and
|
||||
normalized.
|
||||
- A fixture-driven walking skeleton proves fake input, chunk, extract, merge,
|
||||
normalize, and output modules compose end to end with a fake LLM client.
|
||||
- Merge and normalize are distinct concepts in code and tests.
|
||||
- The runner no longer implies whole-document-only extraction as the core
|
||||
application model.
|
||||
- No concrete input module, domain extract module, LLM provider, prompt,
|
||||
response schema, diagnostics, config, or D&D artifact code is added.
|
||||
|
||||
## Review Questions
|
||||
|
||||
- Is the workflow clearly represented as input, chunk, extract, merge,
|
||||
normalize, and output?
|
||||
- Are merge and normalize cleanly separated?
|
||||
- Does the resolved pipeline model avoid becoming a general-purpose workflow
|
||||
engine?
|
||||
- Are module capabilities simple flat strings?
|
||||
- Does lane selection avoid creating ad hoc pipelines?
|
||||
- Can a generic merger handle simple chronological artifact streams?
|
||||
- Can a later domain-specific normalizer handle duplicates and consistency
|
||||
without changing core runner contracts?
|
||||
- Does the design allow serial and parallel chunk processing later?
|
||||
@@ -1,114 +0,0 @@
|
||||
# Checkpoint 4: Portable Audita Infrastructure
|
||||
|
||||
## Status
|
||||
|
||||
This document describes planned work, not implemented behavior.
|
||||
|
||||
## Goal
|
||||
|
||||
Port or adapt reusable Audita infrastructure that directly supports Notarius
|
||||
contracts while avoiding Audita's transcript-correction model.
|
||||
|
||||
This checkpoint should add reusable runtime plumbing, not real extraction
|
||||
behavior.
|
||||
|
||||
## Scope
|
||||
|
||||
In scope:
|
||||
|
||||
- structured LLM client interface implementation;
|
||||
- LLM scheduler;
|
||||
- prompt registry pattern;
|
||||
- response-schema registry pattern;
|
||||
- diagnostics run directory pattern;
|
||||
- config loading and validation for named pipeline profiles and implemented
|
||||
runtime pieces.
|
||||
|
||||
Out of scope:
|
||||
|
||||
- correction proposals;
|
||||
- replacement policies;
|
||||
- transcript mutation;
|
||||
- correction ledger terminology;
|
||||
- Audita module or validator behavior;
|
||||
- real D&D prompts or schemas;
|
||||
- domain-specific prompt or schema assets;
|
||||
- embedded built-in pipeline profiles.
|
||||
|
||||
## Target End State
|
||||
|
||||
The repository should contain reusable runtime infrastructure adapted from
|
||||
Audita where it directly supports Notarius contracts:
|
||||
|
||||
- an OpenAI-compatible structured-output LLM client behind the existing
|
||||
`StructuredLLMClient` interface;
|
||||
- an LLM scheduler for bounded concurrency;
|
||||
- an embedded response-schema registry pattern;
|
||||
- an embedded prompt registry pattern;
|
||||
- a diagnostics run directory pattern using extraction-oriented artifact names;
|
||||
- config structs, loading, defaults, redaction, and validation for named
|
||||
pipeline profiles.
|
||||
|
||||
Framework code should remain source-agnostic and domain-agnostic. Provider HTTP
|
||||
details should stay inside the LLM runtime package. Prompt and schema registries
|
||||
should use placeholder/test assets until real extractor prompts and schemas are
|
||||
introduced by later checkpoints.
|
||||
|
||||
Config should support:
|
||||
|
||||
- `llm_profiles`;
|
||||
- `pipelines.<pipeline_id>.input`;
|
||||
- `pipelines.<pipeline_id>.chunk`;
|
||||
- `pipelines.<pipeline_id>.artifacts.<lane>.extract`;
|
||||
- lane `merge`, `normalize`, and validator settings;
|
||||
- output module selection;
|
||||
- inline module-binding object form and string shorthand;
|
||||
- default `chunk`, `merge`, `normalize`, `output`, and `llm_profile`;
|
||||
- selected pipeline ID and lane filtering for runtime use;
|
||||
- concurrency;
|
||||
- work directory;
|
||||
- diagnostics retention.
|
||||
|
||||
Config loading should support the standard precedence model:
|
||||
|
||||
1. built-in defaults
|
||||
2. configuration file
|
||||
3. environment variables
|
||||
4. CLI flags
|
||||
|
||||
Structural module selection should come from pipeline config. CLI flags may
|
||||
override operational settings and artifact lane selection, but should not offer
|
||||
ad hoc `--extractor` or `--chunker` wiring.
|
||||
|
||||
Config validation should fail fast for unknown pipeline IDs, unknown module
|
||||
keys, missing required slots, missing capabilities, unknown LLM profiles, empty
|
||||
artifact-lane sets, and invalid lane selections.
|
||||
|
||||
If the CLI shell is ready, the checkpoint should expose discovery/validation
|
||||
commands for config and pipeline profiles:
|
||||
|
||||
```sh
|
||||
notarius config validate
|
||||
notarius pipelines list
|
||||
```
|
||||
|
||||
Implementation staging belongs in [`implementation.md`](implementation.md).
|
||||
|
||||
## Done Criteria
|
||||
|
||||
- `go test ./...` passes.
|
||||
- Audita runtime pieces are adapted to Notarius package names and contracts.
|
||||
- No correction proposal, replacement policy, transcript mutation, or correction
|
||||
ledger code has been copied.
|
||||
- Runtime tests cover secret redaction, schema registry lookup, prompt metadata,
|
||||
and scheduler behavior where applicable.
|
||||
- Config tests cover named pipeline profiles, defaults, lane selection,
|
||||
capability validation, and resolved pipeline digesting.
|
||||
|
||||
## Review Questions
|
||||
|
||||
- Did we copy only reusable infrastructure?
|
||||
- Do provider-specific types stay behind adapter/runtime boundaries?
|
||||
- Are diagnostics names and report concepts extraction-oriented?
|
||||
- Is config limited to named pipeline profiles and implemented behavior?
|
||||
- Are structural pipeline changes kept out of ad hoc CLI flags?
|
||||
@@ -1,107 +0,0 @@
|
||||
# Checkpoint 5: Seriatim Input Module
|
||||
|
||||
## Status
|
||||
|
||||
This document describes planned work, not implemented behavior.
|
||||
|
||||
## Goal
|
||||
|
||||
Add the first real input source while keeping transcript-specific behavior
|
||||
isolated inside an input-stage module.
|
||||
|
||||
This checkpoint should allow Seriatim minimal transcript JSON to become a
|
||||
generic `SourceDocument`.
|
||||
|
||||
## Scope
|
||||
|
||||
In scope:
|
||||
|
||||
- `internal/modules/input/seriatim`;
|
||||
- parser for Seriatim minimal output JSON;
|
||||
- mapping into `SourceDocument` and `SourceUnit`;
|
||||
- source-document validation;
|
||||
- input adapter registry wiring;
|
||||
- module metadata/capabilities for pipeline validation;
|
||||
- fixtures and tests;
|
||||
- config compatibility through named pipeline profiles.
|
||||
|
||||
Out of scope:
|
||||
|
||||
- D&D extraction;
|
||||
- LLM extraction calls;
|
||||
- transcript-specific behavior in runner/core packages;
|
||||
- support for every possible Seriatim schema variant.
|
||||
|
||||
## Target End State
|
||||
|
||||
The repository should contain a real Seriatim input-stage module that translates
|
||||
Seriatim minimal transcript JSON into the generic source model.
|
||||
|
||||
The Seriatim module should be registered under the stable input adapter key
|
||||
`seriatim`. It should be selectable through the existing input adapter registry
|
||||
and through pipeline-profile resolution when a profile binds `input: seriatim`.
|
||||
|
||||
The module should accept the Seriatim minimal output shape:
|
||||
|
||||
- top-level `metadata`;
|
||||
- top-level `segments`;
|
||||
- segment `id`;
|
||||
- segment `start`;
|
||||
- segment `end`;
|
||||
- segment `speaker`;
|
||||
- segment `text`.
|
||||
|
||||
The module should map Seriatim data into generic source values:
|
||||
|
||||
- segment `id` becomes `SourceUnit.ID`;
|
||||
- segment `text` becomes `SourceUnit.Text`;
|
||||
- the document and unit kind strings identify transcript-like source material
|
||||
without adding transcript-specific fields or types to core packages;
|
||||
- `speaker`, `start`, and `end` become source-unit metadata;
|
||||
- top-level Seriatim metadata becomes source-document metadata;
|
||||
- the resulting source document passes core source validation.
|
||||
|
||||
The module should reject invalid Seriatim input with clear module-specific
|
||||
errors. Validation should cover:
|
||||
|
||||
- valid JSON;
|
||||
- required top-level metadata and segments;
|
||||
- required segment fields;
|
||||
- unique segment IDs;
|
||||
- non-empty segment text;
|
||||
- valid start and end values.
|
||||
|
||||
The module should declare flat capabilities for pipeline validation. Initial
|
||||
capabilities should describe transcript-oriented source properties preserved by
|
||||
the adapter, including speaker and timestamp metadata.
|
||||
|
||||
Implementation staging belongs in
|
||||
[`implementation.md`](implementation.md).
|
||||
|
||||
## Fixtures And Tests
|
||||
|
||||
The checkpoint should add synthetic fixtures and focused tests for:
|
||||
|
||||
- valid Seriatim minimal transcript;
|
||||
- malformed JSON;
|
||||
- missing metadata;
|
||||
- missing or duplicate segment IDs;
|
||||
- empty segment text;
|
||||
- source-reference compatibility with generated unit IDs.
|
||||
|
||||
## Done Criteria
|
||||
|
||||
- `go test ./...` passes.
|
||||
- Seriatim minimal transcript JSON maps into `SourceDocument`.
|
||||
- Transcript fields do not appear in core runner contracts.
|
||||
- The input module is selectable through the registry and pipeline-profile
|
||||
configuration when config support exists.
|
||||
- The input module declares capabilities needed for pipeline validation.
|
||||
- Tests prove transcript-specific assumptions are isolated to the input module.
|
||||
|
||||
## Review Questions
|
||||
|
||||
- Are segment, speaker, and timestamp assumptions contained inside the input module?
|
||||
- Are unit IDs stable and suitable for source references?
|
||||
- Does the input module preserve enough metadata for transcript-oriented output later?
|
||||
- Should the input module accept only Seriatim minimal output for now?
|
||||
@@ -1,120 +0,0 @@
|
||||
# Checkpoint 6: D&D Spells Extractor
|
||||
|
||||
## Status
|
||||
|
||||
This document records the target scope for checkpoint 6. The implemented
|
||||
integration contract is documented in
|
||||
[`docs/integrations/dnd-spells.md`](../integrations/dnd-spells.md).
|
||||
|
||||
## Goal
|
||||
|
||||
Implement the first useful extract-stage module: D&D spell casts from a
|
||||
Seriatim transcript source document.
|
||||
|
||||
This checkpoint should produce the first meaningful vertical slice from real
|
||||
source input to validated artifact output.
|
||||
|
||||
## Scope
|
||||
|
||||
In scope:
|
||||
|
||||
- D&D spell artifact schema and Go structs;
|
||||
- structured response schema asset;
|
||||
- prompt assets;
|
||||
- `internal/modules/extract/dnd/spells`;
|
||||
- module metadata/capability requirements for pipeline validation;
|
||||
- source-reference and schema validators in the extractor chain;
|
||||
- fake LLM tests;
|
||||
- runner-level integration tests with fake infrastructure.
|
||||
|
||||
Out of scope:
|
||||
|
||||
- D&D item extraction;
|
||||
- NPC extraction;
|
||||
- combat extraction;
|
||||
- cross-slice deduplication beyond simple deterministic merging;
|
||||
- broad D&D rules validation;
|
||||
- a `notarius run` command.
|
||||
|
||||
## Target End State
|
||||
|
||||
The repository should contain a real D&D spells extract-stage module at
|
||||
`internal/modules/extract/dnd/spells`.
|
||||
|
||||
The spells module should be registered under the stable extractor key
|
||||
`dnd/spells`. It should be selectable as a named artifact lane in pipeline
|
||||
configuration, for example a lane named `spells` whose extractor module is
|
||||
`dnd/spells`.
|
||||
|
||||
The module should translate generic source chunks into spell-cast artifact
|
||||
candidates with this spell payload:
|
||||
|
||||
- `caster`;
|
||||
- `spell`;
|
||||
- `effect`;
|
||||
- `narrative_description`.
|
||||
|
||||
The LLM structured response should also include source references for each
|
||||
spell cast. The extractor should copy those references into the generic
|
||||
artifact envelope rather than duplicating source references inside the durable
|
||||
spell payload.
|
||||
|
||||
The caster is the in-world character or creature casting the spell, not the
|
||||
table speaker. Speaker metadata from transcript source units may be used as
|
||||
optional prompt context when present, but it should not be a required or durable
|
||||
field in the spell payload.
|
||||
|
||||
Prompts should:
|
||||
|
||||
- describe the generic source-unit input format;
|
||||
- explain that source references must use source-unit IDs exactly;
|
||||
- avoid relying on transcript-specific fields except as optional metadata;
|
||||
- request only D&D spell-cast artifacts.
|
||||
|
||||
The extractor should attach deterministic validators by default. Validation
|
||||
should cover:
|
||||
|
||||
- spell payload shape;
|
||||
- non-empty required spell fields;
|
||||
- at least one source reference per spell cast;
|
||||
- source references that validate against the source document.
|
||||
|
||||
The module should declare flat capabilities for pipeline validation. Initial
|
||||
capabilities should require chunked transcript source material and provide a
|
||||
D&D spell-cast artifact capability.
|
||||
|
||||
Implementation staging belongs in
|
||||
[`implementation.md`](implementation.md).
|
||||
|
||||
## Fixtures And Tests
|
||||
|
||||
The checkpoint should add synthetic fixtures and focused tests for:
|
||||
|
||||
- successful spell extraction with a fake structured LLM client;
|
||||
- empty spell-cast results;
|
||||
- malformed structured output handling;
|
||||
- invalid source-reference rejection;
|
||||
- stable output ordering;
|
||||
- pipeline-profile selection of a `spells` artifact lane;
|
||||
- runner integration from Seriatim input through the spells extractor using
|
||||
fake chunk and output modules.
|
||||
|
||||
## Done Criteria
|
||||
|
||||
- `go test ./...` passes.
|
||||
- Seriatim input can flow through the runner into the D&D spells extractor.
|
||||
- Spell artifacts include valid source references.
|
||||
- D&D concepts are contained in extract module/artifact packages and docs.
|
||||
- The spells module can be selected as a named artifact lane in pipeline
|
||||
configuration.
|
||||
- The first meaningful vertical slice is available through tests.
|
||||
- No CLI `run` behavior is documented or implemented until the CLI path exists.
|
||||
|
||||
## Review Questions
|
||||
|
||||
- Is the spell extract module domain-specific without making the framework
|
||||
D&D-specific?
|
||||
- Are source references valid and useful for downstream validation?
|
||||
- Is prompt/schema ownership clear?
|
||||
- Does this vertical slice reveal contract changes needed before adding items,
|
||||
NPCs, or combat?
|
||||
File diff suppressed because it is too large
Load Diff
@@ -546,10 +546,10 @@ Reuse these architectural patterns:
|
||||
- validator decision cardinality and deterministic validator ordering;
|
||||
- CLI tests and fixture-driven integration tests.
|
||||
|
||||
The fixture-driven integration-test pattern should begin at checkpoint 3 with a
|
||||
walking skeleton over fake modules and a fake LLM client. Later checkpoints
|
||||
should replace fake pieces with real Seriatim, runtime, and D&D modules without
|
||||
losing that end-to-end contract coverage.
|
||||
The fixture-driven integration-test pattern should remain part of the codebase:
|
||||
walking skeleton tests over fake modules and fake LLM clients should be
|
||||
preserved as real Seriatim, runtime, and D&D modules are added, so the
|
||||
end-to-end contract coverage is not lost.
|
||||
|
||||
Avoid copying these Audita concepts directly:
|
||||
|
||||
@@ -565,23 +565,13 @@ extraction-report concepts.
|
||||
|
||||
## Checkpoint Roadmap
|
||||
|
||||
The initial implementation should proceed through six coherent checkpoints.
|
||||
Each checkpoint should leave the repository in a reviewable state, with the code
|
||||
compiling and targeted tests covering the newly introduced contracts or behavior.
|
||||
The initial six checkpoint roadmap has been implemented and retired. The
|
||||
checkpoint files have been removed from `docs/roadmap/` so the active roadmap
|
||||
does not compete with completed implementation history.
|
||||
|
||||
1. [Core Contracts And Skeleton](1-core-contracts-and-skeleton.md)
|
||||
2. [Framework Composition](2-framework-composition.md)
|
||||
3. [Pipeline Stages, Chunking, Merge, And Normalize](3-pipeline-stages-chunking-merge-normalize.md)
|
||||
4. [Portable Audita Infrastructure](4-portable-audita-infrastructure.md)
|
||||
5. [Seriatim Input Module](5-seriatim-input-module.md)
|
||||
6. [D&D Spells Extractor](6-dnd-spells-extractor.md)
|
||||
|
||||
The first contract-level walking skeleton should arrive at checkpoint 3: fixture
|
||||
input through fake input, chunk, extract, merge, normalize, and output modules
|
||||
with a fake LLM client. The first useful vertical slice should arrive at
|
||||
checkpoint 6: Seriatim transcript input to validated D&D spell artifact output.
|
||||
Earlier checkpoints remain contract-first and may not produce useful user output
|
||||
yet.
|
||||
The remaining work needed to reach the first functional MVP is tracked in
|
||||
[`mvp.md`](mvp.md). Future staged implementation plans should be written to
|
||||
[`implementation.md`](implementation.md) from that active MVP roadmap.
|
||||
|
||||
## Architecture Decisions
|
||||
|
||||
|
||||
249
docs/roadmap/mvp.md
Normal file
249
docs/roadmap/mvp.md
Normal file
@@ -0,0 +1,249 @@
|
||||
# MVP Roadmap
|
||||
|
||||
## Status
|
||||
|
||||
This is the active roadmap for reaching the first functional Notarius MVP.
|
||||
|
||||
The previous numbered checkpoint roadmaps have been implemented and retired.
|
||||
This document captures the remaining work needed to turn the implemented
|
||||
architecture into a usable MVP, with the current architectural review findings
|
||||
folded in as first-class work.
|
||||
|
||||
Implementation staging belongs in [`implementation.md`](implementation.md).
|
||||
|
||||
## MVP Goal
|
||||
|
||||
The MVP should let a user run Notarius against a Seriatim minimal transcript
|
||||
JSON file, select a configured pipeline profile, extract D&D spell-cast
|
||||
artifacts with an LLM-backed extractor, validate those artifacts, and write
|
||||
durable JSON output plus diagnostics.
|
||||
|
||||
The intended command shape is:
|
||||
|
||||
```sh
|
||||
notarius run dnd-session --input session-014.json
|
||||
notarius run dnd-session --input session-014.json --only spells
|
||||
```
|
||||
|
||||
The MVP remains intentionally narrow:
|
||||
|
||||
- one production input module: `seriatim`;
|
||||
- one production extract module: `dnd/spells`;
|
||||
- one generic chunk module sufficient for transcript-scale processing;
|
||||
- generic append-order merge;
|
||||
- generic noop normalization;
|
||||
- generic JSON output;
|
||||
- config-driven pipeline profiles;
|
||||
- OpenAI-compatible structured LLM execution through the existing LLM client.
|
||||
|
||||
## MVP Work Areas
|
||||
|
||||
### Framework/Domain Asset Boundaries
|
||||
|
||||
This is the highest-priority remaining architecture correction.
|
||||
|
||||
Framework packages must remain source-agnostic and domain-agnostic. D&D spell
|
||||
prompt assets, response schema assets, prompt IDs, response schema keys, and
|
||||
domain-specific prompt/schema tests should not live in `internal/framework/llm`
|
||||
or `internal/framework/prompt`.
|
||||
|
||||
Target state:
|
||||
|
||||
- `internal/framework/llm` provides generic structured-output client,
|
||||
scheduler, schema metadata, schema loading, and schema lookup/registration
|
||||
primitives.
|
||||
- `internal/framework/prompt` provides generic prompt metadata, prompt loading,
|
||||
rendering, hardening, and lookup/registration primitives.
|
||||
- `internal/modules/extract/dnd/spells` owns the D&D spell prompt assets,
|
||||
response schema assets, stable prompt ID, stable response schema key, and
|
||||
module-specific prompt/schema tests.
|
||||
- Framework tests use placeholder/test assets only.
|
||||
- The D&D spells extractor depends on generic framework APIs, not
|
||||
framework-owned D&D constants.
|
||||
|
||||
This work should not change the external artifact shape or module key. It is an
|
||||
ownership and package-boundary correction.
|
||||
|
||||
### Production Application Catalog Wiring
|
||||
|
||||
The implemented modules and registries are currently exercised mostly through
|
||||
tests that inject catalogs. The MVP needs a production assembly point that
|
||||
builds the catalog and stage registries used by real CLI commands.
|
||||
|
||||
Target state:
|
||||
|
||||
- a small app-level package or CLI wiring function constructs the production
|
||||
`pipeline.ModuleCatalog`;
|
||||
- production wiring registers `seriatim`;
|
||||
- production wiring registers `dnd/spells`;
|
||||
- production wiring registers the default `generic`, `appendorder`, `noop`, and
|
||||
`json` modules;
|
||||
- `notarius config validate --pipeline ...` validates real configured
|
||||
pipelines without test-only catalog injection;
|
||||
- `notarius pipelines list` reports production-registered modules where useful
|
||||
for discoverability.
|
||||
|
||||
The production wiring should not move domain behavior into the CLI. The CLI may
|
||||
compose modules, but module packages should continue to own their own behavior
|
||||
and metadata.
|
||||
|
||||
### Default Production Stage Modules
|
||||
|
||||
Pipeline defaults are already part of the architecture:
|
||||
|
||||
- `chunk: generic`;
|
||||
- `merge: appendorder`;
|
||||
- `normalize: noop`;
|
||||
- `output: json`.
|
||||
|
||||
The MVP should make those defaults real production modules rather than
|
||||
test-only conveniences.
|
||||
|
||||
Target state:
|
||||
|
||||
- `generic` chunking creates ordered chunks over generic source units and is
|
||||
configurable enough for transcript MVP use;
|
||||
- `appendorder` merge serializes artifact candidates in deterministic source
|
||||
and chunk order;
|
||||
- `noop` normalize passes merged artifacts through unchanged while preserving
|
||||
diagnostics;
|
||||
- `json` output encodes approved artifacts, rejected artifacts, warnings,
|
||||
manifest data, and relevant run metadata in a durable JSON shape;
|
||||
- each default module declares module specs and capabilities compatible with
|
||||
pipeline validation;
|
||||
- default modules are registered by production app wiring.
|
||||
|
||||
If a default module remains implemented in `internal/framework/pipeline`, its
|
||||
production registration still needs to be explicit and discoverable. If its
|
||||
logic grows beyond a small generic helper, move it under `internal/modules`.
|
||||
|
||||
### `notarius run`
|
||||
|
||||
The MVP needs a functional run command that drives the already-implemented
|
||||
pipeline runner.
|
||||
|
||||
Target state:
|
||||
|
||||
- command shape:
|
||||
|
||||
```sh
|
||||
notarius run <pipeline-id> --input path/to/source.json
|
||||
notarius run <pipeline-id> --input path/to/source.json --only spells
|
||||
```
|
||||
|
||||
- required flags and arguments produce clear usage errors;
|
||||
- `--config` selects the config file;
|
||||
- `--only` filters artifact lanes without changing structural pipeline config;
|
||||
- operational overrides may cover output directory, work directory,
|
||||
concurrency, and LLM profile/model settings where already supported by config;
|
||||
- structural stage selection remains config-driven;
|
||||
- the command parses input through the configured input adapter;
|
||||
- the command constructs the configured LLM client and scheduler;
|
||||
- the command invokes the pipeline runner;
|
||||
- the command writes durable output and diagnostics;
|
||||
- failures return stable non-zero exit codes and useful error messages.
|
||||
|
||||
The command should be covered by fixture-driven CLI tests with fake LLM behavior
|
||||
where network calls would otherwise be required.
|
||||
|
||||
### MVP Output And Diagnostics Behavior
|
||||
|
||||
The MVP should produce inspectable files that are stable enough for downstream
|
||||
experiments, without pretending to be a final public artifact contract.
|
||||
|
||||
Target state:
|
||||
|
||||
- output path behavior is deterministic and documented in code/tests;
|
||||
- JSON output includes approved artifacts grouped or ordered predictably;
|
||||
- each artifact includes its generic source references;
|
||||
- rejected artifacts and validation decisions remain inspectable;
|
||||
- output-stage warnings remain out-of-band from the durable artifact payload but
|
||||
are captured for CLI reporting and diagnostics;
|
||||
- run manifest data includes source digest, resolved pipeline digest, relevant
|
||||
model/profile information, prompt/schema identifiers, and validation status;
|
||||
- diagnostics redact secrets and include the resolved effective configuration
|
||||
needed to debug a run.
|
||||
|
||||
### MVP Fixtures And Acceptance Tests
|
||||
|
||||
The MVP should be continuously testable without external services.
|
||||
|
||||
Target state:
|
||||
|
||||
- maintained Seriatim transcript fixture for the D&D spells MVP;
|
||||
- maintained minimal config fixture for the MVP pipeline;
|
||||
- fake LLM path for deterministic CLI and runner tests;
|
||||
- config validation tests using the production catalog;
|
||||
- `notarius run` fixture test from input file to output JSON;
|
||||
- failure tests for missing config, unknown pipeline, invalid input,
|
||||
invalid lane selection, LLM failure, and validation rejection;
|
||||
- `go test ./...` is sufficient to exercise the MVP path without network
|
||||
access.
|
||||
|
||||
### Documentation Pass Preparation
|
||||
|
||||
The full documentation pass is intentionally deferred until MVP functionality
|
||||
exists. It should happen before tagging alpha `0.1.0`.
|
||||
|
||||
The MVP implementation should still leave clear hooks for the documentation
|
||||
rewrite:
|
||||
|
||||
- command behavior should be stable enough to document in `docs/cli.md`;
|
||||
- config behavior should be stable enough to document in `docs/config.md`;
|
||||
- output behavior should be stable enough to document in integration docs;
|
||||
- examples should be generated from or validated against maintained fixtures
|
||||
where practical.
|
||||
|
||||
## Out Of Scope For MVP
|
||||
|
||||
- D&D item extraction;
|
||||
- NPC extraction;
|
||||
- combat extraction;
|
||||
- D&D rules validation beyond the spell extractor's deterministic checks;
|
||||
- Markdown or Obsidian input;
|
||||
- cross-lane entity normalization;
|
||||
- cross-chunk semantic deduplication beyond whatever a simple normalizer can
|
||||
safely support;
|
||||
- a general DAG or workflow engine;
|
||||
- ad hoc CLI flags for structural module selection;
|
||||
- release-quality documentation before the MVP behavior is implemented.
|
||||
|
||||
## MVP Done Criteria
|
||||
|
||||
- D&D prompt and response schema assets are owned by the D&D spells module, not
|
||||
by framework packages.
|
||||
- Production CLI commands use a real app catalog rather than test-injected
|
||||
module catalogs.
|
||||
- A config profile can bind `input: seriatim` and an artifact lane with
|
||||
`extract: dnd/spells`.
|
||||
- Default `generic`, `appendorder`, `noop`, and `json` modules resolve through
|
||||
production wiring.
|
||||
- `notarius config validate --config <file> --pipeline <id>` works with the
|
||||
MVP config.
|
||||
- `notarius pipelines list --config <file>` works with the MVP config.
|
||||
- `notarius run <pipeline-id> --input <file>` reads a Seriatim transcript,
|
||||
extracts D&D spell artifacts, validates them, and writes JSON output.
|
||||
- `notarius run <pipeline-id> --input <file> --only spells` runs only the
|
||||
selected artifact lane.
|
||||
- The run manifest records source digest, resolved pipeline digest, LLM profile
|
||||
and model, prompt/schema identifiers, and validation status.
|
||||
- Output warnings are available to CLI/diagnostics without becoming artifact
|
||||
payload fields.
|
||||
- MVP fixture tests cover the full path without network access.
|
||||
- `go test ./...`, `go vet ./...`, and `go build ./cmd/notarius` pass.
|
||||
|
||||
## Deferred Documentation Pass
|
||||
|
||||
After MVP behavior is implemented and before alpha `0.1.0`, complete a full
|
||||
documentation pass/rewrite. That pass should move implemented behavior out of
|
||||
roadmap documents and into canonical docs required by
|
||||
[`../policy/documentation.md`](../policy/documentation.md), including at least:
|
||||
|
||||
- `README.md`;
|
||||
- `docs/cli.md`;
|
||||
- `docs/config.md`;
|
||||
- `docs/operations.md`, if diagnostics/run recovery behavior warrants it;
|
||||
- `docs/internal/` architecture and package-boundary docs;
|
||||
- `docs/integrations/` updates for Seriatim input, D&D spell artifacts, and
|
||||
JSON output;
|
||||
- maintained `examples/` files.
|
||||
Reference in New Issue
Block a user