Update the architecture plan to standardize on input -> chunk -> extract -> merge -> normalize -> output naming conventions

This commit is contained in:
2026-07-03 09:03:45 -05:00
parent 88042174b3
commit 5a6e82f599
18 changed files with 121 additions and 113 deletions

View File

@@ -14,10 +14,10 @@ The first MVP should target audio transcripts generated by Seriatim. That
choice should be implemented as an input-stage module, not as a
transcript-specific assumption in the application core. Later input sources,
such as unstructured Markdown notes or Obsidian documents, should be addable
through new input and process modules without reshaping the framework.
through new input and extract modules without reshaping the framework.
The first extraction domain should be D&D session analysis, starting with spell
casts. That domain should live in process-stage modules and related schemas, not
casts. That domain should live in extract-stage modules and related schemas, not
in core framework packages.
The application should follow the same broad architecture as Audita:
@@ -25,7 +25,7 @@ The application should follow the same broad architecture as Audita:
- deterministic core packages for config, source documents, artifacts, diagnostics, and reporting;
- input-stage modules that translate external source formats into a small internal source model;
- reusable framework packages for contracts, orchestration, LLM runtime, structured output, and validation;
- independent process-stage modules that own domain-specific behavior;
- independent extract-stage modules that own domain-specific behavior;
- independent validator packages;
- embedded prompt and JSON schema assets;
- CLI orchestration that wires the pieces together without owning domain logic.
@@ -37,7 +37,7 @@ artifacts rather than proposing and applying transcript corrections.
- Keep the core input model generic: ordered text units plus metadata.
- Keep source-format details in hexagonal input modules.
- Keep extraction-domain details in process modules.
- Keep extraction-domain details in extract modules.
- Treat evidence as source references, not transcript references.
- Prefer narrow, useful abstractions over a universal document model.
- Preserve enough provenance for validation, replay, and downstream inspection.
@@ -77,10 +77,10 @@ internal/modules/input/markdown
internal/modules/chunk/generic
internal/modules/chunk/dndtranscript
internal/modules/process/dnd/spells
internal/modules/process/dnd/items
internal/modules/process/dnd/npcs
internal/modules/process/dnd/combat
internal/modules/extract/dnd/spells
internal/modules/extract/dnd/items
internal/modules/extract/dnd/npcs
internal/modules/extract/dnd/combat
internal/modules/merge/appendorder
internal/modules/merge/dnd/spells
@@ -193,7 +193,7 @@ type Extractor interface {
ArtifactType() string
SchemaVersion() string
Validators() []Validator
Process(ctx context.Context, req ProcessRequest) (ProcessResult, error)
Extract(ctx context.Context, req ExtractionRequest) (ExtractionResult, error)
}
```
@@ -201,9 +201,9 @@ An extractor should receive either a whole source document or a source chunk,
depending on processing mode. It should return typed artifact candidates plus
warnings. It should not mutate the source document.
Process modules own domain concepts. For example, D&D spell extraction should
live under `internal/modules/process/dnd/spells`; a future to-do extractor for
notes should live under a different process-module path and use the same
Extract modules own domain concepts. For example, D&D spell extraction should
live under `internal/modules/extract/dnd/spells`; a future to-do extractor for
notes should live under a different extract-module path and use the same
framework contract.
### Chunker
@@ -262,9 +262,13 @@ Per-run provenance record.
```go
type RunManifest struct {
InputAdapter string `json:"input_adapter"`
InputModule string `json:"input_module"`
Chunker string `json:"chunker"`
SourceDigests []string `json:"source_digests"`
Extractors []string `json:"extractors"`
Merger string `json:"merger"`
Normalizer string `json:"normalizer"`
OutputEncoder string `json:"output_encoder"`
SchemaVersion string `json:"schema_version"`
ValidationStatus string `json:"validation_status"`
}
@@ -345,7 +349,7 @@ The architecture should support extractors outside the D&D domain. Examples:
- decisions and action items from meeting transcripts;
- named people, places, and dates from research notes.
These should be addable as process modules without changing runner,
These should be addable as extract modules without changing runner,
validator, source-reference, or LLM framework contracts.
## Proposed Pipeline Flow
@@ -353,7 +357,7 @@ validator, source-reference, or LLM framework contracts.
The application workflow should be first-class:
```text
input -> chunk -> process -> merge -> normalize -> output
input -> chunk -> extract -> merge -> normalize -> output
```
Proposed runner flow: