Update the architecture plan to standardize on input -> chunk -> extract -> merge -> normalize -> output naming conventions
This commit is contained in:
@@ -14,10 +14,10 @@ The first MVP should target audio transcripts generated by Seriatim. That
|
||||
choice should be implemented as an input-stage module, not as a
|
||||
transcript-specific assumption in the application core. Later input sources,
|
||||
such as unstructured Markdown notes or Obsidian documents, should be addable
|
||||
through new input and process modules without reshaping the framework.
|
||||
through new input and extract modules without reshaping the framework.
|
||||
|
||||
The first extraction domain should be D&D session analysis, starting with spell
|
||||
casts. That domain should live in process-stage modules and related schemas, not
|
||||
casts. That domain should live in extract-stage modules and related schemas, not
|
||||
in core framework packages.
|
||||
|
||||
The application should follow the same broad architecture as Audita:
|
||||
@@ -25,7 +25,7 @@ The application should follow the same broad architecture as Audita:
|
||||
- deterministic core packages for config, source documents, artifacts, diagnostics, and reporting;
|
||||
- input-stage modules that translate external source formats into a small internal source model;
|
||||
- reusable framework packages for contracts, orchestration, LLM runtime, structured output, and validation;
|
||||
- independent process-stage modules that own domain-specific behavior;
|
||||
- independent extract-stage modules that own domain-specific behavior;
|
||||
- independent validator packages;
|
||||
- embedded prompt and JSON schema assets;
|
||||
- CLI orchestration that wires the pieces together without owning domain logic.
|
||||
@@ -37,7 +37,7 @@ artifacts rather than proposing and applying transcript corrections.
|
||||
|
||||
- Keep the core input model generic: ordered text units plus metadata.
|
||||
- Keep source-format details in hexagonal input modules.
|
||||
- Keep extraction-domain details in process modules.
|
||||
- Keep extraction-domain details in extract modules.
|
||||
- Treat evidence as source references, not transcript references.
|
||||
- Prefer narrow, useful abstractions over a universal document model.
|
||||
- Preserve enough provenance for validation, replay, and downstream inspection.
|
||||
@@ -77,10 +77,10 @@ internal/modules/input/markdown
|
||||
internal/modules/chunk/generic
|
||||
internal/modules/chunk/dndtranscript
|
||||
|
||||
internal/modules/process/dnd/spells
|
||||
internal/modules/process/dnd/items
|
||||
internal/modules/process/dnd/npcs
|
||||
internal/modules/process/dnd/combat
|
||||
internal/modules/extract/dnd/spells
|
||||
internal/modules/extract/dnd/items
|
||||
internal/modules/extract/dnd/npcs
|
||||
internal/modules/extract/dnd/combat
|
||||
|
||||
internal/modules/merge/appendorder
|
||||
internal/modules/merge/dnd/spells
|
||||
@@ -193,7 +193,7 @@ type Extractor interface {
|
||||
ArtifactType() string
|
||||
SchemaVersion() string
|
||||
Validators() []Validator
|
||||
Process(ctx context.Context, req ProcessRequest) (ProcessResult, error)
|
||||
Extract(ctx context.Context, req ExtractionRequest) (ExtractionResult, error)
|
||||
}
|
||||
```
|
||||
|
||||
@@ -201,9 +201,9 @@ An extractor should receive either a whole source document or a source chunk,
|
||||
depending on processing mode. It should return typed artifact candidates plus
|
||||
warnings. It should not mutate the source document.
|
||||
|
||||
Process modules own domain concepts. For example, D&D spell extraction should
|
||||
live under `internal/modules/process/dnd/spells`; a future to-do extractor for
|
||||
notes should live under a different process-module path and use the same
|
||||
Extract modules own domain concepts. For example, D&D spell extraction should
|
||||
live under `internal/modules/extract/dnd/spells`; a future to-do extractor for
|
||||
notes should live under a different extract-module path and use the same
|
||||
framework contract.
|
||||
|
||||
### Chunker
|
||||
@@ -262,9 +262,13 @@ Per-run provenance record.
|
||||
|
||||
```go
|
||||
type RunManifest struct {
|
||||
InputAdapter string `json:"input_adapter"`
|
||||
InputModule string `json:"input_module"`
|
||||
Chunker string `json:"chunker"`
|
||||
SourceDigests []string `json:"source_digests"`
|
||||
Extractors []string `json:"extractors"`
|
||||
Merger string `json:"merger"`
|
||||
Normalizer string `json:"normalizer"`
|
||||
OutputEncoder string `json:"output_encoder"`
|
||||
SchemaVersion string `json:"schema_version"`
|
||||
ValidationStatus string `json:"validation_status"`
|
||||
}
|
||||
@@ -345,7 +349,7 @@ The architecture should support extractors outside the D&D domain. Examples:
|
||||
- decisions and action items from meeting transcripts;
|
||||
- named people, places, and dates from research notes.
|
||||
|
||||
These should be addable as process modules without changing runner,
|
||||
These should be addable as extract modules without changing runner,
|
||||
validator, source-reference, or LLM framework contracts.
|
||||
|
||||
## Proposed Pipeline Flow
|
||||
@@ -353,7 +357,7 @@ validator, source-reference, or LLM framework contracts.
|
||||
The application workflow should be first-class:
|
||||
|
||||
```text
|
||||
input -> chunk -> process -> merge -> normalize -> output
|
||||
input -> chunk -> extract -> merge -> normalize -> output
|
||||
```
|
||||
|
||||
Proposed runner flow:
|
||||
|
||||
Reference in New Issue
Block a user