Update the architecture plan to standardize on input -> chunk -> extract -> merge -> normalize -> output naming conventions
This commit is contained in:
@@ -29,7 +29,7 @@ Source-format details belong in input modules. Transcript-specific concepts
|
||||
such as segments, speakers, timestamps, and transcript schemas must not spread
|
||||
into runner, extractor, or validator framework code.
|
||||
|
||||
Extraction-domain details belong in process modules. D&D-specific concepts
|
||||
Extraction-domain details belong in extract modules. D&D-specific concepts
|
||||
such as spells, NPCs, items, combat turns, and encounters must not spread into
|
||||
core source, runner, or LLM framework packages.
|
||||
|
||||
@@ -39,11 +39,11 @@ should point to generic source units, not to transcript-only structures.
|
||||
The application workflow is:
|
||||
|
||||
```text
|
||||
input -> chunk -> process -> merge -> normalize -> output
|
||||
input -> chunk -> extract -> merge -> normalize -> output
|
||||
```
|
||||
|
||||
These stages should remain explicit in the architecture. Chunking, merging, and
|
||||
normalization must not be hidden inside domain process modules when they represent
|
||||
normalization must not be hidden inside domain extract modules when they represent
|
||||
general pipeline behavior.
|
||||
|
||||
## Dependency Policy
|
||||
@@ -99,7 +99,7 @@ Domain implementations:
|
||||
|
||||
- `internal/modules/input/<name>`: input-stage modules that parse external input into core source documents.
|
||||
- `internal/modules/chunk/<name>`: chunk-stage modules.
|
||||
- `internal/modules/process/<domain>/<name>`: process-stage extractor modules.
|
||||
- `internal/modules/extract/<domain>/<name>`: extract-stage extractor modules.
|
||||
- `internal/modules/merge/<name>` or `internal/modules/merge/<domain>/<name>`: merge-stage modules.
|
||||
- `internal/modules/normalize/<name>` or `internal/modules/normalize/<domain>/<name>`: normalize-stage modules.
|
||||
- `internal/modules/output/<name>`: output-stage modules.
|
||||
@@ -120,7 +120,7 @@ workflow visible in the filesystem:
|
||||
```text
|
||||
internal/modules/input/...
|
||||
internal/modules/chunk/...
|
||||
internal/modules/process/...
|
||||
internal/modules/extract/...
|
||||
internal/modules/merge/...
|
||||
internal/modules/normalize/...
|
||||
internal/modules/output/...
|
||||
@@ -141,7 +141,7 @@ decisions.
|
||||
|
||||
Other packages should interact with source input through adapter contracts and
|
||||
core source types. Input module implementation details and external dependency
|
||||
types must not leak into framework or process module packages.
|
||||
types must not leak into framework or extract module packages.
|
||||
|
||||
Input module metadata may preserve source-specific facts such as transcript speaker,
|
||||
timestamps, Markdown heading path, page number, or block ID. Framework code may
|
||||
@@ -152,7 +152,7 @@ shape.
|
||||
|
||||
Extractors are independent modules that process source chunks or whole source
|
||||
documents and produce one kind of structured artifact candidate.
|
||||
Each process module owns:
|
||||
Each extract module owns:
|
||||
|
||||
- its artifact semantics;
|
||||
- its prompt usage;
|
||||
@@ -160,7 +160,7 @@ Each process module owns:
|
||||
- its validator chain;
|
||||
- any domain-specific mapping or interpretation.
|
||||
|
||||
Process modules should depend on framework contracts and core source/artifact
|
||||
Extract modules should depend on framework contracts and core source/artifact
|
||||
types. They should not depend on concrete input module packages.
|
||||
|
||||
Extractors should not be the only place where chunking, merging, or
|
||||
@@ -181,7 +181,7 @@ The pipeline has six conceptual stages:
|
||||
|
||||
1. input: external source material becomes a `SourceDocument`;
|
||||
2. chunk: a `SourceDocument` becomes ordered source chunks;
|
||||
3. process: extractors produce artifact candidates from chunks or whole documents;
|
||||
3. extract: extractors produce artifact candidates from chunks or whole documents;
|
||||
4. merge: per-chunk candidates become a merged candidate collection;
|
||||
5. normalize: merged candidates are reconciled for duplicates, aliases, consistency, or cross-chunk issues;
|
||||
6. output: final artifacts are serialized.
|
||||
@@ -290,11 +290,11 @@ be opt-in.
|
||||
## Testing
|
||||
|
||||
Core logic should be testable without real external services. Use fakes,
|
||||
fixtures, or local test doubles for adapters, extractors, validators, and LLM
|
||||
clients where practical.
|
||||
fixtures, or local test doubles for input modules, extract modules, validators,
|
||||
and LLM clients where practical.
|
||||
|
||||
Contract-first work should include fake implementations that prove interfaces
|
||||
compose before real adapters or extractors depend on them.
|
||||
compose before real input modules or extract modules depend on them.
|
||||
|
||||
Config examples should be load-tested once config files exist. Important CLI
|
||||
workflows should have parser or command tests. Adapter, extractor, and validator
|
||||
@@ -312,7 +312,7 @@ unit, source reference, input adapter, extractor, chunker, merger, normalizer,
|
||||
artifact, validator, and run manifest.
|
||||
|
||||
Source-format details belong in input module or integration docs.
|
||||
Domain-specific extraction details belong in process module or artifact docs.
|
||||
Domain-specific extraction details belong in extract module or artifact docs.
|
||||
|
||||
When changing architecture, config, CLI behavior, stage modules, extractor
|
||||
contracts, validator contracts, LLM runtime behavior, or artifact schemas, update
|
||||
|
||||
Reference in New Issue
Block a user