Finish the references implementation for the extraction module and update roadmap documentation

This commit is contained in:
2026-07-05 10:53:16 -05:00
parent be6803ffa1
commit a516944086
13 changed files with 280 additions and 562 deletions

View File

@@ -8,6 +8,21 @@ future work only.
- Additional input adapters, such as Markdown or note-export formats.
- Additional D&D extractors beyond spell casts.
- First-class LLM access for all LLM-eligible pipeline stages: `chunk`,
`extract`, and `normalize`.
- Expand reference support beyond extractors to `chunk` and `normalize`
modules, using the same declared-slot model and preserving stage-specific
ownership boundaries.
- Add non-file reference producers, such as prior-run artifacts, derived
summaries, or entity registries, without changing module-facing reference
item contracts.
- Add token budgeting and model context-window management for reference content.
- Add per-slot or per-chunk inclusion policies so modules can avoid repeating
large reference content in every prompt when that becomes important.
- Add structured or parsed references, such as typed roster schemas, when a
module has a clear need for more than opaque UTF-8 text.
- Add reference caching, preprocessing, summarization, embedding, or retrieval
if references become large enough to require preprocessing.
- Cross-lane entity normalization.
- Cross-chunk semantic deduplication.
- Configurable validator chains with production validator modules.
@@ -15,6 +30,13 @@ future work only.
- Parallel execution where it preserves deterministic manifests and diagnostics.
- Additional output encoders.
## Candidate Architecture Work
- Evaluate replacing the local LLM adapter with an import from
`gitea.maximumdirect.net/eric/scriptorium`, provided it preserves Notarius
boundaries around provider plumbing, prompt ownership, diagnostics, and secret
handling.
## Candidate Operational Work
- Packaged release artifacts for alpha distribution.