Prepare roadmap for documentation policy update
This commit is contained in:
380
docs/roadmap/notarius-extract-stage.md
Normal file
380
docs/roadmap/notarius-extract-stage.md
Normal file
@@ -0,0 +1,380 @@
|
||||
# Notarius Extraction Stage
|
||||
|
||||
## Status
|
||||
|
||||
Proposed.
|
||||
|
||||
## Purpose
|
||||
|
||||
Add a first-class Narratio `extract` stage that runs Notarius against the
|
||||
session's final trimmed transcript, validates and collects the resulting
|
||||
structured D&D artifacts, and registers those artifacts for later use by the
|
||||
`analyze` and `publish` stages.
|
||||
|
||||
This feature should integrate Notarius through Narratio's existing stage,
|
||||
adapter, manifest, workspace, and artifact-catalog boundaries. It must not turn
|
||||
Narratio into a generic workflow engine or a second configuration language for
|
||||
Notarius pipelines.
|
||||
|
||||
## User Outcome
|
||||
|
||||
An operator can enable one configured Notarius pipeline for a Narratio
|
||||
campaign. During a normal run, Narratio will:
|
||||
|
||||
1. finish producing the session transcript tiers;
|
||||
2. invoke Notarius once with the final trimmed Seriatim JSON transcript;
|
||||
3. collect and validate the configured structured artifact lanes;
|
||||
4. record their exact files and provenance in the Narratio manifest; and
|
||||
5. make those artifacts selectable as inputs to Scriptorium artifacts in the
|
||||
later `analyze` stage.
|
||||
|
||||
The maintained D&D example should demonstrate all ten lanes emitted by
|
||||
Notarius's complete `dnd-session` pipeline.
|
||||
|
||||
## Target Stage Architecture
|
||||
|
||||
### Canonical Order
|
||||
|
||||
The canonical stage order becomes:
|
||||
|
||||
```text
|
||||
prepare -> transcribe -> merge -> polish -> normalize -> trim -> render
|
||||
-> extract -> analyze -> publish -> notify
|
||||
```
|
||||
|
||||
`extract` is deliberately after all transcript-producing stages and before
|
||||
analysis. Its source document is the manifest-resolved
|
||||
`narratio.transcript.final_trimmed` artifact, normally
|
||||
`transcripts/final.trimmed.json`. It does not consume rendered Markdown.
|
||||
|
||||
Adding the stage must update full-plan construction, explicit stage selection,
|
||||
downstream invalidation, prerequisite checks, resume behavior, run manifests,
|
||||
CLI stage validation and help, and every canonical-stage inventory. Forcing an
|
||||
upstream transcript stage must stale a previously successful `extract` stage
|
||||
and its downstream stages. Forcing `extract` must stale `analyze`, `publish`,
|
||||
and `notify` according to existing rules.
|
||||
|
||||
### Stage Boundary
|
||||
|
||||
The stage owns Narratio policy and state transitions:
|
||||
|
||||
- resolve the final trimmed transcript through the runtime artifact catalog;
|
||||
- build a Narratio-level Notarius request from validated configuration and
|
||||
run-local paths;
|
||||
- call a narrow Notarius adapter;
|
||||
- apply the configured required-output policy;
|
||||
- materialize the validated bundle into its canonical session location;
|
||||
- return manifest-ready artifact references and bounded metadata; and
|
||||
- fail without marking the stage successful when any required contract or
|
||||
materialization step fails.
|
||||
|
||||
The stage must not construct subprocess arguments, infer Notarius output
|
||||
filenames, parse provider logs, or decode individual D&D payload bodies.
|
||||
|
||||
### Adapter Boundary
|
||||
|
||||
Add a dedicated Notarius adapter package with a small interface, production
|
||||
subprocess implementation, and test fake. Its request should contain only the
|
||||
resolved Notarius binary, configuration path, pipeline ID, transcript path,
|
||||
output root, working directory, timeout, and process-log destinations needed
|
||||
for one run.
|
||||
|
||||
The adapter owns:
|
||||
|
||||
- optional `notarius config validate` preflight for the configured pipeline;
|
||||
- exact `notarius run ... --json` argument construction;
|
||||
- stdout and stderr separation;
|
||||
- context cancellation and timeout propagation through Narratio's shared
|
||||
subprocess boundary;
|
||||
- exit-status handling;
|
||||
- decoding the `notarius.run-result.v1` success receipt;
|
||||
- receipt and index path-confinement checks;
|
||||
- decoding `index.json` and resolving descriptor paths safely beneath the
|
||||
reported output directory; and
|
||||
- returning a transport-neutral result containing the bundle location,
|
||||
receipt summary, lane descriptors, pipeline-wide descriptors, warnings and
|
||||
rejection locations, and diagnostic log paths.
|
||||
|
||||
Only exit status zero permits receipt decoding. Receipt, index, or descriptor
|
||||
paths that are absolute where a logical relative path is required, or that
|
||||
escape their owning root, are integration failures. Unknown fields in a
|
||||
supported receipt or index schema should be tolerated. Unsupported schema
|
||||
versions and incompatible descriptor metadata should fail clearly.
|
||||
|
||||
The adapter must not write Narratio manifests, choose required lanes, decide
|
||||
analysis inputs, or contain D&D domain logic.
|
||||
|
||||
## Configuration Contract
|
||||
|
||||
Add a strict optional `pipeline.notarius` configuration section. Omission or
|
||||
`enabled: false` keeps the current workflow usable and causes `extract` to
|
||||
self-skip without outputs.
|
||||
|
||||
The section should provide:
|
||||
|
||||
- `enabled`: explicit opt-in;
|
||||
- `binary`: Notarius executable, defaulting to `notarius`;
|
||||
- `config_path`: required when enabled;
|
||||
- `pipeline_id`: required when enabled;
|
||||
- `timeout`: a positive stage timeout with a documented default;
|
||||
- `working_directory`: optional explicit subprocess working directory,
|
||||
defaulting to the directory containing `config_path`; and
|
||||
- an `outputs` map defining the Notarius lane artifacts Narratio promises to
|
||||
collect.
|
||||
|
||||
Each output-map key is a stable Narratio extraction key. Each value must define:
|
||||
|
||||
- the exact Notarius `lane_id`;
|
||||
- the expected `media_type`;
|
||||
- the expected `schema_id`;
|
||||
- the expected `schema_version`; and
|
||||
- optionally an expected `module_key` when the operator needs to constrain the
|
||||
producing module as part of compatibility.
|
||||
|
||||
Narratio derives the downstream source ID
|
||||
`narratio.extraction.<output-key>` from the map key. Keys and lane IDs must be
|
||||
non-empty, unique after normalization, path-safe under the existing artifact
|
||||
policy, and collision-free with built-in and configured artifact identities.
|
||||
Every configured output is required: a successful Notarius process that omits
|
||||
one, rejects it, or reports incompatible descriptor metadata fails the
|
||||
`extract` stage.
|
||||
|
||||
This explicit map keeps Narratio's consumer contract stable when a Notarius
|
||||
lane ID or schema changes and avoids hard-coding the current D&D family into a
|
||||
generic adapter. It also replaces a separate `required_lanes` list, which would
|
||||
duplicate configuration.
|
||||
|
||||
Narratio should not reproduce Notarius lane selection, references, LLM
|
||||
profiles, model settings, retries, concurrency, or prompt configuration. Those
|
||||
remain in the referenced Notarius configuration. Narratio should not expose a
|
||||
runtime lane-selection flag for `extract`; one stage invocation runs the
|
||||
configured Notarius pipeline as a unit.
|
||||
|
||||
All configured paths should become absolute during Narratio configuration
|
||||
resolution. The deterministic default working directory allows a Notarius
|
||||
profile path relative to that directory, but operator documentation should
|
||||
still recommend absolute deployment paths where practical. Notarius reference
|
||||
paths continue to follow Notarius's own configuration-relative rules.
|
||||
|
||||
## Output And Artifact Model
|
||||
|
||||
### Canonical Bundle
|
||||
|
||||
Run Notarius against a run-local output root. After all configured descriptors
|
||||
are validated, materialize the contents of the exact run-specific Notarius
|
||||
bundle into a fixed canonical session directory:
|
||||
|
||||
```text
|
||||
artifacts/notarius/
|
||||
```
|
||||
|
||||
Preserve its relative layout, including `index.json`, `manifest.json`,
|
||||
`rejected.json`, `warnings.json`, `lanes/`, and any indexed `chunk-map.json` or
|
||||
`evidence-context.json`. Materialize the complete directory as one narrow,
|
||||
transactional replacement so a failed or interrupted rerun cannot mix files
|
||||
from different Notarius runs.
|
||||
|
||||
The raw subprocess receipt and stderr log belong in the run-local `extract`
|
||||
report and log directories. The raw receipt identifies the original run-local
|
||||
Notarius bundle and must not be rewritten to pretend that the canonical copy
|
||||
was its original `output_directory`. Narratio's manifest is the durable ledger
|
||||
for the canonical materialized paths.
|
||||
|
||||
### Registered Artifact Sources
|
||||
|
||||
For each configured output, locate the lane through the canonical copy of
|
||||
`index.json` and record a manifest artifact with:
|
||||
|
||||
- source ID `narratio.extraction.<output-key>`;
|
||||
- canonical lane-file path discovered from the index;
|
||||
- producer stage and Narratio run ID;
|
||||
- checksum;
|
||||
- Notarius lane ID; and
|
||||
- descriptor media type, schema identity/version, and module key when present.
|
||||
|
||||
If the current manifest model cannot carry descriptor compatibility metadata,
|
||||
extend its artifact metadata in a backward-tolerant way rather than encoding
|
||||
that information in filenames or source IDs.
|
||||
|
||||
Also record the canonical Notarius index as a stage output or stage metadata so
|
||||
operators can discover the complete bundle, including non-lane artifacts. The
|
||||
configured lane sources are the stable interface for analysis; the index and
|
||||
bundle remain the provenance and inspection interface.
|
||||
|
||||
## Analysis And Publish Integration
|
||||
|
||||
Extend the runtime artifact catalog and configured Scriptorium input validation
|
||||
so an enabled analysis artifact can declare, for example:
|
||||
|
||||
```yaml
|
||||
inputs:
|
||||
npc_registry:
|
||||
source: narratio.extraction.npc_registry
|
||||
```
|
||||
|
||||
Resolution must remain manifest-first and verify that the recorded artifact
|
||||
was produced by a successful current `extract` stage. A required extraction
|
||||
source that is unavailable must fail analysis with guidance to configure or
|
||||
rerun `extract`; an optional source may be omitted according to the existing
|
||||
Scriptorium input contract.
|
||||
|
||||
Publish source resolution should accept configured
|
||||
`narratio.extraction.<output-key>` sources through the same artifact catalog so
|
||||
operators may publish selected structured artifacts without manually copying
|
||||
paths. The existing `--artifacts` flag remains scoped to Scriptorium artifact
|
||||
selection and must not partially execute the Notarius pipeline.
|
||||
|
||||
No current-session analysis artifact should consume an incidental file from a
|
||||
failed, stale, skipped, or superseded extraction run.
|
||||
|
||||
## Failure, Skip, Resume, And Diagnostics
|
||||
|
||||
- Missing or invalid enabled Notarius configuration fails configuration
|
||||
validation before stage execution where statically discoverable.
|
||||
- A disabled or absent Notarius configuration makes `extract` skip with clear
|
||||
stage metadata and no new outputs.
|
||||
- A missing or invalid final trimmed transcript fails `extract` before starting
|
||||
Notarius.
|
||||
- Preflight failure, nonzero Notarius exit, cancellation, timeout, malformed or
|
||||
unsupported receipt/index data, unsafe paths, incompatible descriptors,
|
||||
rejected required outputs, or missing configured lanes fails the entire
|
||||
stage.
|
||||
- Process success does not override Narratio's required-output policy.
|
||||
- A failed run retains bounded run-local receipt bytes, stderr, and the
|
||||
unpublished Notarius bundle for diagnosis, subject to Narratio's existing
|
||||
sensitive-data and cleanup policies.
|
||||
- The canonical bundle and manifest artifacts are updated only after complete
|
||||
validation and materialization.
|
||||
- Resume skips a succeeded, non-stale `extract` stage only when its
|
||||
manifest-recorded canonical index and configured lane outputs still validate.
|
||||
- Force and staleness behavior follows the ordinary stage contract; it must not
|
||||
depend on merely finding `artifacts/notarius/` on disk.
|
||||
|
||||
Transcripts, Notarius outputs, evidence context, manifests, receipts, and logs
|
||||
are private campaign material. Subprocess arguments and manifest metadata must
|
||||
not contain secrets. Credentials remain in the environment or in mechanisms
|
||||
owned by Notarius and PromptKit.
|
||||
|
||||
## Maintained D&D Example
|
||||
|
||||
Add or update a Narratio example that enables Notarius's complete
|
||||
`dnd-session` pipeline and maps these ten required lanes to stable extraction
|
||||
keys:
|
||||
|
||||
| Output key | Notarius lane ID |
|
||||
| --- | --- |
|
||||
| `item_registry` | `item-registry` |
|
||||
| `npc_registry` | `npc-registry` |
|
||||
| `location_registry` | `location-registry` |
|
||||
| `scene_descriptions` | `scene-descriptions` |
|
||||
| `item_occurrences` | `item-occurrences` |
|
||||
| `spells` | `spells` |
|
||||
| `combat_turns` | `combat-turns` |
|
||||
| `npc_occurrences` | `npc-occurrences` |
|
||||
| `location_occurrences` | `location-occurrences` |
|
||||
| `enemy_events` | `enemy-events` |
|
||||
|
||||
The example must include each lane's current media type and schema identity
|
||||
from Notarius's published contracts. It should also demonstrate at least one
|
||||
Scriptorium analysis artifact consuming one or more
|
||||
`narratio.extraction.*` sources. The example must use placeholders and relative
|
||||
paths suitable for the example tree, contain no credentials, and pass the
|
||||
repository's configuration validation tests.
|
||||
|
||||
## Compatibility Policy
|
||||
|
||||
The initial integration baseline is the public subprocess contract available
|
||||
in Notarius v0.3.0:
|
||||
|
||||
- successful JSON receipt schema `notarius.run-result.v1`;
|
||||
- production JSON bundle discovery through `index.json`; and
|
||||
- the schema IDs and versions explicitly configured for required lanes.
|
||||
|
||||
Runtime compatibility should be decided from those published contracts, not
|
||||
from textual parsing of `notarius --version`. New optional receipt or index
|
||||
fields must not break Narratio. An unsupported receipt version or lane schema
|
||||
must fail before the artifact is registered for analysis.
|
||||
|
||||
## Documentation Deliverables When Implemented
|
||||
|
||||
Update current-behavior documentation in the same change that implements the
|
||||
feature:
|
||||
|
||||
- add `docs/integrations/notarius.md` for the external CLI, receipt, bundle,
|
||||
and adapter contract, linking to Notarius's canonical documentation;
|
||||
- add `docs/internal/stage-extract.md` for stage inputs, outputs, collaborators,
|
||||
state transitions, failures, and focused tests;
|
||||
- update `docs/internal/adapters.md`, `docs/internal/artifacts.md`,
|
||||
`docs/internal/manifest.md`, and the internal stage inventory;
|
||||
- update `docs/architecture.md` to list Notarius among isolated external
|
||||
systems and preserve the adapter/stage boundary;
|
||||
- update `docs/config.md`, `docs/cli.md`, `docs/operations.md`,
|
||||
`docs/troubleshooting.md`, `README.md`, and maintained examples only to the
|
||||
extent their canonical scopes require; and
|
||||
- add the missing `docs/development.md` required by the repository's
|
||||
documentation policy, or track that repository-wide documentation gap in a
|
||||
separate roadmap if it is intentionally outside the implementation scope.
|
||||
|
||||
Outside this roadmap, do not describe `extract`, Notarius configuration, or
|
||||
`narratio.extraction.*` sources as implemented until the code exists.
|
||||
|
||||
## Testing And Validation Expectations
|
||||
|
||||
Implementation should provide focused tests for:
|
||||
|
||||
- strict configuration decoding, defaults, required fields, path resolution,
|
||||
output-map validation, normalized-key collisions, and example loading;
|
||||
- exact stage order, selection, downstream staleness, resume, force, and
|
||||
prerequisite behavior;
|
||||
- adapter command construction, deterministic working directory, environment
|
||||
inheritance, stdout/stderr separation, cancellation, timeout, and nonzero
|
||||
exits;
|
||||
- supported and unsupported receipt versions, unknown optional fields,
|
||||
malformed receipts, index decoding, and path escapes at every boundary;
|
||||
- descriptor lookup by lane ID rather than filename, expected metadata checks,
|
||||
missing/rejected configured lanes, and tolerated unconfigured lanes;
|
||||
- run-local execution, transactional canonical-bundle replacement, checksums,
|
||||
failed-run preservation, and manifest recording;
|
||||
- artifact-catalog resolution from `narratio.extraction.*` into analysis and
|
||||
publish, including required, optional, missing, stale, and skipped cases; and
|
||||
- end-to-end stage execution with a fake Notarius adapter, without live LLM or
|
||||
external subprocess requirements in the ordinary test suite.
|
||||
|
||||
Run the repository-wide Go tests, vet, build, and maintained example validation
|
||||
after focused tests pass.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- `extract` is a first-class transactional stage between `render` and
|
||||
`analyze` everywhere Narratio models stage order or state.
|
||||
- Narratio invokes Notarius only through a narrow, tested adapter.
|
||||
- The stage consumes the manifest-resolved final trimmed Seriatim transcript.
|
||||
- The Notarius configuration remains owned by Notarius; Narratio configures
|
||||
only invocation and its downstream consumer contract.
|
||||
- Every configured output is discovered through the receipt and `index.json`,
|
||||
contract-checked, materialized transactionally, and recorded with a stable
|
||||
`narratio.extraction.*` source ID.
|
||||
- The complete D&D example maps all ten current lanes and passes strict config
|
||||
validation.
|
||||
- Analysis can consume extraction sources through the existing artifact input
|
||||
model, and publish can select them through the artifact catalog.
|
||||
- Failed, partial, rejected, unsafe, stale, or incompatible output never becomes
|
||||
a current analysis input.
|
||||
- Resume and force behavior remains manifest-driven.
|
||||
- Documentation accurately describes the implemented stage, adapter,
|
||||
configuration, operations, and compatibility boundary without duplicating
|
||||
Notarius's canonical schemas.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- Reimplementing Notarius extraction, prompts, schemas, references, retries,
|
||||
profiles, or lane orchestration in Narratio.
|
||||
- Allowing one Narratio run to invoke arbitrary extractor programs or multiple
|
||||
Notarius pipelines.
|
||||
- Making `extract` a configurable DAG or folding it into the Scriptorium
|
||||
`analyze` stage.
|
||||
- Partially selecting Notarius lanes through Narratio's `--artifacts` flag.
|
||||
- Decoding D&D payload bodies in the generic Notarius adapter.
|
||||
- Supporting previous-session extraction artifacts in the initial feature.
|
||||
- Requiring live Notarius, PromptKit, an LLM provider, or external services in
|
||||
the ordinary unit test suite.
|
||||
Reference in New Issue
Block a user