Remove completed feature roadmaps
This commit is contained in:
@@ -1,810 +0,0 @@
|
|||||||
# Post-Transcript Artifact Workflow Implementation Plan
|
|
||||||
|
|
||||||
## Purpose And Status
|
|
||||||
|
|
||||||
This document is the executable implementation plan for
|
|
||||||
[`post-transcript-artifact-workflow.md`](post-transcript-artifact-workflow.md).
|
|
||||||
That roadmap owns the accepted user intent, compatibility policy, and target
|
|
||||||
state. This plan translates it into bounded stages suitable for one
|
|
||||||
`gpt-5.6-terra` implementation prompt apiece.
|
|
||||||
|
|
||||||
All stages below are completed. They were implemented in numeric order, with
|
|
||||||
each later stage relying on the tested and documented contracts established by
|
|
||||||
its predecessors. This accepted roadmap remains available for closeout history;
|
|
||||||
it does not replace the canonical current-behavior documents linked from
|
|
||||||
`docs/development.md`.
|
|
||||||
|
|
||||||
## Settled Implementation Decisions
|
|
||||||
|
|
||||||
The following choices make the plan decision-complete:
|
|
||||||
|
|
||||||
- The canonical order is `prepare`, `transcribe`, `merge`, `polish`,
|
|
||||||
`normalize`, `trim`, `render`, `extract`, `analyze`, `publish`, `notify`.
|
|
||||||
- Execution order remains a flat, fixed registry. Invalidation uses a separate
|
|
||||||
fixed direct-dependency table whose transitive closure is returned in
|
|
||||||
canonical order. It is not configurable and is not an execution DAG.
|
|
||||||
- `run` and `session plan` share one inclusive contiguous-range selector.
|
|
||||||
Omitted endpoints select the beginning or end of the canonical registry.
|
|
||||||
- A bounded `run` requires every excluded prefix stage to have a terminal
|
|
||||||
manifest status of `succeeded` or `skipped`. The first absent, pending,
|
|
||||||
running, failed, stale, or interrupted prerequisite is an actionable error.
|
|
||||||
Excluded prefix stages are not resume-validated or repaired; selected stages
|
|
||||||
still validate the concrete inputs they consume. Stages after `--through`
|
|
||||||
are not prerequisites and receive no analogous check.
|
|
||||||
- `--force` affects only selected pipeline stages. Invalidation may mark a
|
|
||||||
dependent outside the range stale, but cannot execute it.
|
|
||||||
- `--artifacts` is valid only when the selected range contains `analyze` or
|
|
||||||
`publish`. Its existing repeatable, comma-separated normalization and
|
|
||||||
deduplication semantics remain unchanged.
|
|
||||||
- `regenerate-artifacts` performs argument expansion and calls the canonical
|
|
||||||
`run` handler. It owns no planner, runner, prerequisite, or force behavior.
|
|
||||||
Duplicate `--force`, `--from`, or `--through` options are rejected by the
|
|
||||||
shared bounded-run parser, including `--name=value` spellings.
|
|
||||||
- Incremental state is specific to configured Scriptorium artifacts inside the
|
|
||||||
fixed `analyze` stage. Do not create dynamic stages, generic jobs, or a
|
|
||||||
general subtask framework.
|
|
||||||
- The session manifest owns current artifact availability. A configured output
|
|
||||||
file found on disk without current manifest evidence is unavailable.
|
|
||||||
- Analysis fingerprints contain only Narratio-observable semantic inputs. They
|
|
||||||
exclude run IDs, absolute workspace paths, timeouts, and executable contents.
|
|
||||||
They also cannot observe arbitrary files or transitive configuration loaded
|
|
||||||
privately by Scriptorium; those changes require explicit force.
|
|
||||||
- Partial analysis preserves unrelated current records. On an error, the
|
|
||||||
runner accepts a restricted analyze-state result alongside the error so it
|
|
||||||
can record completed, failed, and newly stale artifact state before marking
|
|
||||||
the aggregate stage failed. It must never promote unvalidated files.
|
|
||||||
- Legacy aggregate-only analysis output remains readable but is not current
|
|
||||||
evidence. Only regenerated artifacts enter the new per-artifact state.
|
|
||||||
|
|
||||||
## Instructions For Every Stage
|
|
||||||
|
|
||||||
Before changing code in each stage:
|
|
||||||
|
|
||||||
1. Read `docs/development.md`, its task-specific references, all documents in
|
|
||||||
`docs/policy/`, this plan, and the relevant portions of the feature roadmap.
|
|
||||||
2. Inspect the current implementation before editing. Prefer the codebase
|
|
||||||
knowledge graph for code discovery, then read the exact owning files.
|
|
||||||
3. Confirm the worktree state and preserve unrelated user changes.
|
|
||||||
|
|
||||||
During each stage:
|
|
||||||
|
|
||||||
- Keep changes within that stage's scope and the accepted roadmap.
|
|
||||||
- Follow the architecture policy's dependency direction, filesystem safety,
|
|
||||||
manifest authority, and application-owned lifecycle rules.
|
|
||||||
- Follow the testing policy: place behavior at its narrowest public owner, use
|
|
||||||
fakes at owned boundaries, keep the default suite offline and deterministic,
|
|
||||||
and avoid duplicating the same assertion across layers.
|
|
||||||
- Update a canonical current-behavior document in the same stage when that
|
|
||||||
stage changes already usable behavior. Do not describe unimplemented later
|
|
||||||
stages as current.
|
|
||||||
- Preserve backward-compatible manifest reads and emit only the new canonical
|
|
||||||
representation after its writer is introduced.
|
|
||||||
- Run focused tests during development, then `go test ./...` before completing
|
|
||||||
the stage. Run the repository's required formatting, static checks, and
|
|
||||||
documentation checks identified by `docs/development.md`.
|
|
||||||
|
|
||||||
At the end of each stage, leave a cohesive change that can be reviewed and
|
|
||||||
committed independently. Do not begin a later stage while an earlier stage has
|
|
||||||
failing tests or incomplete acceptance criteria.
|
|
||||||
|
|
||||||
## Stage 1 — Canonical Order And Dependency-Aware Invalidation
|
|
||||||
|
|
||||||
**Status: Completed**
|
|
||||||
|
|
||||||
### Goal
|
|
||||||
|
|
||||||
Move `render` before `extract` and replace suffix-based invalidation with the
|
|
||||||
fixed dependency relation required by the feature roadmap.
|
|
||||||
|
|
||||||
### Required Work
|
|
||||||
|
|
||||||
1. Change the single canonical stage registry so `render` immediately follows
|
|
||||||
`trim` and `extract` immediately follows `render`. Do not add a second stage
|
|
||||||
list in a command or test helper.
|
|
||||||
2. In the application orchestration owner, define a fixed table of direct
|
|
||||||
invalidation edges:
|
|
||||||
- `prepare -> transcribe`
|
|
||||||
- `transcribe -> merge`
|
|
||||||
- `merge -> polish`
|
|
||||||
- `polish -> normalize`
|
|
||||||
- `normalize -> trim`
|
|
||||||
- `trim -> render, extract`
|
|
||||||
- `render -> analyze`
|
|
||||||
- `extract -> analyze`
|
|
||||||
- `analyze -> publish`
|
|
||||||
- `publish -> notify`
|
|
||||||
- `notify ->` none
|
|
||||||
3. Compute transitive dependents from that table and return them in canonical
|
|
||||||
order. Unknown stages must return an error; callers must not silently treat
|
|
||||||
them as having no dependents.
|
|
||||||
4. Validate the relation against the canonical registry when it is constructed
|
|
||||||
or first used. Reject duplicate registry names, unknown edge endpoints,
|
|
||||||
cycles, missing stage classifications, and registry/table drift.
|
|
||||||
5. Route every existing invalidation trigger through this owner: force,
|
|
||||||
non-resumable success, failure, and changed effective outcome. Retain the
|
|
||||||
existing rule that only succeeded dependent records become stale and retain
|
|
||||||
the existing no-op treatment for an identical repeated self-skip.
|
|
||||||
6. Remove assumptions that invalidation is the canonical suffix. In
|
|
||||||
particular, `render` and `extract` must never invalidate one another, while
|
|
||||||
either invalidates `analyze`, `publish`, and `notify`.
|
|
||||||
7. Update the architecture policy, internal overview, and focused render,
|
|
||||||
extract, and manifest documentation for the implemented order and separate
|
|
||||||
invalidation owner.
|
|
||||||
|
|
||||||
### Tests And Exit Criteria
|
|
||||||
|
|
||||||
- Registry/planner tests assert the complete new order.
|
|
||||||
- Table-driven invalidation tests assert every transitive set in the feature
|
|
||||||
roadmap, canonical output ordering, render/extract independence, unknown
|
|
||||||
stages, cycle rejection, and inventory drift rejection.
|
|
||||||
- Representative runner tests prove each invalidation trigger uses the new
|
|
||||||
relation and does not execute newly stale stages implicitly.
|
|
||||||
- Existing manifests with stable stage names remain readable and resumable;
|
|
||||||
no version-based invalidation or manifest migration is introduced.
|
|
||||||
|
|
||||||
## Stage 2 — Shared Bounded Plan Selection
|
|
||||||
|
|
||||||
**Status: Completed**
|
|
||||||
|
|
||||||
### Goal
|
|
||||||
|
|
||||||
Give the application one validated representation of a contiguous canonical
|
|
||||||
stage range, shared by execution and plan preview.
|
|
||||||
|
|
||||||
### Required Work
|
|
||||||
|
|
||||||
1. Extend the application planner with a bounded-plan constructor accepting
|
|
||||||
optional `from` and `through` names. The returned plan contains an inclusive
|
|
||||||
contiguous slice of the canonical registry.
|
|
||||||
2. Default an omitted `from` to the first stage and an omitted `through` to the
|
|
||||||
last stage. Preserve the current full plan when both are omitted.
|
|
||||||
3. Reject unknown endpoints and a `from` endpoint occurring after `through`.
|
|
||||||
Errors must name the bad value or reversed pair and list or point to valid
|
|
||||||
canonical stage names.
|
|
||||||
4. Put range membership and endpoint information on one application-level
|
|
||||||
value so `run`, `session plan`, prerequisite validation, and adapter
|
|
||||||
composition do not independently recalculate bounds.
|
|
||||||
5. Keep `run-stage` and the dedicated `analyze` and `publish` conveniences on
|
|
||||||
their existing single-stage paths. Do not reinterpret them through new
|
|
||||||
range flags.
|
|
||||||
|
|
||||||
### Tests And Exit Criteria
|
|
||||||
|
|
||||||
- Planner tests cover every endpoint, one-stage ranges, omitted starts, omitted
|
|
||||||
ends, a full omitted range, unknown names, reversed ranges, and stable order.
|
|
||||||
- Tests assert that selecting a range cannot create a non-contiguous plan or
|
|
||||||
mutate the canonical registry.
|
|
||||||
- No CLI behavior changes in this stage; the new planner API is ready for both
|
|
||||||
`run` and `session plan` to consume next.
|
|
||||||
|
|
||||||
## Stage 3 — Bounded `run` And `session plan` Command Contracts
|
|
||||||
|
|
||||||
**Status: Completed**
|
|
||||||
|
|
||||||
### Goal
|
|
||||||
|
|
||||||
Expose the shared range through both commands with one parsing and structural
|
|
||||||
validation contract.
|
|
||||||
|
|
||||||
### Required Work
|
|
||||||
|
|
||||||
1. Add repeat-safe singleton option parsing for `--from`, `--through`, and
|
|
||||||
`--force` to the shared bounded-run command layer. Reject duplicates whether
|
|
||||||
written as separate arguments or `--name=value`; never use argument order to
|
|
||||||
choose a winner.
|
|
||||||
2. Preserve the existing session-oriented argument conventions and common
|
|
||||||
configuration flags. Preserve repeatable/comma-separated `--artifacts`
|
|
||||||
normalization and deduplication.
|
|
||||||
3. Add `--from` and `--through` to `run`, and pass the resulting bounded plan to
|
|
||||||
the existing runner rather than filtering stages after planning.
|
|
||||||
4. Add the same bounds, force flag, and artifact selection to `session plan`.
|
|
||||||
Both commands must call the Stage 2 selector and surface identical range
|
|
||||||
validation errors.
|
|
||||||
5. Reject `--artifacts` when the range contains neither `analyze` nor `publish`.
|
|
||||||
Accept it when either consumer is present, including a single-stage range.
|
|
||||||
6. Update command help, usage errors, `docs/cli.md`, and `docs/operations.md` for
|
|
||||||
the inclusive/defaulting behavior. Do not document the alias until Stage 5.
|
|
||||||
|
|
||||||
### Tests And Exit Criteria
|
|
||||||
|
|
||||||
- Command tests cover valid ranges, defaults, invalid/reversed ranges,
|
|
||||||
singleton duplicates in both syntaxes, and artifact/range compatibility.
|
|
||||||
- Unbounded `run` and unbounded `session plan` retain their prior behavior
|
|
||||||
except for the canonical order implemented in Stage 1.
|
|
||||||
- Parsing tests assert structural equivalence between the run and plan range
|
|
||||||
values, without duplicating all planner cases at the command layer.
|
|
||||||
|
|
||||||
## Stage 4 — Bounded Runner Prerequisites And Composition
|
|
||||||
|
|
||||||
**Status: Completed**
|
|
||||||
|
|
||||||
### Goal
|
|
||||||
|
|
||||||
Make bounded execution honor its mutation boundary while still failing safely
|
|
||||||
when excluded upstream work cannot support the selected stages.
|
|
||||||
|
|
||||||
### Required Work
|
|
||||||
|
|
||||||
1. Before starting a bounded run, inspect the session manifest for every
|
|
||||||
canonical stage before `--from`. Accept only `succeeded` or `skipped` as a
|
|
||||||
terminal prerequisite status. Treat an absent record and every other status
|
|
||||||
as unusable.
|
|
||||||
2. Report the first unusable prefix stage in canonical order, its status (or
|
|
||||||
absence), the selected start, and an actionable suggestion to widen the
|
|
||||||
range or recover that stage explicitly. Do not mutate the manifest, create a
|
|
||||||
run record, or invoke an adapter before this check succeeds.
|
|
||||||
3. Do not call resume validators for excluded prefix stages. Concrete selected
|
|
||||||
stages continue to resolve and validate their own manifest-authoritative
|
|
||||||
inputs, so an unsafe or missing artifact still fails at its owning boundary.
|
|
||||||
4. Do not inspect stages after `--through` as prerequisites. They may become
|
|
||||||
stale through Stage 1 invalidation, but must not be scheduled.
|
|
||||||
5. Ensure force decisions are calculated only for stages in the bounded plan.
|
|
||||||
A forced selected stage may stale dependencies outside the range but cannot
|
|
||||||
execute them.
|
|
||||||
6. Make production composition plan-aware. Initialize and validate only the
|
|
||||||
external adapters, object storage, remote locks, and other collaborators
|
|
||||||
needed by selected stages or by shared session lifecycle requirements.
|
|
||||||
Preserve each selected stage's current fail-fast configuration validation.
|
|
||||||
7. Keep post-publish cleanup conditional on publish actually executing.
|
|
||||||
8. Update operational and troubleshooting documentation for prerequisite
|
|
||||||
failures, recovery, excluded-stage behavior, and plan-aware composition.
|
|
||||||
|
|
||||||
### Tests And Exit Criteria
|
|
||||||
|
|
||||||
- Runner tests cover every unacceptable prefix status, accepted skipped
|
|
||||||
prerequisites, no prefix for a first-stage run, and no check after the end.
|
|
||||||
- Tests prove excluded stages are neither executed nor resume-validated and
|
|
||||||
that a failed prerequisite check performs no persistent run mutation.
|
|
||||||
- Adapter-composition tests prove a bounded render-only run does not require
|
|
||||||
Notarius or Scriptorium, an extract-only run requires Notarius but not
|
|
||||||
Scriptorium, and analyze requires only its actual collaborators.
|
|
||||||
- Tests prove force/invalidation can stale an out-of-range dependent without
|
|
||||||
executing it and that stop-on-selected-stage-failure remains unchanged.
|
|
||||||
|
|
||||||
## Stage 5 — Exact `regenerate-artifacts` Alias
|
|
||||||
|
|
||||||
**Status: Completed**
|
|
||||||
|
|
||||||
### Goal
|
|
||||||
|
|
||||||
Add the transparent convenience command without creating another orchestration
|
|
||||||
path.
|
|
||||||
|
|
||||||
### Required Work
|
|
||||||
|
|
||||||
1. Register the top-level `regenerate-artifacts` command.
|
|
||||||
2. Implement it only by constructing the equivalent canonical arguments and
|
|
||||||
invoking the shared `run` parser/handler:
|
|
||||||
|
|
||||||
```text
|
|
||||||
run <session_id> --force --from extract --through analyze [caller options]
|
|
||||||
```
|
|
||||||
|
|
||||||
Preserve common session/configuration arguments and all repeatable
|
|
||||||
`--artifacts` values.
|
|
||||||
3. Do not add an alias-specific loader, plan, runner, prerequisite check,
|
|
||||||
summary, artifact rule, or force rule. Canonical run diagnostics may call
|
|
||||||
the operation `run`.
|
|
||||||
4. Let the shared duplicate-singleton validation from Stage 3 reject a caller's
|
|
||||||
`--force`, `--from`, or `--through`, since the alias already supplies them.
|
|
||||||
5. Give the alias concise help that states the exact expansion, that extraction
|
|
||||||
always runs, that selected analysis artifacts and required prerequisites are
|
|
||||||
rebuilt, and that publish/notify never run.
|
|
||||||
6. Add the alias to `docs/cli.md` and the development workflow in
|
|
||||||
`docs/operations.md`.
|
|
||||||
|
|
||||||
### Tests And Exit Criteria
|
|
||||||
|
|
||||||
- A narrow command test captures the forwarded arguments or resulting shared
|
|
||||||
command request and proves exact equivalence, including option pass-through.
|
|
||||||
- Tests prove alias help does not execute, duplicate owned options fail through
|
|
||||||
the shared parser, and unknown/private alias options are not accepted.
|
|
||||||
- Do not duplicate runner integration cases under the alias name; its only
|
|
||||||
behavior is expansion.
|
|
||||||
|
|
||||||
## Stage 6 — Versioned Analyze-Artifact Manifest State
|
|
||||||
|
|
||||||
**Status: Completed**
|
|
||||||
|
|
||||||
### Goal
|
|
||||||
|
|
||||||
Introduce a backward-compatible, analyze-specific session-manifest model that
|
|
||||||
can represent independently current artifacts without treating them as stages.
|
|
||||||
|
|
||||||
### Required Work
|
|
||||||
|
|
||||||
1. Add an explicit analysis-state contract version to the `analyze` stage
|
|
||||||
record. Absence means legacy aggregate-only state; a present supported
|
|
||||||
version distinguishes a valid empty set from legacy data.
|
|
||||||
2. Add a map keyed by normalized configured artifact key. Each record must
|
|
||||||
contain:
|
|
||||||
- key and disposition/status (`current`, `stale`, `missing`, `failed`, or
|
|
||||||
`unselected`);
|
|
||||||
- fingerprint contract version and fingerprint when evaluated;
|
|
||||||
- normalized configured dependency keys;
|
|
||||||
- the existing output artifact record when current, including source ID,
|
|
||||||
contract, canonical confined path, and checksum, plus a separate
|
|
||||||
analyze-record output-size field (do not broaden every artifact schema
|
|
||||||
solely to carry this analyze-specific evidence);
|
|
||||||
- producer run ID and update time;
|
|
||||||
- a bounded non-secret error for failed work; and
|
|
||||||
- useful non-secret Scriptorium provenance, logs, and generated config paths
|
|
||||||
already allowed by manifest policy.
|
|
||||||
3. Add the corresponding analyze-artifact collection to invocation-stage
|
|
||||||
records. A session record describes the reconciled current set; an
|
|
||||||
invocation record describes only work evaluated or attempted by that run.
|
|
||||||
4. Keep the schema explicitly owned by analyze. Do not add dynamic stage names
|
|
||||||
or generic pipeline-subtask abstractions.
|
|
||||||
5. Extend validation and canonical serialization for the new fields. Validate
|
|
||||||
normalized unique keys, supported versions, status-specific required and
|
|
||||||
forbidden fields, output identity, checksum/size, and non-secret bounded
|
|
||||||
metadata. Maintain deterministic JSON output.
|
|
||||||
6. Readers must accept old manifests with no new fields. Writers must not
|
|
||||||
fabricate fingerprints from legacy aggregate outputs or emit parallel
|
|
||||||
legacy state as current evidence.
|
|
||||||
7. Preserve per-artifact analysis state when ordinary stage lifecycle helpers
|
|
||||||
clear aggregate result details during running, failure, or skip. Other stage
|
|
||||||
records keep their existing behavior.
|
|
||||||
8. Update `docs/internal/manifest.md` with the new authority, legacy meaning,
|
|
||||||
status model, and session-versus-invocation distinction.
|
|
||||||
|
|
||||||
### Tests And Exit Criteria
|
|
||||||
|
|
||||||
- Manifest round-trip tests cover every status and deterministic map output.
|
|
||||||
- Validation tests cover malformed keys, unsupported versions, impossible
|
|
||||||
status/field combinations, incomplete current output, and bad checksum/size.
|
|
||||||
- Legacy fixtures remain readable and are explicitly identified as lacking
|
|
||||||
current artifact evidence.
|
|
||||||
- Lifecycle tests prove unrelated current records survive transitions while
|
|
||||||
aggregate output/log/config/metadata fields retain their prior clearing
|
|
||||||
semantics.
|
|
||||||
|
|
||||||
## Stage 7 — Runner Projection And Partial-Error State Boundary
|
|
||||||
|
|
||||||
**Status: Completed**
|
|
||||||
|
|
||||||
### Goal
|
|
||||||
|
|
||||||
Give analyze one safe way to promote reconciled session state and invocation
|
|
||||||
history through the existing application-owned runner transaction.
|
|
||||||
|
|
||||||
### Required Work
|
|
||||||
|
|
||||||
1. Extend the stage result contract with an optional analyze-specific state
|
|
||||||
projection containing:
|
|
||||||
- the complete reconciled session analysis state; and
|
|
||||||
- the invocation subset attempted or evaluated in the current run.
|
|
||||||
Keep ordinary stage results unchanged.
|
|
||||||
2. On successful analyze completion, have the runner validate and apply the
|
|
||||||
session projection to the session manifest and the invocation projection to
|
|
||||||
the run manifest. Rebuild aggregate session `Outputs` deterministically from
|
|
||||||
current per-artifact records only. Keep invocation `Outputs` limited to
|
|
||||||
artifacts actually produced by that invocation.
|
|
||||||
3. Add a restricted result-plus-error path for analyze. If a stage returns an
|
|
||||||
error with an analyze-state projection, validate and persist only that state
|
|
||||||
before marking the aggregate stage failed. Ignore/reject ordinary success
|
|
||||||
outputs, success disposition, or unrelated stage projections alongside an
|
|
||||||
error.
|
|
||||||
4. Apply state only after the stage has validated run-local outputs and safely
|
|
||||||
materialized the records it marks current. The runner must never derive
|
|
||||||
records by scanning output directories.
|
|
||||||
5. On projection validation or persistence failure, fail conservatively and do
|
|
||||||
not advertise newly attempted artifacts as current. Preserve the last
|
|
||||||
durable unrelated current records.
|
|
||||||
6. Add `Force` to the stage environment as application-owned invocation
|
|
||||||
context, set it from the selected stage decision, and leave unselected
|
|
||||||
stages unaffected. This will distinguish explicit targets from ordinary
|
|
||||||
stale prerequisite rebuilding in later stages.
|
|
||||||
|
|
||||||
### Tests And Exit Criteria
|
|
||||||
|
|
||||||
- Runner tests prove separate session and invocation projections on success.
|
|
||||||
- Error-path tests prove completed artifacts can be durably represented, the
|
|
||||||
failed target is not current, unrelated prior current records survive, and
|
|
||||||
aggregate analyze/publish state remains conservative.
|
|
||||||
- Tests reject state projection from non-analyze stages and malformed or
|
|
||||||
contradictory result-plus-error payloads.
|
|
||||||
- Existing stages and ordinary error behavior remain unchanged.
|
|
||||||
|
|
||||||
## Stage 8 — Manifest-Authoritative Configured Artifact Evidence
|
|
||||||
|
|
||||||
**Status: Completed**
|
|
||||||
|
|
||||||
### Goal
|
|
||||||
|
|
||||||
Make configured analysis outputs available to analyze and publish only through
|
|
||||||
validated current manifest evidence.
|
|
||||||
|
|
||||||
### Required Work
|
|
||||||
|
|
||||||
1. Add an analysis-evidence inspector and catalog hydrator analogous to the
|
|
||||||
extraction-evidence owner, but specific to configured Scriptorium artifacts.
|
|
||||||
2. For a `current` record, verify the supported state/fingerprint version,
|
|
||||||
configured key and source ID, contract, configured canonical path, confined
|
|
||||||
no-follow regular file, stored size, and stored checksum. Return a typed
|
|
||||||
current/non-current result with an actionable reason.
|
|
||||||
3. Treat stale, missing, failed, unselected, legacy, removed, malformed,
|
|
||||||
unsafe, missing, or checksum-mismatched evidence as unavailable. Do not
|
|
||||||
silently rewrite status during read-only catalog hydration.
|
|
||||||
4. Remove current-session configured-artifact fallbacks that call `stat` and
|
|
||||||
mark canonical files available merely because they exist. Hydrate analyze,
|
|
||||||
publish, operator display, and artifact resolution from the evidence owner.
|
|
||||||
5. Retain existing behavior for prior-session inputs or other source kinds only
|
|
||||||
where their canonical owner already has an explicit compatibility policy;
|
|
||||||
do not broaden filesystem fallback.
|
|
||||||
6. When current configuration removes or renames a key, omit its old record
|
|
||||||
from the current catalog even if the manifest retains history until the next
|
|
||||||
analyze reconciliation.
|
|
||||||
7. Update `docs/internal/artifacts.md` and `docs/internal/stage-publish.md` for
|
|
||||||
manifest-authoritative configured results.
|
|
||||||
|
|
||||||
### Tests And Exit Criteria
|
|
||||||
|
|
||||||
- Evidence tests cover valid current output, each non-current status, legacy
|
|
||||||
absence, config/path/source mismatch, symlink/non-regular files, missing
|
|
||||||
files, size mismatch, and checksum mismatch.
|
|
||||||
- Analyze and publish catalog tests prove an incidental file is unavailable and
|
|
||||||
a validated current record is available.
|
|
||||||
- Publish tests prove stale configured artifacts cannot be selected or uploaded
|
|
||||||
while unrelated current artifacts remain publishable.
|
|
||||||
|
|
||||||
## Stage 9 — Deterministic Analyze Input Identity
|
|
||||||
|
|
||||||
**Status: Completed**
|
|
||||||
|
|
||||||
### Goal
|
|
||||||
|
|
||||||
Resolve every Narratio-visible analyze input into a stable semantic identity
|
|
||||||
that the fingerprint engine can consume without depending on workspace paths or
|
|
||||||
producer runs.
|
|
||||||
|
|
||||||
### Required Work
|
|
||||||
|
|
||||||
1. Define an ordered input-identity record containing the configured input
|
|
||||||
name, source ID, required/optional policy, presence/absence, contract,
|
|
||||||
content checksum, and size. Include a stable session-relative or source-based
|
|
||||||
logical identity where needed; never include an absolute path.
|
|
||||||
2. Resolve transcript, prepared-input, extraction-lane, previous-session, and
|
|
||||||
configured-artifact sources through their existing artifact resolvers and
|
|
||||||
catalogs. Configured-artifact dependencies must pass Stage 8 evidence.
|
|
||||||
3. Hash validated regular files using the repository's streaming file-safety
|
|
||||||
primitives and central size limits. Reuse an already validated manifest
|
|
||||||
checksum when its owning evidence contract proves it represents the same
|
|
||||||
bytes; do not read whole artifacts into memory.
|
|
||||||
4. Represent an absent optional input explicitly so appearance/disappearance
|
|
||||||
changes identity. A missing required input remains an error.
|
|
||||||
5. Normalize input ordering independently of Go map iteration while preserving
|
|
||||||
any configured order whose semantics are observable to Scriptorium.
|
|
||||||
6. Keep identity resolution read-only. It may inspect files and manifests but
|
|
||||||
cannot materialize output, update status, invoke adapters, or create run
|
|
||||||
records.
|
|
||||||
|
|
||||||
### Tests And Exit Criteria
|
|
||||||
|
|
||||||
- Table-driven tests cover every supported source kind, optional absence,
|
|
||||||
missing required input, unsafe path/type, checksum reuse, and changed bytes.
|
|
||||||
- Identical bytes at relocated workspace roots produce identical semantic
|
|
||||||
identities; changed bytes or contracts produce different identities.
|
|
||||||
- Configured dependencies cannot resolve from incidental or stale files.
|
|
||||||
- Ordering remains deterministic across randomized map insertion.
|
|
||||||
|
|
||||||
## Stage 10 — Versioned Analysis Fingerprints And Reconciliation
|
|
||||||
|
|
||||||
**Status: Completed**
|
|
||||||
|
|
||||||
### Goal
|
|
||||||
|
|
||||||
Classify configured artifacts as current or requiring work from one deterministic
|
|
||||||
fingerprint contract shared by resume validation, planning, and execution.
|
|
||||||
|
|
||||||
### Required Work
|
|
||||||
|
|
||||||
1. Define a centrally named fingerprint contract version and a canonical
|
|
||||||
serialization used only as hash input. Do not use ad hoc string concatenation
|
|
||||||
or Go map serialization.
|
|
||||||
2. Include all Narratio-observable result-affecting fields:
|
|
||||||
- normalized artifact key and normalized Scriptorium artifact configuration;
|
|
||||||
- deterministic dependency keys;
|
|
||||||
- effective prompt/profile identifiers, render-debug behavior, normalized
|
|
||||||
output identity, ordered input declarations, and sorted effective vars;
|
|
||||||
- Stage 9 resolved input identities and current configured-dependency output
|
|
||||||
identities; and
|
|
||||||
- result-affecting global Scriptorium configuration that Narratio directly
|
|
||||||
passes or interprets.
|
|
||||||
3. Exclude timeout/retry settings, absolute binary/config/output/workspace
|
|
||||||
paths, run IDs, timestamps, log/config output locations, and executable or
|
|
||||||
arbitrary transitive file contents. Document that explicit force is required
|
|
||||||
when an unobserved Scriptorium-private input changes.
|
|
||||||
4. Compute dependency artifact fingerprints in deterministic topological order
|
|
||||||
and reject unknown dependencies and cycles through the existing configuration
|
|
||||||
validation owner.
|
|
||||||
5. Build a read-only reconciliation function that compares current config,
|
|
||||||
stored record/version/fingerprint, Stage 8 output evidence, and newly resolved
|
|
||||||
identities. Return a typed reason for `current`, `stale`, `missing`, `failed`,
|
|
||||||
`legacy`, `removed`, or otherwise non-resumable state.
|
|
||||||
6. Fingerprint byte-identical input content identically even when it came from
|
|
||||||
a forced upstream run with a different producer run ID.
|
|
||||||
|
|
||||||
### Tests And Exit Criteria
|
|
||||||
|
|
||||||
- Golden or table-driven tests prove deterministic fingerprints across map
|
|
||||||
order, workspace relocation, and producer run IDs.
|
|
||||||
- Sensitivity tests change each included semantic field independently and
|
|
||||||
assert a new fingerprint. Exclusion tests cover timeout, timestamp, run ID,
|
|
||||||
and absolute-root-only changes.
|
|
||||||
- Reconciliation tests cover current, tampered output, changed dependency,
|
|
||||||
optional input transition, legacy record, removed config, version mismatch,
|
|
||||||
and byte-identical upstream replacement.
|
|
||||||
- `docs/internal/stage-analyze.md` records the implemented fingerprint boundary
|
|
||||||
and its explicit limitations.
|
|
||||||
|
|
||||||
## Stage 11 — Incremental Analysis Work Planning
|
|
||||||
|
|
||||||
**Status: Completed**
|
|
||||||
|
|
||||||
### Goal
|
|
||||||
|
|
||||||
Turn reconciliation results, explicit selection, dependencies, and force into
|
|
||||||
a deterministic artifact execution plan without invoking Scriptorium.
|
|
||||||
|
|
||||||
### Required Work
|
|
||||||
|
|
||||||
1. Define explicit targets as the normalized `--artifacts` selection when
|
|
||||||
present, otherwise all enabled configured artifacts. Preserve existing
|
|
||||||
unknown/ambiguous-selection validation.
|
|
||||||
2. Compute the transitive configured prerequisite closure of explicit targets
|
|
||||||
and order the closure deterministically and topologically.
|
|
||||||
3. Reuse a prerequisite when Stage 10 classifies it current. Schedule a stale,
|
|
||||||
missing, failed, legacy, or invalid prerequisite before its dependent.
|
|
||||||
4. Force only explicit targets. A prerequisite is forced only when it is also
|
|
||||||
an explicit target; otherwise a current prerequisite is reused.
|
|
||||||
5. Preserve the existing selection rule: default selection includes enabled
|
|
||||||
artifacts, while an explicitly named disabled artifact is a valid target.
|
|
||||||
A disabled configured prerequisite may be reused when current or executed
|
|
||||||
when it is in a selected target's required closure. Never execute an
|
|
||||||
unrelated disabled artifact.
|
|
||||||
6. Retain valid unselected records. Classify artifacts removed or renamed from
|
|
||||||
current configuration as unavailable in the reconciled session projection.
|
|
||||||
7. Produce a typed plan containing explicit targets, prerequisite-only work,
|
|
||||||
reused current artifacts, invalidated/removed records, reasons, and the
|
|
||||||
deterministic execution order. Keep the function read-only and free of
|
|
||||||
adapter calls.
|
|
||||||
|
|
||||||
### Tests And Exit Criteria
|
|
||||||
|
|
||||||
- Planning tests cover full/default selection, partial selection, dependency
|
|
||||||
closure, nested dependencies, current prerequisite reuse, stale prerequisite
|
|
||||||
rebuilding, target-only force, disabled prerequisites, unknown keys, cycles,
|
|
||||||
removed configuration, and deterministic order.
|
|
||||||
- Tests prove valid unrelated records survive the projected result and legacy
|
|
||||||
unselected outputs do not become current.
|
|
||||||
- The planner exposes enough typed information for `session plan` and analyze
|
|
||||||
execution to share decisions rather than recomputing them differently.
|
|
||||||
|
|
||||||
## Stage 12 — Incremental Analyze Execution And Promotion
|
|
||||||
|
|
||||||
**Status: Completed**
|
|
||||||
|
|
||||||
### Goal
|
|
||||||
|
|
||||||
Execute the Stage 11 work plan on the successful path, safely promote validated
|
|
||||||
results, and preserve or invalidate records according to actual output identity.
|
|
||||||
|
|
||||||
### Required Work
|
|
||||||
|
|
||||||
1. Refactor analyze to execute only scheduled artifacts in deterministic order.
|
|
||||||
Reused current artifacts must be exposed to later scheduled dependents
|
|
||||||
through the runtime catalog without invoking Scriptorium.
|
|
||||||
2. For each scheduled artifact, keep output run-local until the adapter result
|
|
||||||
and output pass existing safety, contract, size, and checksum validation.
|
|
||||||
Materialize canonically only through the established file-operation owner.
|
|
||||||
3. Record the new fingerprint, full output evidence, producer run ID, bounded
|
|
||||||
non-secret provenance, logs, and generated configs in the Stage 7 projection.
|
|
||||||
4. After replacement, compare semantic output identity. If bytes/contract are
|
|
||||||
unchanged, allow an unselected dependent whose recomputed fingerprint is
|
|
||||||
equal to remain current. If identity changes, mark every unselected
|
|
||||||
configured dependent stale without executing it.
|
|
||||||
5. Preserve valid unrelated current records and outputs during partial runs.
|
|
||||||
Reconstruct aggregate session outputs from all current records; report only
|
|
||||||
newly produced outputs in the invocation manifest.
|
|
||||||
6. A partial invocation succeeds when every explicit target and required
|
|
||||||
prerequisite succeeds, even if unrelated configured records remain stale.
|
|
||||||
7. Never synthesize a current record from an existing canonical output.
|
|
||||||
|
|
||||||
### Tests And Exit Criteria
|
|
||||||
|
|
||||||
- Analyze tests cover current reuse, partial rerun preservation, nested stale
|
|
||||||
prerequisite rebuilding, selected force, dependency ordering, changed-output
|
|
||||||
dependent invalidation, and identical-output dependent preservation.
|
|
||||||
- Legacy tests prove full selection rebuilds the effective set, while partial
|
|
||||||
selection promotes only targets/prerequisites and leaves legacy unselected
|
|
||||||
outputs unavailable.
|
|
||||||
- Adapter fakes remain offline and assertions focus on requests, resulting
|
|
||||||
files, manifest records, and catalogs rather than private implementation maps.
|
|
||||||
|
|
||||||
## Stage 13 — Incremental Analyze Failure Safety
|
|
||||||
|
|
||||||
**Status: Completed**
|
|
||||||
|
|
||||||
### Goal
|
|
||||||
|
|
||||||
Complete the incremental executor with conservative, durable behavior for
|
|
||||||
partial adapter, validation, materialization, and persistence failures.
|
|
||||||
|
|
||||||
### Required Work
|
|
||||||
|
|
||||||
1. At each artifact boundary, retain enough reconciled state to return the
|
|
||||||
restricted Stage 7 projection if later work fails. Do not report a completion
|
|
||||||
until its run-local output has been validated and canonically materialized.
|
|
||||||
2. On failure, mark the active target `failed` with a bounded non-secret error.
|
|
||||||
Mark any artifact whose current identity depends on the unavailable result
|
|
||||||
stale, including unselected dependents, without executing them.
|
|
||||||
3. Preserve unrelated previous current records. Preserve earlier completions
|
|
||||||
from this invocation only when they crossed the defined materialization and
|
|
||||||
validation boundary; include them in invocation history.
|
|
||||||
4. Return the restricted state projection alongside the original error so the
|
|
||||||
runner can persist artifact state before marking aggregate analyze failed.
|
|
||||||
Dependency-aware invalidation must keep publish and notify conservative.
|
|
||||||
5. If persistence of the partial projection itself fails, surface that failure
|
|
||||||
with the original context, retain the last durable manifest, and treat any
|
|
||||||
newly materialized file as incidental rather than current evidence.
|
|
||||||
6. Never mark a failed or unverified artifact current merely because old bytes
|
|
||||||
remain at its canonical path. Never reconstruct partial success by scanning
|
|
||||||
output directories after an error.
|
|
||||||
|
|
||||||
### Tests And Exit Criteria
|
|
||||||
|
|
||||||
- Tests cover first, middle, and last artifact adapter failures; unsafe or
|
|
||||||
invalid adapter output; canonical materialization failure; and session/run
|
|
||||||
manifest persistence failure.
|
|
||||||
- Each case proves the failed target is unavailable, dependent state is stale,
|
|
||||||
unrelated current work survives, and only durably completed work appears in
|
|
||||||
invocation history.
|
|
||||||
- Tests prove old canonical bytes and incidental newly materialized bytes do
|
|
||||||
not override the durable manifest authority.
|
|
||||||
- Error wrapping remains actionable and non-secret, and the original adapter or
|
|
||||||
filesystem cause remains discoverable.
|
|
||||||
|
|
||||||
## Stage 14 — Analyze Resume Validation And Plan/Run Parity
|
|
||||||
|
|
||||||
**Status: Completed**
|
|
||||||
|
|
||||||
### Goal
|
|
||||||
|
|
||||||
Make aggregate analyze skipping and `session plan` reflect artifact-level
|
|
||||||
freshness using the same read-only decision engine as execution.
|
|
||||||
|
|
||||||
### Required Work
|
|
||||||
|
|
||||||
1. Implement an analyze-specific resume validator that runs Stages 8–11
|
|
||||||
reconciliation for the requested artifact set. It is resumable only when all
|
|
||||||
explicit targets and required prerequisites are current and no selected work
|
|
||||||
is scheduled.
|
|
||||||
2. A succeeded aggregate analyze record with stale unrelated artifacts may
|
|
||||||
still skip for a partial selection that does not require them. A later full
|
|
||||||
selection must not skip them.
|
|
||||||
3. If coarse stage invalidation marked analyze stale but recomputation proves
|
|
||||||
selected fingerprints and outputs unchanged, allow analyze execution to
|
|
||||||
perform zero Scriptorium calls and restore the correct successful aggregate
|
|
||||||
state through the ordinary runner boundary.
|
|
||||||
4. Extend `session plan` to invoke the same stage resume validators and analyze
|
|
||||||
work planner as `run`, using a cloned/in-memory manifest transition model to
|
|
||||||
account for earlier selected stages and their invalidation. It must not
|
|
||||||
persist a session manifest, create an invocation directory, materialize a
|
|
||||||
file, or call an external adapter.
|
|
||||||
5. Plan output must distinguish pipeline-stage run/skip decisions and, for
|
|
||||||
analyze, explicit targets, prerequisite-only rebuilds, and current reuse with
|
|
||||||
concise reasons. Do not promise output identities that require execution.
|
|
||||||
6. Preserve the rule that excluded stages are never resume-validated. The
|
|
||||||
Stage 4 prefix status check remains structural and read-only.
|
|
||||||
|
|
||||||
### Tests And Exit Criteria
|
|
||||||
|
|
||||||
- Resume tests cover full and partial selections, force, current/stale/legacy
|
|
||||||
mixtures, current prerequisites, changed inputs, output tampering, and a
|
|
||||||
stale aggregate record with semantically current selected artifacts.
|
|
||||||
- Paired plan/run tests feed the same fixture and assert identical selected
|
|
||||||
stage and analyze-artifact decisions before execution.
|
|
||||||
- Side-effect tests prove planning performs no manifest write, run-directory
|
|
||||||
creation, canonical materialization, or adapter invocation.
|
|
||||||
- Existing extract resume validation remains functional in bounded and
|
|
||||||
unbounded plans.
|
|
||||||
|
|
||||||
## Stage 15 — Assembled Workflow And Compatibility Coverage
|
|
||||||
|
|
||||||
**Status: Completed**
|
|
||||||
|
|
||||||
### Goal
|
|
||||||
|
|
||||||
Verify the complete feature through production composition and representative
|
|
||||||
historical state without adding another behavior path.
|
|
||||||
|
|
||||||
### Required Work
|
|
||||||
|
|
||||||
1. Add assembled application tests for:
|
|
||||||
- an ordinary full run in the new canonical order;
|
|
||||||
- `run --from extract --through analyze --force` with and without artifact
|
|
||||||
selection;
|
|
||||||
- the exact alias reaching the same bounded-run request;
|
|
||||||
- render-only and extract-only forced runs demonstrating sibling
|
|
||||||
invalidation independence;
|
|
||||||
- a later publish consuming only current configured artifacts; and
|
|
||||||
- stop-on-failure with dependent stages outside the range left stale but
|
|
||||||
unexecuted.
|
|
||||||
2. Add compatibility fixtures or focused tests for manifests created under the
|
|
||||||
old render/extract order and aggregate-only analyze state. Confirm no
|
|
||||||
transcript stage is invalidated solely by version or old relative order.
|
|
||||||
3. Exercise a complete legacy transition: partial regeneration, unavailable
|
|
||||||
unselected legacy output, later full regeneration, then successful publish
|
|
||||||
from current per-artifact records.
|
|
||||||
4. Verify production composition does not eagerly require excluded adapters
|
|
||||||
and that generated run manifests list only the bounded requested stages and
|
|
||||||
actual invocation artifacts.
|
|
||||||
5. Remove obsolete suffix-invalidation, aggregate-analysis-replacement, and
|
|
||||||
filesystem-discovery compatibility code that no current path requires. Keep
|
|
||||||
any unavoidable compatibility shim narrow, clearly commented with its
|
|
||||||
removal condition, and covered by a legacy test.
|
|
||||||
|
|
||||||
### Tests And Exit Criteria
|
|
||||||
|
|
||||||
- Assembled tests use controlled fakes and real application wiring; they remain
|
|
||||||
offline and deterministic.
|
|
||||||
- The full suite passes under the standard test command and repository race or
|
|
||||||
platform checks required by policy.
|
|
||||||
- Coverage is allocated to behavior owners; this stage adds only integration
|
|
||||||
assertions that smaller tests cannot prove.
|
|
||||||
|
|
||||||
## Stage 16 — Canonical Documentation And Quality Closure
|
|
||||||
|
|
||||||
**Status: Completed**
|
|
||||||
|
|
||||||
### Goal
|
|
||||||
|
|
||||||
Bring every current-behavior owner into agreement and perform the final
|
|
||||||
repository-wide policy and regression check.
|
|
||||||
|
|
||||||
### Required Work
|
|
||||||
|
|
||||||
1. Audit current documentation for the old extract/render order, suffix-only
|
|
||||||
invalidation, unbounded-only run behavior, aggregate analysis replacement,
|
|
||||||
and filesystem-discovered configured artifacts.
|
|
||||||
2. Finalize:
|
|
||||||
- `docs/cli.md` for range syntax, duplicate singleton rejection, artifact
|
|
||||||
selection, exact alias expansion, and examples;
|
|
||||||
- `docs/operations.md` for the post-transcript development loop, force scope,
|
|
||||||
plan preview, explicit publish, and recovery;
|
|
||||||
- `docs/troubleshooting.md` for bounded prerequisite failures, stale or
|
|
||||||
tampered analysis evidence, legacy regeneration, fingerprint limitations,
|
|
||||||
and explicit force for unobserved Scriptorium inputs;
|
|
||||||
- `docs/internal/overview.md`, `stage-render.md`, `stage-extract.md`,
|
|
||||||
`stage-analyze.md`, `stage-publish.md`, `artifacts.md`, `manifest.md`, and
|
|
||||||
`adapters.md` for their implemented contracts; and
|
|
||||||
- `docs/policy/architecture.md` for flat canonical execution plus separate
|
|
||||||
dependency-aware invalidation and contiguous bounded runs.
|
|
||||||
3. Keep volatile syntax and schema details in their canonical owners. Other
|
|
||||||
documents should link rather than duplicate large flag or field inventories.
|
|
||||||
4. Confirm all examples use the new order and that none describe the fixed
|
|
||||||
dependency relation as a configurable DAG or claim perfect observation of
|
|
||||||
Scriptorium-private inputs.
|
|
||||||
5. Run formatting, documentation checks, static analysis, `go test ./...`, and
|
|
||||||
any race/platform checks required by repository policy. Review the final diff
|
|
||||||
for accidental generated files, secrets, broad refactors, stale compatibility
|
|
||||||
branches, and policy violations.
|
|
||||||
6. Update this plan's status only after all prior acceptance criteria pass. Do
|
|
||||||
not delete the accepted feature roadmap until the project's normal roadmap
|
|
||||||
closeout process authorizes it.
|
|
||||||
|
|
||||||
### Tests And Exit Criteria
|
|
||||||
|
|
||||||
- All maintained documentation agrees with implemented behavior and passes the
|
|
||||||
documentation policy's discoverability and ownership requirements.
|
|
||||||
- All repository-required checks pass from a clean checkout without installed
|
|
||||||
external adapter binaries.
|
|
||||||
- The worktree contains only intentional feature changes, no unresolved TODOs
|
|
||||||
standing in for this plan, and no known divergence from the feature roadmap.
|
|
||||||
|
|
||||||
## Open Questions
|
|
||||||
|
|
||||||
None. The accepted feature roadmap and the settled decisions above are
|
|
||||||
sufficient to implement every stage without a further product or architecture
|
|
||||||
choice.
|
|
||||||
@@ -1,274 +0,0 @@
|
|||||||
# Notarius v0.6 CLI Reference Integration
|
|
||||||
|
|
||||||
## Status
|
|
||||||
|
|
||||||
Accepted target state. Delivery sequencing and implementation status are owned
|
|
||||||
by [implementation.md](implementation.md).
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
|
|
||||||
Upgrade Narratio's extraction boundary to the Notarius v0.6.0 subprocess
|
|
||||||
contract and supply session reference documents explicitly with repeatable
|
|
||||||
`--reference selector=path` arguments.
|
|
||||||
|
|
||||||
The maintained D&D integration must make the prepared party roster, player
|
|
||||||
context, glossary, and optional spell catalog available to every compatible
|
|
||||||
Notarius target. Notarius must continue to own pipeline topology, reference-slot
|
|
||||||
compatibility, generated artifact handoffs, prompts, and D&D schemas. Narratio
|
|
||||||
owns selection and preparation of its external reference files, exact CLI
|
|
||||||
invocation, provenance, and extraction reuse correctness.
|
|
||||||
|
|
||||||
## Current State And Gap
|
|
||||||
|
|
||||||
Narratio currently invokes Notarius as:
|
|
||||||
|
|
||||||
```text
|
|
||||||
notarius run <pipeline_id> --config <config_path> --input <transcript> --output-dir <staging_dir> --json
|
|
||||||
```
|
|
||||||
|
|
||||||
The `prepare` stage already materializes campaign/session party, players, and
|
|
||||||
glossary files under the session `inputs/` directory, but `extract` does not
|
|
||||||
pass them to Notarius. Narratio also has no stable spell-catalog input. As a
|
|
||||||
result, a Notarius deployment must duplicate these paths in its own
|
|
||||||
configuration, cannot reliably receive session overrides, and may extract
|
|
||||||
without the same campaign context supplied to Narratio's analysis stage.
|
|
||||||
|
|
||||||
Notarius v0.6.0 makes an unqualified CLI selector pipeline-scoped. For example,
|
|
||||||
`--reference party=/absolute/path/party.yml` supplies the file to every
|
|
||||||
selected target that declares `party`. Scoped selectors remain available for
|
|
||||||
exceptional overrides. CLI paths are resolved from the Notarius process working
|
|
||||||
directory, so subprocess callers are expected to provide absolute paths.
|
|
||||||
|
|
||||||
The v0.6.0 receipt, index, warning, diagnostic, and ten-lane D&D artifact
|
|
||||||
contracts remain compatible with Narratio's current v0.5 integration. This
|
|
||||||
feature changes the invocation and input-provenance contract rather than the
|
|
||||||
accepted output inventory.
|
|
||||||
|
|
||||||
## User Outcome
|
|
||||||
|
|
||||||
With the maintained complete D&D configuration, an operator can declare the
|
|
||||||
campaign reference sources once in Narratio. For each extraction Narratio will:
|
|
||||||
|
|
||||||
1. materialize the effective campaign/session files during `prepare`;
|
|
||||||
2. resolve those prepared files by stable Narratio source ID;
|
|
||||||
3. pass absolute paths for `party`, `players`, `glossary`, and, when configured,
|
|
||||||
`spell_catalog` to Notarius through repeatable CLI arguments;
|
|
||||||
4. fail before launching Notarius when a configured reference is unavailable;
|
|
||||||
5. rerun extraction when a selector, source binding, or reference file changes;
|
|
||||||
and
|
|
||||||
6. retain bounded reference identities and checksums for diagnosis and
|
|
||||||
provenance without copying reference contents into manifest metadata.
|
|
||||||
|
|
||||||
Session-level stable-input overrides must flow through the same mechanism. A
|
|
||||||
custom Notarius pipeline may bind different external slots without requiring a
|
|
||||||
Narratio code change.
|
|
||||||
|
|
||||||
## Chosen Architecture
|
|
||||||
|
|
||||||
### Explicit Reference Bindings
|
|
||||||
|
|
||||||
Extend `pipeline.notarius` with an explicit map from a Notarius CLI selector to
|
|
||||||
a prepared Narratio input source:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
notarius:
|
|
||||||
enabled: true
|
|
||||||
binary: notarius
|
|
||||||
config_path: /usr/local/etc/notarius/config.yml
|
|
||||||
pipeline_id: dnd-session
|
|
||||||
working_directory: /usr/local/etc/notarius
|
|
||||||
references:
|
|
||||||
party: narratio.input.party
|
|
||||||
players: narratio.input.players
|
|
||||||
glossary: narratio.input.glossary
|
|
||||||
spell_catalog: narratio.input.spell_catalog
|
|
||||||
outputs:
|
|
||||||
# Existing required lane contracts remain unchanged.
|
|
||||||
```
|
|
||||||
|
|
||||||
Each configured binding is required. An operator who does not maintain an
|
|
||||||
optional Notarius reference, such as a spell catalog, omits that binding. This
|
|
||||||
keeps missing-input behavior explicit and avoids a second required/optional
|
|
||||||
policy inside each entry.
|
|
||||||
|
|
||||||
The maintained complete D&D example will show all four external reference
|
|
||||||
slots. The three existing campaign context bindings use the canonical `party`,
|
|
||||||
`players`, and `glossary` spellings. Narratio will not emit the deprecated
|
|
||||||
`roster` alias.
|
|
||||||
|
|
||||||
The binding is deliberately source-based rather than path-based. Pipeline
|
|
||||||
configuration should not reconstruct session workspace paths or bypass
|
|
||||||
`prepare`; it names the stable input whose effective campaign/session value is
|
|
||||||
already owned by Narratio. The map also avoids hard-coded behavior keyed to the
|
|
||||||
literal `dnd-session` pipeline ID, preserving custom-pipeline support.
|
|
||||||
|
|
||||||
Narratio accepts the selector forms published by Notarius v0.6.0:
|
|
||||||
|
|
||||||
- `slot`;
|
|
||||||
- `chunk.slot`;
|
|
||||||
- `lane.slot`; and
|
|
||||||
- `lane.extract.slot`, `lane.merge.slot`, or `lane.normalize.slot`.
|
|
||||||
|
|
||||||
Configuration validation will reject empty or structurally invalid selectors,
|
|
||||||
selectors containing `=`, unsupported source IDs, and duplicate YAML keys.
|
|
||||||
Notarius remains authoritative for whether a selected target actually declares
|
|
||||||
the slot and whether a file satisfies that slot's media type and size contract.
|
|
||||||
Narratio will not duplicate the Notarius module registry.
|
|
||||||
|
|
||||||
### Stable Reference Inputs
|
|
||||||
|
|
||||||
Continue to use the existing prepared sources and canonical files:
|
|
||||||
|
|
||||||
| Narratio source | Prepared file | Notarius slot |
|
|
||||||
| --- | --- | --- |
|
|
||||||
| `narratio.input.party` | `inputs/party.yml` | `party` |
|
|
||||||
| `narratio.input.players` | `inputs/players.yml` | `players` |
|
|
||||||
| `narratio.input.glossary` | `inputs/glossary.yml` | `glossary` |
|
|
||||||
| `narratio.input.spell_catalog` | `inputs/spell_catalog.json` | `spell_catalog` |
|
|
||||||
|
|
||||||
Add optional `spell_catalog_file` fields to campaign and session inputs, with
|
|
||||||
the existing campaign-default/session-override resolution behavior. When
|
|
||||||
provided, `prepare` copies it into the session input area and records its
|
|
||||||
origin and checksum consistently with the other stable inputs. The prepared
|
|
||||||
filename remains JSON so Notarius can apply its published spell-catalog media
|
|
||||||
contract.
|
|
||||||
|
|
||||||
The new source must be added everywhere stable inputs are enumerated: strict
|
|
||||||
configuration decoding and merging, validation, prepare materialization,
|
|
||||||
artifact policy/source descriptions, operator inspection, manifest input
|
|
||||||
records, examples, and canonical documentation. It remains optional at the
|
|
||||||
campaign level; a configured Notarius binding makes it mandatory for that
|
|
||||||
extraction.
|
|
||||||
|
|
||||||
Extract and analyze should use one shared prepared-input source resolver rather
|
|
||||||
than maintain separate source-to-filename tables. The resolver must return an
|
|
||||||
absolute, regular, non-empty file beneath the current session workspace and
|
|
||||||
produce actionable `prepare --force` guidance when a configured source is
|
|
||||||
missing. It must not fall back to the original campaign path after preparation.
|
|
||||||
|
|
||||||
### Adapter Request And CLI Construction
|
|
||||||
|
|
||||||
Extend the transport-neutral Notarius run request with an ordered collection of
|
|
||||||
resolved reference bindings. Each binding contains only its selector and
|
|
||||||
absolute prepared-file path. The extraction stage resolves source IDs and file
|
|
||||||
identity; the subprocess adapter validates and serializes the request.
|
|
||||||
|
|
||||||
The production command becomes:
|
|
||||||
|
|
||||||
```text
|
|
||||||
notarius run <pipeline_id>
|
|
||||||
--config <config_path>
|
|
||||||
--input <trimmed_json>
|
|
||||||
--output-dir <staging_dir>
|
|
||||||
--reference party=<absolute_prepared_party_path>
|
|
||||||
--reference players=<absolute_prepared_players_path>
|
|
||||||
--reference glossary=<absolute_prepared_glossary_path>
|
|
||||||
--reference spell_catalog=<absolute_prepared_spell_catalog_path>
|
|
||||||
--json
|
|
||||||
```
|
|
||||||
|
|
||||||
Only configured bindings are emitted. Selectors are sorted before request
|
|
||||||
construction so argument order, tests, logs, and fingerprints are deterministic.
|
|
||||||
Arguments are passed directly to the subprocess without shell interpretation;
|
|
||||||
paths containing spaces or platform-specific separators remain one argument.
|
|
||||||
|
|
||||||
CLI bindings intentionally override matching external paths in the deployed
|
|
||||||
Notarius configuration. Narratio must not pass `--without-reference` and must
|
|
||||||
not synthesize CLI bindings for `location_registry`, `item_registry`,
|
|
||||||
`npc_registry`, `scene_descriptions`, `combat_turns`, or `npc_occurrences`.
|
|
||||||
Those are generated same-run artifact handoffs in the complete D&D pipeline and
|
|
||||||
remain entirely under Notarius configuration and execution control. A custom
|
|
||||||
configuration that collides an external CLI binding with a generated handoff is
|
|
||||||
expected to fail with Notarius's normal resolution error.
|
|
||||||
|
|
||||||
### Fingerprints, Resume, And Provenance
|
|
||||||
|
|
||||||
Reference identity is part of the extraction input contract. The extraction
|
|
||||||
fingerprint and resume validator must include, in deterministic selector order:
|
|
||||||
|
|
||||||
- the selector;
|
|
||||||
- the configured Narratio source ID;
|
|
||||||
- the resolved prepared path identity; and
|
|
||||||
- the prepared file's content checksum and size.
|
|
||||||
|
|
||||||
This is required even though Notarius generates a prompt session ID from the
|
|
||||||
input module and transcript bytes: Notarius intentionally does not include
|
|
||||||
references in that identifier. Narratio must therefore prevent an old
|
|
||||||
extraction from being reused after a roster, player list, glossary, spell
|
|
||||||
catalog, selector, or source mapping changes.
|
|
||||||
|
|
||||||
A changed reference makes the prior `extract` result non-reusable and follows
|
|
||||||
Narratio's normal downstream invalidation rules. A failed reference-resolution
|
|
||||||
or checksum check also prevents reuse; it must not silently accept the prior
|
|
||||||
bundle.
|
|
||||||
|
|
||||||
Successful extract metadata should record a bounded, deterministic list of
|
|
||||||
selector, source ID, workspace-relative path, checksum, and size. It must not
|
|
||||||
record reference contents, original absolute operator paths, or values from the
|
|
||||||
files. Existing receipt and bundle provenance behavior remains unchanged.
|
|
||||||
|
|
||||||
### Error And Compatibility Behavior
|
|
||||||
|
|
||||||
Narratio's documented minimum supported Notarius version becomes v0.6.0 for an
|
|
||||||
enabled reference binding. Compatibility remains contract-based rather than
|
|
||||||
dependent on parsing `notarius --version`: an older or incompatible executable
|
|
||||||
will fail at the CLI boundary with captured diagnostics.
|
|
||||||
|
|
||||||
Errors must identify the responsible selector and Narratio source without
|
|
||||||
including file contents. Configuration errors are reported before pipeline
|
|
||||||
execution. Missing, empty, non-regular, unsafe, or unreadable prepared files
|
|
||||||
fail extraction before the Notarius subprocess starts. Notarius continues to
|
|
||||||
report undeclared slots, media incompatibility, size limits, required-slot
|
|
||||||
failures, and generated-handoff collisions.
|
|
||||||
|
|
||||||
When Notarius is disabled, extraction retains its current explicit skip
|
|
||||||
behavior and does not resolve reference inputs. Receipt v2 ingestion, bundle
|
|
||||||
confinement, ten-lane selection, and downstream artifact source IDs are not
|
|
||||||
otherwise changed by this feature.
|
|
||||||
|
|
||||||
## Target End State
|
|
||||||
|
|
||||||
Narratio and Notarius have a clear orchestration boundary:
|
|
||||||
|
|
||||||
- `prepare` owns the effective, immutable session copies of external campaign
|
|
||||||
context;
|
|
||||||
- `extract` maps configured stable source IDs to Notarius v0.6 CLI selectors,
|
|
||||||
supplies absolute file paths, and owns reuse/provenance policy;
|
|
||||||
- the Notarius adapter owns exact subprocess serialization and supported result
|
|
||||||
decoding;
|
|
||||||
- Notarius owns slot compatibility, reference precedence within its pipeline,
|
|
||||||
generated artifact handoffs, and output schemas; and
|
|
||||||
- `analyze` consumes the resulting ten structured lane artifacts exactly as it
|
|
||||||
does today.
|
|
||||||
|
|
||||||
The maintained complete D&D workflow passes party, players, glossary, and spell
|
|
||||||
catalog context from the same prepared session inputs used elsewhere in
|
|
||||||
Narratio. Updating any of those documents deterministically causes fresh
|
|
||||||
extraction, and operators can diagnose the effective bindings without exposing
|
|
||||||
file contents.
|
|
||||||
|
|
||||||
## Out Of Scope
|
|
||||||
|
|
||||||
- Reproducing Notarius pipeline, lane, binding, or media-type validation in
|
|
||||||
Narratio.
|
|
||||||
- Passing or overriding Notarius generated artifact handoffs.
|
|
||||||
- Adding `--without-reference`, Notarius resume/recompute controls, lane
|
|
||||||
selection, model selection, profile selection, or session-ID overrides.
|
|
||||||
- Changing the ten accepted D&D lane contracts or the Scriptorium analysis
|
|
||||||
design.
|
|
||||||
- Reading reference payloads into Narratio manifests or logs.
|
|
||||||
- Automatically running `notarius config validate` for every session.
|
|
||||||
|
|
||||||
## Settled Policy Choices
|
|
||||||
|
|
||||||
The implementation must preserve these choices unless implementation evidence
|
|
||||||
shows a contract conflict:
|
|
||||||
|
|
||||||
- explicit selector-to-source mappings are preferred over pipeline-ID-specific
|
|
||||||
defaults;
|
|
||||||
- every configured mapping is required;
|
|
||||||
- `spell_catalog_file` is optional until a mapping requests its prepared
|
|
||||||
source;
|
|
||||||
- the complete D&D example demonstrates all four external references; and
|
|
||||||
- Notarius v0.6.0 is the minimum supported CLI contract for reference-enabled
|
|
||||||
extraction.
|
|
||||||
@@ -1,409 +0,0 @@
|
|||||||
# Post-Transcript Artifact Development Workflow
|
|
||||||
|
|
||||||
## Status
|
|
||||||
|
|
||||||
Implemented. The completed delivery sequence is retained in
|
|
||||||
[`implementation.md`](implementation.md).
|
|
||||||
|
|
||||||
## Goal
|
|
||||||
|
|
||||||
Make repeated development of extraction and analysis artifacts fast, explicit,
|
|
||||||
and safe after a session's transcripts are complete. Narratio should expose a
|
|
||||||
clear transcript/post-transcript boundary, allow operators to run a bounded
|
|
||||||
part of its canonical pipeline, provide one transparent convenience alias for
|
|
||||||
the common forced-regeneration workflow, and reuse analysis artifacts whose
|
|
||||||
observable inputs have not changed.
|
|
||||||
|
|
||||||
The feature must preserve Narratio's intentionally simple orchestration model:
|
|
||||||
one fixed stage sequence, explicit stages, manifest-authoritative state, and no
|
|
||||||
configurable workflow graph.
|
|
||||||
|
|
||||||
## User Intent
|
|
||||||
|
|
||||||
Transcript production is comparatively infrequent after a session reaches a
|
|
||||||
good final transcript. Development of Notarius extraction and Scriptorium
|
|
||||||
artifacts continues much longer and commonly requires repeated execution.
|
|
||||||
|
|
||||||
The normal development workflow should therefore:
|
|
||||||
|
|
||||||
- treat completed transcript work as read-only unless the operator explicitly
|
|
||||||
selects transcript stages;
|
|
||||||
- regenerate extraction and analysis without attempting earlier stages;
|
|
||||||
- make forced scope visible in the command itself;
|
|
||||||
- support focused Scriptorium artifact selection; and
|
|
||||||
- avoid rerunning unrelated analysis artifacts when their meaningful inputs
|
|
||||||
and dependencies are unchanged.
|
|
||||||
|
|
||||||
Persistent transcript seals and run-to-run comparison tools are intentionally
|
|
||||||
deferred. Bounded execution provides the immediate mutation boundary without
|
|
||||||
adding another kind of durable lock.
|
|
||||||
|
|
||||||
## Canonical Pipeline And Phase Boundary
|
|
||||||
|
|
||||||
Move `render` before `extract` so all transcript representations are complete
|
|
||||||
before post-transcript artifact generation begins. The canonical order becomes:
|
|
||||||
|
|
||||||
1. `prepare`
|
|
||||||
2. `transcribe`
|
|
||||||
3. `merge`
|
|
||||||
4. `polish`
|
|
||||||
5. `normalize`
|
|
||||||
6. `trim`
|
|
||||||
7. `render`
|
|
||||||
8. `extract`
|
|
||||||
9. `analyze`
|
|
||||||
10. `publish`
|
|
||||||
11. `notify`
|
|
||||||
|
|
||||||
This creates two useful conceptual regions without making phases configurable:
|
|
||||||
|
|
||||||
- transcript production: `prepare` through `render`; and
|
|
||||||
- post-transcript processing and delivery: `extract` through `notify`.
|
|
||||||
|
|
||||||
`render` and `extract` are independent sibling consumers of completed
|
|
||||||
transcript data. Render uses the canonical final and final-trimmed transcripts
|
|
||||||
to create Markdown representations. Extract uses the canonical final-trimmed
|
|
||||||
transcript and prepared references to create Notarius artifacts. Neither stage
|
|
||||||
consumes the other's output.
|
|
||||||
|
|
||||||
Narratio's render path is deterministic. A render failure will stop a full run
|
|
||||||
before extraction under the new order, and that behavior is accepted: a full
|
|
||||||
run should complete transcript representations before starting post-transcript
|
|
||||||
work. Recovery remains available through continuation and bounded or
|
|
||||||
single-stage execution.
|
|
||||||
|
|
||||||
## Execution Order And Invalidation
|
|
||||||
|
|
||||||
Execution order and invalidation dependencies must have separate explicit
|
|
||||||
owners. The canonical stage registry defines when stages are attempted. A
|
|
||||||
central, fixed, configuration-independent invalidation relation defines which
|
|
||||||
recorded results may no longer be trustworthy after a stage outcome changes.
|
|
||||||
|
|
||||||
The relation is conservative across supported configurations. For example,
|
|
||||||
`analyze` remains dependent on `render` because a configured artifact may
|
|
||||||
consume rendered Markdown, even if one particular pipeline does not.
|
|
||||||
|
|
||||||
The required transitive invalidation sets, returned in canonical execution
|
|
||||||
order, are:
|
|
||||||
|
|
||||||
| Changed stage | Succeeded stages eligible to become stale |
|
|
||||||
| --- | --- |
|
|
||||||
| `prepare` | `transcribe`, `merge`, `polish`, `normalize`, `trim`, `render`, `extract`, `analyze`, `publish`, `notify` |
|
|
||||||
| `transcribe` | `merge`, `polish`, `normalize`, `trim`, `render`, `extract`, `analyze`, `publish`, `notify` |
|
|
||||||
| `merge` | `polish`, `normalize`, `trim`, `render`, `extract`, `analyze`, `publish`, `notify` |
|
|
||||||
| `polish` | `normalize`, `trim`, `render`, `extract`, `analyze`, `publish`, `notify` |
|
|
||||||
| `normalize` | `trim`, `render`, `extract`, `analyze`, `publish`, `notify` |
|
|
||||||
| `trim` | `render`, `extract`, `analyze`, `publish`, `notify` |
|
|
||||||
| `render` | `analyze`, `publish`, `notify` |
|
|
||||||
| `extract` | `analyze`, `publish`, `notify` |
|
|
||||||
| `analyze` | `publish`, `notify` |
|
|
||||||
| `publish` | `notify` |
|
|
||||||
| `notify` | none |
|
|
||||||
|
|
||||||
In particular, render and extract must not invalidate one another. A change to
|
|
||||||
either still invalidates analysis and delivery, while a change to trim
|
|
||||||
invalidates both branches and their consumers.
|
|
||||||
|
|
||||||
The relation applies to every existing invalidation trigger, including forced
|
|
||||||
replacement, a non-resumable success, failure, and a changed effective outcome.
|
|
||||||
Only succeeded dependent stage records become stale under the existing status
|
|
||||||
rules. Failed and incomplete records retain their meaning, and an identical
|
|
||||||
repeated self-skip does not cause perpetual reruns.
|
|
||||||
|
|
||||||
The application owner must validate the fixed relation against the canonical
|
|
||||||
stage inventory so a stage addition, removal, rename, duplication, or missing
|
|
||||||
classification cannot silently produce incorrect invalidation behavior. This
|
|
||||||
relation is not configurable and is not an alternate execution planner.
|
|
||||||
|
|
||||||
## Bounded Canonical Execution
|
|
||||||
|
|
||||||
Extend `run` with inclusive canonical bounds:
|
|
||||||
|
|
||||||
```text
|
|
||||||
narratio run <session_id> [--from <stage>] [--through <stage>] [--force]
|
|
||||||
```
|
|
||||||
|
|
||||||
Examples:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
narratio run SESSION --from extract --through analyze --force
|
|
||||||
narratio run SESSION --from render --through render --force
|
|
||||||
narratio run SESSION --from analyze --through analyze
|
|
||||||
```
|
|
||||||
|
|
||||||
The bounds have these settled semantics:
|
|
||||||
|
|
||||||
- they select one contiguous slice of the fixed canonical stage sequence;
|
|
||||||
- `--from` defaults to the first stage and `--through` defaults to the last
|
|
||||||
stage when omitted;
|
|
||||||
- both stage names must exist, and `--from` must not occur after `--through`;
|
|
||||||
- with neither option, `run` retains its current full-pipeline behavior;
|
|
||||||
- `--force` applies only to stages inside the selected range;
|
|
||||||
- stages before and after the range are not executed or resume-validated;
|
|
||||||
- excluded upstream records and artifacts may be resolved and validated as
|
|
||||||
stage inputs, but Narratio must not repair or regenerate them implicitly;
|
|
||||||
- missing, stale, unsafe, or otherwise unusable prerequisites produce an
|
|
||||||
actionable error rather than widening the requested range;
|
|
||||||
- invalidation caused by an executed stage may mark dependent stages outside
|
|
||||||
the range stale, but those stages are not executed; and
|
|
||||||
- stage failure retains the existing stop-on-failure behavior.
|
|
||||||
|
|
||||||
`session plan` must accept the same bounds, force scope, and artifact selection
|
|
||||||
needed to preview the corresponding `run` without executing stages. Plan and
|
|
||||||
run must use one selection implementation so their range validation and
|
|
||||||
run/skip decisions cannot drift.
|
|
||||||
|
|
||||||
`--artifacts` retains its existing meaning for `analyze` and `publish` when
|
|
||||||
those stages are inside the selected range. Supplying artifact selection for a
|
|
||||||
range containing neither consumer is an error rather than a silent no-op.
|
|
||||||
Repeated artifact-selection flags retain their existing normalization and
|
|
||||||
deduplication behavior.
|
|
||||||
|
|
||||||
Production composition should follow the bounded plan. Selecting a range must
|
|
||||||
not require an adapter used only by an excluded stage, while a selected stage
|
|
||||||
continues to require and validate its own collaborators.
|
|
||||||
|
|
||||||
## `regenerate-artifacts` Convenience Alias
|
|
||||||
|
|
||||||
Add this top-level command:
|
|
||||||
|
|
||||||
```text
|
|
||||||
narratio regenerate-artifacts <session_id> [--artifacts <name[,name...]>]
|
|
||||||
```
|
|
||||||
|
|
||||||
It is exactly a convenience alias for:
|
|
||||||
|
|
||||||
```text
|
|
||||||
narratio run <session_id> --force --from extract --through analyze [--artifacts <name[,name...]>]
|
|
||||||
```
|
|
||||||
|
|
||||||
The alias has no independent orchestration semantics, prerequisites, force
|
|
||||||
rules, or execution path. Its implementation must rewrite or construct the
|
|
||||||
equivalent `run` arguments before invoking the shared run parser and handler.
|
|
||||||
All common session/configuration arguments and repeatable `--artifacts` values
|
|
||||||
pass through to `run` unchanged.
|
|
||||||
|
|
||||||
The shared parser owns validation, planning, execution, errors, and summaries.
|
|
||||||
Alias help must state the exact equivalence. It is acceptable and desirable for
|
|
||||||
runtime errors and summaries to identify the canonical `run` operation. The
|
|
||||||
alias must not gain private flags or behavior; a future capability belongs on
|
|
||||||
`run` first.
|
|
||||||
|
|
||||||
Because `--force`, `--from`, and `--through` define the alias, callers must not
|
|
||||||
override them. The shared command parsing layer should reject duplicate
|
|
||||||
singleton options rather than use ordering to choose a winner. That rule should
|
|
||||||
apply consistently to bounded `run` itself, not only to the alias.
|
|
||||||
|
|
||||||
Without `--artifacts`, the alias force-runs extraction and all enabled
|
|
||||||
configured analysis artifacts, matching the existing default analysis
|
|
||||||
selection. With `--artifacts`, extraction still produces its complete
|
|
||||||
configured Notarius bundle, while forced analysis targets only the selected
|
|
||||||
Scriptorium artifacts and any prerequisites required to build them. An
|
|
||||||
explicitly selected disabled artifact remains a valid target under the existing
|
|
||||||
selection rules. The alias never runs publish or notify. Changed results may
|
|
||||||
correctly leave those later stages stale.
|
|
||||||
|
|
||||||
The existing `analyze` command remains the convenience path for forcing
|
|
||||||
analysis without rerunning Notarius.
|
|
||||||
|
|
||||||
## Incremental Analysis Artifacts
|
|
||||||
|
|
||||||
### Analyze-Owned State
|
|
||||||
|
|
||||||
`analyze` currently has one aggregate stage result. Extend its manifest-owned
|
|
||||||
state so each configured Scriptorium artifact has an explicit current result
|
|
||||||
identity. Keep this model specific to analysis artifacts; do not introduce
|
|
||||||
dynamic pipeline stages or a generic subtask framework without another proven
|
|
||||||
consumer.
|
|
||||||
|
|
||||||
Each current artifact result must identify at least:
|
|
||||||
|
|
||||||
- the normalized configured artifact key;
|
|
||||||
- a versioned input fingerprint;
|
|
||||||
- the output source ID, contract, confined canonical path, checksum, and size;
|
|
||||||
- the producing Narratio run identity and useful non-secret Scriptorium
|
|
||||||
provenance; and
|
|
||||||
- enough status or disposition information to distinguish current, stale,
|
|
||||||
missing, failed, and intentionally unselected work.
|
|
||||||
|
|
||||||
The session manifest remains the authority for current availability. An
|
|
||||||
incidental output file is not current merely because it exists. Invocation
|
|
||||||
manifests continue to record what one run attempted and produced.
|
|
||||||
|
|
||||||
### Artifact Fingerprints
|
|
||||||
|
|
||||||
Define one deterministic, versioned fingerprint per configured artifact using
|
|
||||||
all Narratio-observable inputs that can change its result:
|
|
||||||
|
|
||||||
- its normalized Scriptorium artifact configuration;
|
|
||||||
- its ordered input names, source IDs, required/optional policy, and resolved
|
|
||||||
input content identities;
|
|
||||||
- transcript, prepared-input, previous-session, extraction-lane, and other
|
|
||||||
configured artifact contracts and content checksums;
|
|
||||||
- the current content identities of configured artifact dependencies;
|
|
||||||
- result-affecting Scriptorium adapter configuration visible to Narratio; and
|
|
||||||
- an explicit fingerprint contract version.
|
|
||||||
|
|
||||||
Fingerprint ordering must be deterministic. Identity must not change solely
|
|
||||||
because a workspace moved, an absolute path changed, or an otherwise identical
|
|
||||||
producer used a new run ID. In particular, a forced Notarius invocation that
|
|
||||||
produces byte-identical lanes must not make unrelated analysis artifacts stale
|
|
||||||
solely because the extraction run identity changed.
|
|
||||||
|
|
||||||
Narratio cannot observe arbitrary files, prompts, modules, executable contents,
|
|
||||||
or transitive configuration loaded privately by Scriptorium. Documentation must
|
|
||||||
state that changing an unobserved external input requires explicit force. Do
|
|
||||||
not claim perfect content-addressed reuse beyond Narratio's declared inputs.
|
|
||||||
|
|
||||||
### Freshness And Selection
|
|
||||||
|
|
||||||
Before skipping a succeeded `analyze` stage, an analyze-specific resume
|
|
||||||
validator must reconcile the requested artifact set against current
|
|
||||||
configuration, dependencies, input fingerprints, output records, confined
|
|
||||||
regular files, and stored output checksums.
|
|
||||||
|
|
||||||
The execution rules are:
|
|
||||||
|
|
||||||
- an ordinary run executes only requested artifacts that are missing, stale,
|
|
||||||
invalid, or no longer resumable;
|
|
||||||
- forcing analyze rebuilds all requested targets even when their fingerprints
|
|
||||||
are current;
|
|
||||||
- `--artifacts` identifies explicit targets, not the complete set of records
|
|
||||||
that may remain current;
|
|
||||||
- a selected target's configured prerequisites are processed in deterministic
|
|
||||||
dependency order, reusing them when current and rebuilding them when stale;
|
|
||||||
- forcing a target does not force an otherwise current prerequisite unless it
|
|
||||||
was also explicitly selected;
|
|
||||||
- valid unselected artifact records and outputs survive a partial rerun;
|
|
||||||
- artifacts removed or renamed in current configuration cease to be advertised
|
|
||||||
as current;
|
|
||||||
- an artifact whose dependency or resolved input changes becomes stale unless
|
|
||||||
the new semantic content identity is unchanged; and
|
|
||||||
- stale, missing, failed, or unverified artifacts are unavailable to downstream
|
|
||||||
catalog and publish resolution even if an older file remains on disk.
|
|
||||||
|
|
||||||
If a rebuilt artifact changes, configured dependents that were not part of the
|
|
||||||
invocation are not silently rebuilt. They become stale and will be rebuilt by a
|
|
||||||
later run that selects them. If the rebuilt output is content-identical and the
|
|
||||||
dependent fingerprint remains equal, the dependent may remain current.
|
|
||||||
|
|
||||||
A partial invocation succeeds when its explicit targets and required
|
|
||||||
prerequisites succeed. The aggregate stage record may therefore describe a
|
|
||||||
successful partial invocation while other configured artifacts are stale. The
|
|
||||||
resume validator, not aggregate status alone, must ensure a later full run does
|
|
||||||
not skip unresolved artifact work.
|
|
||||||
|
|
||||||
### Replacement And Failure Safety
|
|
||||||
|
|
||||||
Artifact replacement must preserve unrelated current results while ensuring a
|
|
||||||
failed target is not presented as freshly generated. Run-local output must be
|
|
||||||
validated before canonical materialization and manifest promotion, consistent
|
|
||||||
with existing stage safety policy.
|
|
||||||
|
|
||||||
On partial failure:
|
|
||||||
|
|
||||||
- successfully completed and validated targets may be recorded in the
|
|
||||||
invocation history according to existing runner transaction boundaries;
|
|
||||||
- the failed target and any result whose current identity depends on it must
|
|
||||||
not be advertised as current;
|
|
||||||
- unrelated previously validated artifacts must not be erased merely because
|
|
||||||
they share the `analyze` stage; and
|
|
||||||
- publish and later stage state must remain conservatively stale or failed.
|
|
||||||
|
|
||||||
The implementation must define one clear manifest transition boundary and must
|
|
||||||
not synthesize current output records from directory contents.
|
|
||||||
|
|
||||||
### Legacy Analyze Results
|
|
||||||
|
|
||||||
Existing manifests may contain only an aggregate analyze success and outputs,
|
|
||||||
without versioned per-artifact fingerprints. They remain readable, but Narratio
|
|
||||||
must not invent trustworthy fingerprints for work whose exact inputs were not
|
|
||||||
recorded.
|
|
||||||
|
|
||||||
On first incremental evaluation, legacy analysis artifacts are non-resumable.
|
|
||||||
A full analysis selection rebuilds the effective configured set. A partial
|
|
||||||
selection may rebuild its targets and prerequisites, but unselected legacy
|
|
||||||
outputs remain stale and unavailable until regenerated. Old files and
|
|
||||||
invocation manifests may remain for inspection under existing retention rules.
|
|
||||||
No wholesale manifest rewrite or version-based transcript invalidation is
|
|
||||||
required.
|
|
||||||
|
|
||||||
## Resume And Existing Pipeline Manifests
|
|
||||||
|
|
||||||
The render/extract order change itself requires no manifest migration because
|
|
||||||
stage records use stable names. Under the new sequence:
|
|
||||||
|
|
||||||
- succeeded render and extract records remain eligible for ordinary reuse and
|
|
||||||
their stage-specific validation;
|
|
||||||
- stale, failed, interrupted, and absent records execute in the new order; and
|
|
||||||
- neither result is discarded merely because its relative position changed.
|
|
||||||
|
|
||||||
Bounded execution does not rewrite excluded stage records. Compatibility logic
|
|
||||||
must remain name- and evidence-based; do not invalidate historical transcript
|
|
||||||
work solely because it was produced by an earlier Narratio version.
|
|
||||||
|
|
||||||
The incremental-analysis model may add backward-compatible manifest fields or
|
|
||||||
versioned metadata. Readers must accept older manifests, while new writers must
|
|
||||||
emit one canonical representation and must not maintain parallel legacy and new
|
|
||||||
analysis state indefinitely.
|
|
||||||
|
|
||||||
## Compatibility And Operational Effects
|
|
||||||
|
|
||||||
- Existing unbounded `run`, `run-stage`, `analyze`, and `publish` commands keep
|
|
||||||
their current meanings except for the accepted render/extract order change
|
|
||||||
and more precise analysis reuse.
|
|
||||||
- `--from` and `--through` are additive CLI options; configuration does not gain
|
|
||||||
stage-order or range fields.
|
|
||||||
- `regenerate-artifacts` adds no semantics beyond its documented `run` alias.
|
|
||||||
- Bounded forced runs cannot mutate transcript stages outside their range.
|
|
||||||
- Full runs attempt deterministic render before invoking Notarius.
|
|
||||||
- Forcing render no longer regenerates an otherwise valid Notarius bundle, and
|
|
||||||
forcing extraction no longer regenerates Markdown.
|
|
||||||
- Artifact-level validation adds filesystem hashing and fingerprint work before
|
|
||||||
some analyze skips, trading modest local inspection cost for fewer
|
|
||||||
Scriptorium invocations.
|
|
||||||
- The first analysis evaluation after upgrade may require regeneration because
|
|
||||||
legacy aggregate results do not contain sufficient freshness evidence.
|
|
||||||
|
|
||||||
## Out Of Scope
|
|
||||||
|
|
||||||
- Persistent transcript seals, transcript freeze state, or another lock type.
|
|
||||||
- Run-history listing, run-to-run artifact comparison, or draft promotion.
|
|
||||||
- Parallel execution of render and extract or of analysis artifacts.
|
|
||||||
- Non-contiguous stage selection.
|
|
||||||
- User-configurable stage order or invalidation dependencies.
|
|
||||||
- A generic DAG, phase, job, workflow, or manifest-subtask framework.
|
|
||||||
- New Seriatim, Notarius, or Scriptorium CLI capabilities.
|
|
||||||
- Changes to Notarius lane contracts or Scriptorium output schemas.
|
|
||||||
- Automatic observation of arbitrary transitive Scriptorium files or executable
|
|
||||||
contents.
|
|
||||||
- New render retry, caching, resumability, or fingerprint behavior.
|
|
||||||
- Automatic publish or notification as part of `regenerate-artifacts`.
|
|
||||||
|
|
||||||
## Target End State
|
|
||||||
|
|
||||||
Narratio has one comprehensible pipeline in which transcript production ends at
|
|
||||||
render and post-transcript generation begins at extract. Operators can run any
|
|
||||||
contiguous canonical range without accidentally executing stages outside it,
|
|
||||||
and force applies only within the requested range.
|
|
||||||
|
|
||||||
The common development command:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
narratio regenerate-artifacts SESSION
|
|
||||||
```
|
|
||||||
|
|
||||||
is transparently identical to a forced bounded run from extract through
|
|
||||||
analyze. It preserves transcript state, regenerates the complete Notarius
|
|
||||||
bundle, rebuilds the requested Scriptorium targets, and leaves delivery as a
|
|
||||||
separate explicit action.
|
|
||||||
|
|
||||||
Analysis artifacts have independent, manifest-authoritative freshness within
|
|
||||||
the fixed `analyze` stage. Narratio reuses valid unselected work, rebuilds stale
|
|
||||||
dependencies and selected targets deterministically, withholds stale outputs
|
|
||||||
from downstream consumers, and recognizes content-identical upstream results
|
|
||||||
without tying reuse to ephemeral run paths or IDs.
|
|
||||||
|
|
||||||
Together, canonical ordering, dependency-aware invalidation, bounded execution,
|
|
||||||
the transparent alias, and artifact-level analysis reuse provide an ergonomic
|
|
||||||
development loop without turning Narratio into a general workflow engine.
|
|
||||||
Reference in New Issue
Block a user