Complete downstream semantic resume coverage

This commit is contained in:
2026-08-30 13:14:33 +00:00
parent 7ee637803d
commit 8c1171478d
14 changed files with 737 additions and 67 deletions

View File

@@ -207,12 +207,28 @@ runner marks the aggregate record stale and executes it. Analyze's validator
can still accept a partial selection when only unrelated artifact records are
stale.
The current production semantic contracts cover prepare, transcribe, merge,
polish, normalize, trim, and render. They record resolved choices Narratio can
observe, not operational runner tuning. External model, module, prompt,
profile, and configuration-file contents that a tool privately loads remain
outside the manifest contract when their configured identifier is unchanged;
operators must force the affected stage after such a private content change.
Implemented reuse coverage is deliberately split between aggregate semantic
evidence and focused durable validators:
| Work | Reuse authority | Focused owners |
| --- | --- | --- |
| prepare | aggregate semantic fingerprint | [prepare](stage-prepare.md) |
| transcribe | aggregate semantic fingerprint | [transcribe](stage-transcribe.md), [WhisperX](../integrations/whisperx.md) |
| merge | aggregate semantic fingerprint | [merge](stage-merge.md), [Seriatim](../integrations/seriatim.md) |
| polish | aggregate semantic fingerprint | [polish](stage-polish.md), [Audita](../integrations/audita.md) |
| normalize | aggregate semantic fingerprint | [normalize](stage-normalize.md), [Seriatim](../integrations/seriatim.md) |
| trim | aggregate semantic fingerprint | [trim](stage-trim.md), [Scriptorium](../integrations/scriptorium.md), [Seriatim](../integrations/seriatim.md) |
| render | aggregate semantic fingerprint | [render](stage-render.md), [Seriatim](../integrations/seriatim.md) |
| extract | aggregate semantic fingerprint plus reference/output validator | [extract](stage-extract.md), [Notarius](../integrations/notarius.md) |
| analyze artifacts | per-artifact fingerprint, reconciliation, and output validator | [analyze](stage-analyze.md), [Scriptorium](../integrations/scriptorium.md) |
| publish | aggregate semantic fingerprint plus immediate lock/commit checks | [publish](stage-publish.md), [storage adapter](adapters.md) |
| notify | aggregate delivery-mode fingerprint | [pipeline overview](overview.md), [configuration](../config.md#notifications) |
These contracts record resolved choices Narratio can observe, not operational
runner tuning. External model, module, prompt, profile, and configuration-file
contents that a tool privately loads remain outside the contract when their
configured identifier is unchanged; operators must force the affected work
after such a private content change.
Session manifest is the authoritative stage-progress ledger across invocations.
Run manifest is invocation-scoped audit state.
@@ -222,8 +238,8 @@ comparison and invalidation rules to a cloned manifest. It predicts the runner
decision without persisting session or invocation state. The shared helper
hashes deterministic JSON from stage-owned typed structs; stage providers must
exclude secrets, complete effective-configuration dumps, and operational
values that cannot affect canonical results. Concrete stage coverage is owned
by the focused stage documents as providers are added.
values that cannot affect canonical results. Concrete coverage is owned by the
focused stage and integration documents linked above.
Before an explicitly bounded execution starts after `prepare`, the application
reads the session manifest and accepts only `succeeded` or `skipped` for every

View File

@@ -68,8 +68,9 @@ The implemented canonical order is:
11. `notify` (no-op)
`notify` currently has no persisted pipeline outputs and uses the explicit
`noop` notification mode. The focused stage documents own implementation
mechanics. The
`noop` notification mode. Its versioned semantic evidence records that delivery
mode and excludes adapter credentials and response data. The focused stage
documents own implementation mechanics. The
[CLI](../cli.md) and [Operations](../operations.md) own user-visible invocation
and execution semantics.

View File

@@ -21,8 +21,8 @@ procedures belong in [Operations](../operations.md).
manifest-authoritative identity resolver before creating run-local output;
3. streams each verified reference into an invocation-local snapshot and
rejects any source change observed while copying;
4. fingerprints the Notarius invocation contract, including sorted reference
identities;
4. fingerprints the byte- and provenance-bearing Notarius invocation evidence,
including sorted reference identities;
5. creates a run-local staging directory and invokes the injected
`notarius.Runner`;
6. revalidates the reference snapshots, then validates the v2 successful
@@ -56,14 +56,20 @@ outputs is stable and does not repeatedly invalidate dependent stages.
## Resume Validation
`internal/stage/extract_resume.go` permits a skip only when the existing stage
record succeeded and still matches the current invocation fingerprint. The
fingerprint covers the resolved executable and config paths, pipeline ID,
timeout, working directory, sorted configured output contracts, the current
direct trimmed-transcript identity, and sorted prepared-reference identities.
The same reference helper and transcript identity are resolved again for
artifact evidence, so changing the current transcript bytes or producer
identity makes the prior extraction obsolete.
Before the focused validator runs, the application compares extract's versioned
semantic fingerprint. It covers enablement, Notarius pipeline identity, sorted
reference selector/source mappings, sorted declared output contracts, and each
canonical `narratio.extraction.<key>` output identity. It excludes executable,
timeout, working directory, config path, and private Notarius config contents.
`internal/stage/extract_resume.go` then permits a skip only when the existing
stage record still matches the current byte- and provenance-bearing invocation
evidence. That evidence covers the current direct trimmed-transcript identity,
sorted prepared-reference identities, pipeline identity, and configured output
contracts. The same reference helper and transcript identity are resolved again
for artifact evidence, so changing current transcript bytes, reference bytes,
or producer identity makes the prior extraction obsolete. Operational runner
settings do not invalidate otherwise current durable evidence.
A valid prepared-reference change makes extraction non-resumable. Missing,
unsafe, or checksum-inconsistent prepared evidence is a hard validation error
@@ -76,9 +82,10 @@ contracts and provenance, regular-file status, and stored checksums. Missing or
obsolete results are non-resumable and run again; unsafe filesystem conditions
return an error rather than silently accepting or replacing data.
The fingerprint cannot observe files imported by Notarius configuration,
profile contents, prompt/module definitions, or other transitive inputs.
Operators must force extraction after changing any such input.
Neither contract can observe files imported by Notarius configuration, profile
contents, prompt/module definitions, or other transitive inputs. Operators must
force extraction after changing any such private input behind a stable
identifier.
## Failure Behavior
@@ -101,7 +108,8 @@ available for audit and recovery.
- Stage execution, selection, and resume validation: `internal/stage/extract.go`,
`internal/stage/extract_resume.go`,
`internal/stage/extract_test.go`
`internal/stage/extract_test.go`,
`internal/stage/semantic_contracts_delivery.go`
- Subprocess boundary: `internal/adapters/notarius/subprocess.go`,
`internal/adapters/notarius/subprocess_test.go`
- Catalog hydration: `internal/artifacts/extraction_catalog.go`,

View File

@@ -27,8 +27,8 @@ Exact remote placement and the operator workflow belong in
- when publishing or run upload is disabled, completes successfully with no
outputs and records explanatory metadata. This is not an explicit self-skip:
both manifests record success, and an ordinary later run reuses that result
until publish is forced.
both manifests record success. Enablement and upload policy are fingerprinted,
so changing either automatically makes the prior result non-resumable.
- validates prerequisite stage success and object-store availability.
- derives a deterministic run-archive allowlist from the validated run
`manifest.json`: declared run-local outputs, logs, generated configs, and the
@@ -89,6 +89,16 @@ Includes counts/lists for:
- post-commit local cleanup is authorized by the committed publish metadata and
is durably recorded by the application lifecycle before any local deletion.
## Resume Semantics
The versioned semantic fingerprint covers enabled behavior, run-upload policy,
normalized source/destination/required output rules, static lock policy, and
the remote backend, bucket, region, endpoint, and root-prefix identity. Rule
and lock ordering is canonicalized. Credential environment names,
path-addressing transport mode, local workspace placement, and run identifiers
are excluded. Remote locks remain mutable state and are still revalidated at
the commit boundary; semantic evidence does not replace that safety check.
The commit boundary and cleanup gate are normative architecture invariants; see
[Architecture](../policy/architecture.md#publish-commit-boundary).
@@ -101,5 +111,7 @@ The commit boundary and cleanup gate are normative architecture invariants; see
- [Artifact Internals](artifacts.md) explains source resolution and current-state
helpers.
- Implementation and tests: `internal/stage/publish.go`,
`internal/stage/publish_test.go`, `internal/app/operator_helpers_test.go`,
`internal/stage/publish_test.go`,
`internal/stage/semantic_contracts_delivery.go`,
`internal/app/operator_helpers_test.go`, and
`internal/app/post_publish_cleanup_test.go`

View File

@@ -99,7 +99,9 @@ Execution rules:
repeated self-skip with the same reason and no outputs is stable and does not
perpetually rerun dependent work.
Prepare through render currently provide semantic-configuration evidence.
Every aggregate stage except analyze currently provides semantic-configuration
evidence; analyze retains its more precise per-artifact fingerprints and
validator.
Changing prepare selection semantics reruns all fixed descendants; changing
WhisperX language/service identity reuses prepare; changing a Seriatim merge
transformation reuses prepare and transcribe; and changing an Audita model
@@ -297,6 +299,10 @@ narratio run-stage extract 2026-04-04 --force
Narratio automatically reruns extraction when its recorded invocation contract,
prepared Narratio reference identities, or durable output validation changes.
The semantic portion covers Notarius enablement, pipeline identity, declared
reference mapping, and output contracts. Executable, timeout, working directory,
and private config-file paths are operational and do not invalidate a current
result.
It cannot fingerprint configuration files, profiles, prompts, modules, or
other references loaded transitively by Notarius itself. Force extraction after
changing any of those inputs, even when the top-level Narratio and Notarius
@@ -305,6 +311,12 @@ marks successful downstream stages stale. Ordinary extraction failures or
outcome changes also stale affected downstream stages, while an identical
repeated `notarius_disabled` self-skip does not repeatedly invalidate them.
Publish reuse additionally tracks enabled/run-upload behavior, normalized
output rules, static locks, and remote backend/bucket/region/endpoint/root
identity. Credential environment names, local workspace placement, and run IDs
are excluded. Regardless of semantic reuse evidence, executing publish still
revalidates mutable remote locks immediately before commit selection.
## Publish Workflow
Run publish only:

View File

@@ -392,7 +392,7 @@ prompt changes without invalidating unaffected upstream transcript work.
## Stage 6 — Extract, Analyze, Publish, And Notify Resume Audit
**Status: Pending**
**Status: Completed**
### Goal