Complete downstream semantic resume coverage
This commit is contained in:
@@ -207,12 +207,28 @@ runner marks the aggregate record stale and executes it. Analyze's validator
|
||||
can still accept a partial selection when only unrelated artifact records are
|
||||
stale.
|
||||
|
||||
The current production semantic contracts cover prepare, transcribe, merge,
|
||||
polish, normalize, trim, and render. They record resolved choices Narratio can
|
||||
observe, not operational runner tuning. External model, module, prompt,
|
||||
profile, and configuration-file contents that a tool privately loads remain
|
||||
outside the manifest contract when their configured identifier is unchanged;
|
||||
operators must force the affected stage after such a private content change.
|
||||
Implemented reuse coverage is deliberately split between aggregate semantic
|
||||
evidence and focused durable validators:
|
||||
|
||||
| Work | Reuse authority | Focused owners |
|
||||
| --- | --- | --- |
|
||||
| prepare | aggregate semantic fingerprint | [prepare](stage-prepare.md) |
|
||||
| transcribe | aggregate semantic fingerprint | [transcribe](stage-transcribe.md), [WhisperX](../integrations/whisperx.md) |
|
||||
| merge | aggregate semantic fingerprint | [merge](stage-merge.md), [Seriatim](../integrations/seriatim.md) |
|
||||
| polish | aggregate semantic fingerprint | [polish](stage-polish.md), [Audita](../integrations/audita.md) |
|
||||
| normalize | aggregate semantic fingerprint | [normalize](stage-normalize.md), [Seriatim](../integrations/seriatim.md) |
|
||||
| trim | aggregate semantic fingerprint | [trim](stage-trim.md), [Scriptorium](../integrations/scriptorium.md), [Seriatim](../integrations/seriatim.md) |
|
||||
| render | aggregate semantic fingerprint | [render](stage-render.md), [Seriatim](../integrations/seriatim.md) |
|
||||
| extract | aggregate semantic fingerprint plus reference/output validator | [extract](stage-extract.md), [Notarius](../integrations/notarius.md) |
|
||||
| analyze artifacts | per-artifact fingerprint, reconciliation, and output validator | [analyze](stage-analyze.md), [Scriptorium](../integrations/scriptorium.md) |
|
||||
| publish | aggregate semantic fingerprint plus immediate lock/commit checks | [publish](stage-publish.md), [storage adapter](adapters.md) |
|
||||
| notify | aggregate delivery-mode fingerprint | [pipeline overview](overview.md), [configuration](../config.md#notifications) |
|
||||
|
||||
These contracts record resolved choices Narratio can observe, not operational
|
||||
runner tuning. External model, module, prompt, profile, and configuration-file
|
||||
contents that a tool privately loads remain outside the contract when their
|
||||
configured identifier is unchanged; operators must force the affected work
|
||||
after such a private content change.
|
||||
|
||||
Session manifest is the authoritative stage-progress ledger across invocations.
|
||||
Run manifest is invocation-scoped audit state.
|
||||
@@ -222,8 +238,8 @@ comparison and invalidation rules to a cloned manifest. It predicts the runner
|
||||
decision without persisting session or invocation state. The shared helper
|
||||
hashes deterministic JSON from stage-owned typed structs; stage providers must
|
||||
exclude secrets, complete effective-configuration dumps, and operational
|
||||
values that cannot affect canonical results. Concrete stage coverage is owned
|
||||
by the focused stage documents as providers are added.
|
||||
values that cannot affect canonical results. Concrete coverage is owned by the
|
||||
focused stage and integration documents linked above.
|
||||
|
||||
Before an explicitly bounded execution starts after `prepare`, the application
|
||||
reads the session manifest and accepts only `succeeded` or `skipped` for every
|
||||
|
||||
@@ -68,8 +68,9 @@ The implemented canonical order is:
|
||||
11. `notify` (no-op)
|
||||
|
||||
`notify` currently has no persisted pipeline outputs and uses the explicit
|
||||
`noop` notification mode. The focused stage documents own implementation
|
||||
mechanics. The
|
||||
`noop` notification mode. Its versioned semantic evidence records that delivery
|
||||
mode and excludes adapter credentials and response data. The focused stage
|
||||
documents own implementation mechanics. The
|
||||
[CLI](../cli.md) and [Operations](../operations.md) own user-visible invocation
|
||||
and execution semantics.
|
||||
|
||||
|
||||
@@ -21,8 +21,8 @@ procedures belong in [Operations](../operations.md).
|
||||
manifest-authoritative identity resolver before creating run-local output;
|
||||
3. streams each verified reference into an invocation-local snapshot and
|
||||
rejects any source change observed while copying;
|
||||
4. fingerprints the Notarius invocation contract, including sorted reference
|
||||
identities;
|
||||
4. fingerprints the byte- and provenance-bearing Notarius invocation evidence,
|
||||
including sorted reference identities;
|
||||
5. creates a run-local staging directory and invokes the injected
|
||||
`notarius.Runner`;
|
||||
6. revalidates the reference snapshots, then validates the v2 successful
|
||||
@@ -56,14 +56,20 @@ outputs is stable and does not repeatedly invalidate dependent stages.
|
||||
|
||||
## Resume Validation
|
||||
|
||||
`internal/stage/extract_resume.go` permits a skip only when the existing stage
|
||||
record succeeded and still matches the current invocation fingerprint. The
|
||||
fingerprint covers the resolved executable and config paths, pipeline ID,
|
||||
timeout, working directory, sorted configured output contracts, the current
|
||||
direct trimmed-transcript identity, and sorted prepared-reference identities.
|
||||
The same reference helper and transcript identity are resolved again for
|
||||
artifact evidence, so changing the current transcript bytes or producer
|
||||
identity makes the prior extraction obsolete.
|
||||
Before the focused validator runs, the application compares extract's versioned
|
||||
semantic fingerprint. It covers enablement, Notarius pipeline identity, sorted
|
||||
reference selector/source mappings, sorted declared output contracts, and each
|
||||
canonical `narratio.extraction.<key>` output identity. It excludes executable,
|
||||
timeout, working directory, config path, and private Notarius config contents.
|
||||
|
||||
`internal/stage/extract_resume.go` then permits a skip only when the existing
|
||||
stage record still matches the current byte- and provenance-bearing invocation
|
||||
evidence. That evidence covers the current direct trimmed-transcript identity,
|
||||
sorted prepared-reference identities, pipeline identity, and configured output
|
||||
contracts. The same reference helper and transcript identity are resolved again
|
||||
for artifact evidence, so changing current transcript bytes, reference bytes,
|
||||
or producer identity makes the prior extraction obsolete. Operational runner
|
||||
settings do not invalidate otherwise current durable evidence.
|
||||
|
||||
A valid prepared-reference change makes extraction non-resumable. Missing,
|
||||
unsafe, or checksum-inconsistent prepared evidence is a hard validation error
|
||||
@@ -76,9 +82,10 @@ contracts and provenance, regular-file status, and stored checksums. Missing or
|
||||
obsolete results are non-resumable and run again; unsafe filesystem conditions
|
||||
return an error rather than silently accepting or replacing data.
|
||||
|
||||
The fingerprint cannot observe files imported by Notarius configuration,
|
||||
profile contents, prompt/module definitions, or other transitive inputs.
|
||||
Operators must force extraction after changing any such input.
|
||||
Neither contract can observe files imported by Notarius configuration, profile
|
||||
contents, prompt/module definitions, or other transitive inputs. Operators must
|
||||
force extraction after changing any such private input behind a stable
|
||||
identifier.
|
||||
|
||||
## Failure Behavior
|
||||
|
||||
@@ -101,7 +108,8 @@ available for audit and recovery.
|
||||
|
||||
- Stage execution, selection, and resume validation: `internal/stage/extract.go`,
|
||||
`internal/stage/extract_resume.go`,
|
||||
`internal/stage/extract_test.go`
|
||||
`internal/stage/extract_test.go`,
|
||||
`internal/stage/semantic_contracts_delivery.go`
|
||||
- Subprocess boundary: `internal/adapters/notarius/subprocess.go`,
|
||||
`internal/adapters/notarius/subprocess_test.go`
|
||||
- Catalog hydration: `internal/artifacts/extraction_catalog.go`,
|
||||
|
||||
@@ -27,8 +27,8 @@ Exact remote placement and the operator workflow belong in
|
||||
|
||||
- when publishing or run upload is disabled, completes successfully with no
|
||||
outputs and records explanatory metadata. This is not an explicit self-skip:
|
||||
both manifests record success, and an ordinary later run reuses that result
|
||||
until publish is forced.
|
||||
both manifests record success. Enablement and upload policy are fingerprinted,
|
||||
so changing either automatically makes the prior result non-resumable.
|
||||
- validates prerequisite stage success and object-store availability.
|
||||
- derives a deterministic run-archive allowlist from the validated run
|
||||
`manifest.json`: declared run-local outputs, logs, generated configs, and the
|
||||
@@ -89,6 +89,16 @@ Includes counts/lists for:
|
||||
- post-commit local cleanup is authorized by the committed publish metadata and
|
||||
is durably recorded by the application lifecycle before any local deletion.
|
||||
|
||||
## Resume Semantics
|
||||
|
||||
The versioned semantic fingerprint covers enabled behavior, run-upload policy,
|
||||
normalized source/destination/required output rules, static lock policy, and
|
||||
the remote backend, bucket, region, endpoint, and root-prefix identity. Rule
|
||||
and lock ordering is canonicalized. Credential environment names,
|
||||
path-addressing transport mode, local workspace placement, and run identifiers
|
||||
are excluded. Remote locks remain mutable state and are still revalidated at
|
||||
the commit boundary; semantic evidence does not replace that safety check.
|
||||
|
||||
The commit boundary and cleanup gate are normative architecture invariants; see
|
||||
[Architecture](../policy/architecture.md#publish-commit-boundary).
|
||||
|
||||
@@ -101,5 +111,7 @@ The commit boundary and cleanup gate are normative architecture invariants; see
|
||||
- [Artifact Internals](artifacts.md) explains source resolution and current-state
|
||||
helpers.
|
||||
- Implementation and tests: `internal/stage/publish.go`,
|
||||
`internal/stage/publish_test.go`, `internal/app/operator_helpers_test.go`,
|
||||
`internal/stage/publish_test.go`,
|
||||
`internal/stage/semantic_contracts_delivery.go`,
|
||||
`internal/app/operator_helpers_test.go`, and
|
||||
`internal/app/post_publish_cleanup_test.go`
|
||||
|
||||
@@ -99,7 +99,9 @@ Execution rules:
|
||||
repeated self-skip with the same reason and no outputs is stable and does not
|
||||
perpetually rerun dependent work.
|
||||
|
||||
Prepare through render currently provide semantic-configuration evidence.
|
||||
Every aggregate stage except analyze currently provides semantic-configuration
|
||||
evidence; analyze retains its more precise per-artifact fingerprints and
|
||||
validator.
|
||||
Changing prepare selection semantics reruns all fixed descendants; changing
|
||||
WhisperX language/service identity reuses prepare; changing a Seriatim merge
|
||||
transformation reuses prepare and transcribe; and changing an Audita model
|
||||
@@ -297,6 +299,10 @@ narratio run-stage extract 2026-04-04 --force
|
||||
|
||||
Narratio automatically reruns extraction when its recorded invocation contract,
|
||||
prepared Narratio reference identities, or durable output validation changes.
|
||||
The semantic portion covers Notarius enablement, pipeline identity, declared
|
||||
reference mapping, and output contracts. Executable, timeout, working directory,
|
||||
and private config-file paths are operational and do not invalidate a current
|
||||
result.
|
||||
It cannot fingerprint configuration files, profiles, prompts, modules, or
|
||||
other references loaded transitively by Notarius itself. Force extraction after
|
||||
changing any of those inputs, even when the top-level Narratio and Notarius
|
||||
@@ -305,6 +311,12 @@ marks successful downstream stages stale. Ordinary extraction failures or
|
||||
outcome changes also stale affected downstream stages, while an identical
|
||||
repeated `notarius_disabled` self-skip does not repeatedly invalidate them.
|
||||
|
||||
Publish reuse additionally tracks enabled/run-upload behavior, normalized
|
||||
output rules, static locks, and remote backend/bucket/region/endpoint/root
|
||||
identity. Credential environment names, local workspace placement, and run IDs
|
||||
are excluded. Regardless of semantic reuse evidence, executing publish still
|
||||
revalidates mutable remote locks immediately before commit selection.
|
||||
|
||||
## Publish Workflow
|
||||
|
||||
Run publish only:
|
||||
|
||||
@@ -392,7 +392,7 @@ prompt changes without invalidating unaffected upstream transcript work.
|
||||
|
||||
## Stage 6 — Extract, Analyze, Publish, And Notify Resume Audit
|
||||
|
||||
**Status: Pending**
|
||||
**Status: Completed**
|
||||
|
||||
### Goal
|
||||
|
||||
|
||||
Reference in New Issue
Block a user