narratio/docs/roadmap/contract-audit.md

Contract Simplification Audit

Status

Complete and approved as a statement of intended direction. This document records product and architecture decisions for future work; it does not describe implemented behavior and is not an implementation plan.

This audit applies the same contract-challenge premise used by the File Operations Simplification roadmap to other maintainability findings in audit.md: current behavior is not automatically a durable product requirement merely because production code, tests, or documentation currently preserve it.

Context

Narratio is a trusted, single-session workflow orchestrator for intentionally shareable D&D material. Its most important operational outcomes are correct pipeline progress, economical reuse of expensive successful work, coherent remote publication, understandable recovery, safe cleanup scope, and protection of API keys. It does not need compliance-grade invocation history, indefinite compatibility with already-deprecated formats, or distributed coordination machinery disproportionate to its actual use.

The maintainability audit found several areas where implementation mechanisms had become apparent contracts. This follow-up review explicitly permits those contracts to change when a simpler policy preserves the user outcome that matters.

Governing Decisions

• Remove deprecated legacy and compatibility behavior completely rather than continuing to isolate and maintain it.
• Keep the session manifest as the sole durable local progress authority.
• Preserve successful analyze artifacts individually and checkpoint them as they complete.
• Express local cleanup as an ordinary final pipeline stage rather than a publish side effect or special manifest obligation.
• Represent each remote artifact lock as one independently managed remote object and carry locked artifacts forward into later current snapshots.
• Preserve configuration semantics and useful diagnostics without treating exact validation order, prose, or formatting as compatibility contracts.
• Release this work as a minor version. The removed compatibility surfaces were already deprecated and explicitly subject to removal; their deletion does not require a major-version release under the project decision recorded here.

Finding 1 — Deprecated Compatibility Has Become Cross-Cutting Product Code

Underlying issue

Narratio retains multiple compatibility paths for state and configuration that predate current canonical contracts. These paths include legacy remote current state, candidate probing for previous-session artifacts, unversioned party and separate players inputs, aggregate-only analyze state, older extraction evidence, and manifests without current semantic evidence.

Although several paths are labeled temporary or migration-only, their presence still requires alternate parsing and selection algorithms, ambiguity handling, branches in current workflows, compatibility-specific errors, fixtures, tests, and documentation. Isolating these paths makes later removal easier, but does not remove their ongoing cost. Continued support also makes it harder to reason about which representation is authoritative.

Approved changes

Purge deprecated compatibility behavior across production code, tests, configuration, documentation, examples, and schemas. The purge includes:

• the legacy remote current/manifest.json and current/run_id.txt reader;
• legacy previous-session artifact path candidates and existence probing;
• unversioned party documents and separate players_file behavior;
• legacy session-level party and players overrides that exist only for that input model;
• aggregate-only legacy analyze availability and fallback to incidental files;
• compatibility for extraction records that lack the current contract and provenance evidence;
• compatibility branches for manifests or artifact records that predate the current supported schema; and
• associated legacy constants, types, helpers, errors, tests, fixtures, documentation, and examples.

Current state must have an explicit supported version or enough required authority markers to be recognized unambiguously. Unsupported old local or remote state must fail early with a clear migration/version error. Narratio must not silently normalize, reinterpret, probe around, or partially accept it.

Migration guidance may explain how to regenerate, republish, or otherwise replace old state, but migration readers must not remain in the steady-state runtime merely to preserve automatic compatibility.

Rationale

Compatibility was already deprecated and bounded for removal. Retaining it indefinitely converts a temporary cost into permanent architecture. One canonical representation is easier to validate, test, document, restore, and extend, and a clear rejection is safer than heuristic interpretation of old state.

Final state

Every supported configuration, party document, local manifest, remote current snapshot, extraction record, analyze record, and previous-session artifact is resolved through one canonical contract. Deprecated formats are absent from production code and tests and receive a clear unsupported-version or migration error rather than fallback behavior.

Finding 2 — Invocation Diagnostics Participate in Pipeline Correctness

Underlying issue

Narratio currently maintains both an authoritative session manifest and a structured run manifest for each invocation. The runner persists overlapping stage actions, statuses, outputs, semantic evidence, failures, and terminal state to both documents. This creates dual-save ordering, joined persistence errors, separate session and invocation projections, terminal-failure recovery machinery, and tests for keeping a diagnostic record synchronized with the actual progress ledger.

Historical per-invocation records are not operationally important. They are at most useful diagnostics and do not justify being a second participant in pipeline transactions. Their placement inside a run directory that cleanup may delete further demonstrates that they are not durable progress authority.

Approved changes

Eliminate the structured run manifest and its persistence lifecycle.

Retain:

• the run ID used for invocation identity, run-local paths, adapter provenance, and immutable remote publication;
• the session manifest as the sole durable local record of current pipeline and artifact state;
• ordinary bounded logs, generated configuration, receipts, and other diagnostics where already useful; and
• the in-memory command result or summary reported to the caller.

Remove:

• run-manifest types, storage methods, path helpers, schema and validation;
• before-stage, after-stage, skip, failure, and terminal run-manifest saves;
• invocation-only analyze projections and output projections;
• publish handling that archives or classifies a run manifest;
• terminal persistence code whose only purpose is to coordinate session and run records; and
• run-manifest documentation, compatibility handling, fixtures, and tests.

No replacement historical record is required. A lightweight diagnostic summary may be reconsidered later only if it can remain independent of pipeline correctness and cleanup.

Rationale

One durable authority eliminates synchronization and split-brain questions. The session manifest already records the state needed for resume, status, restore, artifact resolution, and publication. Logs and the immediate command summary provide proportionate invocation diagnostics without making diagnostic write failures fail otherwise valid pipeline work.

Final state

The session manifest alone determines durable local pipeline progress. A run ID still identifies an invocation and its files, but no second manifest mirrors session transitions. Pipeline success or failure cannot be changed by an error writing historical invocation metadata.

Finding 3 — Analyze Recovery Is Obscured by Dual Projection Machinery

Underlying issue

Narratio correctly models configured analyze artifacts individually, but the implementation combines that valuable behavior with separate session and invocation collections, partial-result projection validation, rollback of in-memory authority after persistence failure, and special runner handling for a stage that returns both a result and an error.

The desired user behavior is simpler than the machinery. Expensive successful artifacts should remain reusable; a failed artifact and work that depends on it should be retried. Invocation-specific audit projections are unnecessary once run manifests are removed.

Approved behavior

Assume an ordered artifact plan contains A, B, C, and D. If A and B succeed, C fails, and D is not attempted:

• A and B remain current, provided their output and semantic evidence still validate;
• C is recorded as failed and has no current output authority;
• D remains missing if it has never succeeded, or becomes stale if an older current D depends directly or transitively on C;
• the aggregate analyze stage is failed and execution stops;
• an ordinary later run or resume retains A and B, retries C, and proceeds to D after C succeeds; and
• a forced analyze invocation reruns the effective selection regardless of current records.

Without --artifacts, force applies to the full configured analyze artifact set. With --artifacts, force applies to the selected artifacts and the dependency closure required by the established artifact-selection contract; unrelated valid artifacts remain outside that invocation.

Approved changes

Use the session manifest's analyze artifact collection as the only durable artifact authority. Remove the invocation collection and all logic that exists only to construct, validate, apply, or roll back an invocation-specific projection.

Execute the effective artifact set in deterministic dependency order. After each artifact succeeds and its canonical output has been validated and materialized:

1. construct or update its current session artifact record;
2. stale any current transitive dependents when the new result invalidates their dependency evidence;
3. atomically save the session manifest; and
4. only then proceed to the next artifact.

If the checkpoint save fails, the new file alone does not establish authority; the invocation fails and the artifact is eligible to run again. This safe recomputation is preferable to a rollback transaction around incidental output bytes.

When an artifact fails, record its failed state, stale current transitive dependents, mark the aggregate stage failed, and atomically save those changes to the session manifest before returning the artifact error. A later run reconciles each selected artifact from the persisted collection instead of rerunning the entire stage automatically.

Retain:

• per-artifact fingerprints and validated output evidence;
• deterministic dependency ordering;
• selected-artifact and dependency-closure semantics;
• invalidation when configuration, inputs, dependencies, or output identity changes; and
• the rule that incidental canonical files never create current authority.

Simplify failure and persistence tests around these observable checkpoints rather than the current projection and rollback choreography.

Rationale

Checkpointing after each successful artifact matches the economic reality that LLM work is expensive while one manifest write is cheap. It preserves A and B after an ordinary error, cancellation, or process interruption. The manifest remains the one authority, and a failed checkpoint safely causes recomputation rather than ambiguous reuse.

Dependency-aware reuse is worth retaining because character families, artifact selection, and ongoing prompt development make whole-stage reruns unnecessarily expensive. Invocation-only state is not.

Final state

Analyze behaves as a resumable deterministic artifact queue backed by one session collection. Successful valid artifacts survive later failures, the first failed or invalid artifact is retried, downstream work resumes when its prerequisites become current, and --force deliberately replaces normal reuse within its effective selection.

Finding 4 — Cleanup Is a Hidden Publish Transaction

Underlying issue

Remote publication and local deletion are independent operator outcomes, but Narratio currently couples them. After publish commits remotely, the runner creates a special durable cleanup obligation in the session manifest, deletes eligible local targets one by one, persists completion evidence after each, and reports the invocation as incomplete if cleanup fails. An operator can therefore observe an apparent publish failure even though the remote commit is already authoritative.

The special obligation duplicates behavior already provided by pipeline stage status and resume.

Approved changes

Add cleanup as the final stage in the canonical pipeline, after every stage that may need run-local material or diagnostics. Remove post-publish cleanup from the runner and remove the special post_publish_cleanup manifest model.

The cleanup stage must:

• use existing cleanup configuration to determine whether spool audio, run workspace data, both, or neither should be removed;
• require evidence of a successful current remote publish before deleting data whose retention policy is conditioned on publication;
• bind cleanup targets to the applicable successfully published run rather than infer arbitrary paths from current configuration;
• self-skip clearly when no cleanup is configured;
• use the root-scoped destructive safeguards defined by fileops.md;
• treat an already-absent eligible target as success so retry is idempotent;
• record ordinary stage success or failure in the session manifest; and
• produce no canonical artifact outputs.

A cleanup failure must not alter the succeeded publish record or remote current state. A later ordinary run skips still-valid upstream work and retries cleanup. If an earlier stage is forced or otherwise changes, the fixed invalidation relation must make the prior cleanup result stale where appropriate.

Selecting a range that ends before cleanup must not delete local data. A full default run includes cleanup because it is the canonical final stage.

Rationale

Stage boundaries should match outcomes an operator can understand. Publication has succeeded when the remote commit becomes current; local deletion has succeeded when the selected local targets are absent. Modeling them separately gives each its own status, errors, retry, selection, and documentation without a custom transaction layered after the stage loop.

The removal of run manifests also eliminates the present problem in which cleanup deletes the directory containing an invocation record that the runner otherwise attempts to finalize.

Final state

publish owns only coherent remote publication. cleanup is the final, idempotent local-maintenance stage. A failure is reported as cleanup failure, publish remains visibly successful, and normal resume retries cleanup without republishing unchanged artifacts.

Finding 5 — Remote Locks Use a Shared Mutable Control Plane

Underlying issue

The current lock system stores remote locks in one mutable locks.yml, merges them with static pipeline locks, defines precedence, validates the combined document, uses generation-bound conditional replacement and bounded retries to avoid concurrent lost updates, injects the effective set into runtime configuration, and reloads it before current-pointer selection.

This complexity follows from choosing one shared mutable document and two lock authorities. The product requirement is narrower: an already canonical logical artifact can be frozen so a later publish does not replace the canonical version until an operator unlocks it.

There is an additional semantic mismatch in treating a locked artifact as a simple omission. Narratio publishes immutable run-scoped objects and advances a current commit pointer; it does not overwrite a stable artifact object in place. Omitting a locked artifact from a later commit can make it disappear from current-state discovery rather than preserve the locked canonical version.

Approved lock model

Use one remote lock object per stable logical artifact source. Its location is:

{session_prefix}/locks/{sha256(normalized_source_id)}.lock

The lowercase hexadecimal SHA-256 name provides a fixed, portable object-key segment; the bounded object payload is versioned JSON containing the normalized source ID, an optional reason, and its creation time. Readers verify that the payload source hashes to the object name. The stable artifact source ID, not the configured remote destination, is the lock identity. Configuration can change a destination; it must not accidentally evade a lock on the logical artifact.

Lock object presence is authoritative, subject to successful bounded decoding and identity validation of that payload. No central index is authoritative.

Approved changes

• Remove static pipeline.publish.locks, the mutable central lock-store schema, static/remote merging and precedence, shared-document mutation retries, and runtime injection of locks into resolved pipeline configuration.
• Extend the narrow object-store contract with deletion so an individual lock object can be removed. Implement and test the operation in each storage backend and fake.
• Keep the existing lock list/add/remove user workflow, adapted to per-source objects:
  • list enumerates and validates the lock prefix;
  • add creates one object, fails clearly when it already exists, and permits --force to replace its non-authoritative reason and creation metadata; and
  • remove deletes that object, treating an already-absent object as an idempotent success with a clear informational result.
• Check each relevant lock before spending work to publish its artifact and check again immediately before advancing the current commit pointer.
• Do not introduce a distributed transaction protocol. Narratio accepts the proportionate, documented boundary that a lock created after the final check may race the pointer write; its trusted, low-concurrency operating model does not justify cross-object transaction machinery.

Locked-artifact carry-forward

When building a new remote commit:

• an unlocked artifact uses the newly validated and uploaded artifact declaration;
• a locked artifact already present in the prior current commit carries that exact prior immutable declaration into the new commit;
• a locked artifact with no prior current declaration remains absent and is not treated as a missing required output; and
• invalid or unverifiable prior current state fails publication rather than silently replacing or dropping an artifact that is supposed to be locked.

The initial lock read determines which prior declarations are carried into the new immutable commit. If the final recheck finds that an artifact became locked after that commit was constructed, publication must not advance the current pointer; the unselected immutable attempt may remain, and the next run rebuilds the commit with the prior declaration carried forward. If a lock was removed after the initial read, the in-progress commit may conservatively carry the old declaration once; a later publish can advance it.

The remote commit contract and validation must permit a carried declaration to reference an immutable object under an earlier run prefix within the same campaign/session namespace. Its destination key, checksum, size, storage generation, source identity, and originating run identity remain unchanged and fully declared. Restore and current-state readers continue to trust only the objects explicitly declared by the selected commit; they must not scan mutable prefixes to find a carried artifact.

Publish metadata and inspection commands should distinguish newly published, carried-forward locked, locked-but-never-published, optional missing, and unselected artifacts.

Rationale

Independent lock objects make the existence check the concurrency model. Adding or removing one lock cannot overwrite another lock's state, so a shared YAML merge and retry loop are unnecessary. A stable source identity prevents a configuration destination change from bypassing the operator's intent.

Carrying the prior declaration forward matches the meaning of canonical lock: other artifacts may advance while the locked artifact remains exactly the version that was current when locked. It preserves coherent immutable snapshots without freezing the entire session or copying the old bytes into every new run prefix.

Final state

Each artifact lock is one independently visible remote object managed by small list/add/remove commands. Publish performs a direct lock lookup and a final recheck. New commits preserve prior locked artifact declarations while allowing unlocked artifacts to advance, with no static lock authority, shared mutable lock manifest, precedence rules, or lost-update retry loop.

Finding 6 — Configuration Diagnostics Are Treated as Semantic Contracts

Underlying issue

Configuration composition and validation correctly enforce meaningful product rules, but maintenance guidance and tests have also tended to preserve exact validation order, exact error prose, incidental formatting, and provenance detail whether or not a user-facing operation consumes it. These expectations increase refactoring cost and can discourage clearer validation structure.

The same configuration area also carries deprecated party/player and other legacy compatibility branches addressed by Finding 1.

Approved changes

Retain these configuration contracts:

• strict YAML decoding and rejection of unknown fields;
• additive imports with no duplicate field ownership;
• deterministic selection and application of one profile;
• meaningful explicit empty, false, zero, list, and map overlay behavior;
• centralized defaults and semantic validation;
• canonical party and artifact-family resolution;
• deterministic effective configuration and secret-free digest behavior; and
• diagnostics that identify the relevant field and, for composition conflicts, the conflicting sources well enough for an operator to correct the problem.

Do not treat these as durable contracts unless a specific machine-readable interface explicitly says otherwise:

• the order in which independent validation failures are discovered;
• complete prose wording or punctuation;
• incidental formatting of errors;
• private helper boundaries or validation call order; and
• per-node provenance retained only because the current implementation can produce it.

Remove legacy configuration fields and branches under Finding 1. Refactor validators and composition code around semantic domains without preserving obsolete diagnostics solely to keep exact-string tests unchanged.

Tests should prefer typed errors, error categories, field paths, structured data, or the smallest stable semantic fragment. Exact full-string assertions are appropriate only for documented CLI syntax or an intentionally stable machine-consumed contract.

Rationale

Strictness and deterministic effective behavior prevent ambiguous execution; exact diagnostic choreography does not. Useful errors can remain precise without making their full prose a versioned API. This aligns configuration tests with the repository testing policy: protect meaningful outcomes while allowing legitimate internal reorganization.

Final state

Configuration remains strict, deterministic, source-aware where useful, and operator-friendly. Validators may be reorganized and diagnostics improved without preserving incidental order or wording. Only the canonical party, composition, profile, and artifact-family models remain.

Consolidated Target State

After the approved work is complete:

• Narratio supports one current configuration and state model with no legacy runtime fallbacks;
• the session manifest is the only durable local progress ledger;
• analyze checkpoints each successful artifact, preserves valid completed work, and resumes from failed, stale, or missing work in dependency order;
• cleanup is the final canonical stage and cannot make a committed publish appear to have failed;
• remote locks are independent per-artifact objects, and locked canonical artifacts are carried forward across new current snapshots;
• configuration tests protect semantic rules rather than diagnostic choreography;
• ordinary logs and command summaries provide proportionate diagnostics without participating in correctness; and
• the combined simplification is eligible for a minor release because the removed compatibility behavior was already deprecated and removable.

Open Questions

None. The policy and target-state decisions required for a later feature roadmap and staged implementation plan are resolved by this audit.