narratio/docs/roadmap/fileops.md

# File Operations Simplification

## Status

Accepted feature roadmap. This document defines the desired filesystem policy
and target state. It is not an implementation plan and does not describe
implemented behavior. It refines the filesystem conclusions in
[`audit.md`](audit.md).

## Goal

Make Narratio's filesystem code proportionate to its real operating model:
single-invocation work under a session lock in a trusted, group-owned workspace
containing intentionally shareable D&D material. Use the Go standard library
directly for ordinary filesystem work and retain custom code only where it
expresses a concrete Narratio requirement.

The change should materially reduce production code, platform-specific code,
test machinery, full-tree copying, and the number of filesystem guarantees that
maintainers must preserve.

## Underlying Issue

`internal/fileops` has accumulated a general confined-filesystem abstraction,
two directory-promotion implementations, adversarial race hooks and inode
identity rechecks, handle-relative no-replace promotion, explicit file and
directory syncing, platform capability errors, universal symlink and regular-
file restrictions, and tests for each mechanism.

Those mechanisms are internally coherent, but they solve a stronger problem
than Narratio has. Narratio does not operate on a hostile multi-writer tree, and
its ordinary workspace data is not private. The session lock prevents normal
concurrent mutation by Narratio, while run IDs make durable extract destinations
unique. Treating every current mechanism as a permanent policy has therefore
created a self-reinforcing contract: implementation detail drives tests and
documentation, which in turn appear to require the implementation detail.

This roadmap resets that contract around required outcomes. It does not assume
that a replacement must preserve identical mechanics or failure behavior.

## Policy Decisions

### Operating and threat model

- Campaign and session workspaces are trusted operational inputs, not hostile
  filesystems. Narratio does not defend ordinary reads and writes against an
  attacker racing directory entries during an invocation.
- Workspace transcripts, artifacts, manifests, logs, and configuration are
  intended to be shareable. World readability is acceptable. Group-owned,
  group-writable operation remains a requirement.
- API keys are the exception. Narratio must not write credentials into runtime
  artifacts or diagnostics, and any filesystem path that can contain API-key
  material retains appropriately strict handling.
- Narratio must still prevent accidental destructive action or restore outside
  the selected application-owned root. Trusted input does not justify an
  unbounded delete or an escaped restore target.

### Guarantees to retain

- Corruption-sensitive single-file updates use same-directory temporary files
  and replacement so readers do not observe partially written content. This
  applies to such state as manifests and generated configuration; it is an
  atomic-visibility requirement, not a power-loss durability promise.
- Files downloaded from remote storage are completely received and validated
  before they become the canonical local file.
- Adapter and configuration input remains bounded by centralized, discoverable
  constants. Limits remain deliberately far above expected workloads and
  produce unmistakable errors naming both the limit and the enforcing adapter.
- Workspace files and directories preserve the repository's group-owned,
  group-writable mode policy, including after temporary-file replacement or
  directory rename.
- Cleanup and restore operate relative to an explicitly opened root and reject
  targets outside that scope. Recursive removal uses the standard-library root
  primitive available at the maintained Go baseline.
- Restore treats leaf or ancestor symlinks as non-regular conflicts rather than
  following them to classify or overwrite a target. This is a restore boundary,
  not a universal rule for ordinary workspace access.
- Staging cleanup is limited to paths that Narratio created and verified as
  descendants of the applicable invocation or session workspace.

### Guarantees to retire

- Narratio does not promise that a successful write or directory install has
  been forced through every operating-system and storage cache before return.
  File and directory `fsync` calls, directory-sync capability probes, and
  unsupported-durability errors are not part of the target contract.
- Narratio does not defend against a concurrent adversary replacing a file or
  directory after it has been inspected. Inode identity snapshots, `SameFile`
  rechecks, and race-injection hooks are not part of the target contract.
- Ordinary workspace operations do not universally reject symlinks or require
  every source to be a regular file. Components may validate a concrete external
  contract where needed, but there is no package-wide no-follow policy.
- Directory installation does not require a custom atomic no-replace primitive
  or an exact cross-platform race guarantee. The session lock, absent-destination
  check, and unique run ID are the intended concurrency controls.
- “Durable” in Narratio documentation means managed persistent application
  state. It does not mean guaranteed survival of every acknowledged update
  across immediate power loss.

## Required Changes

### Narrow the filesystem API

Replace the broad `internal/fileops` facade with direct `os`, `io`, and
`path/filepath` operations at ordinary call sites. Retain only small helpers
that enforce a shared product rule rather than wrapping the standard library.
Expected shared concerns are:

- corruption-resistant single-file replacement with explicit final mode;
- bounded reads or copies with centralized limits and contextual errors;
- application workspace file and directory modes; and
- narrowly scoped root-relative cleanup and restore operations.

The target is not a one-for-one rewrite of existing exported functions. Remove
unused or single-purpose wrappers when the calling package can express the
operation clearly with the standard library. Keep platform-specific code only
if it is necessary for the small atomic-file replacement helper to provide the
supported cross-platform behavior.

Do not add a general filesystem framework. Same-directory atomic replacement is
a generic abstraction, but a small local helper is the preferred target unless
implementation research identifies a maintained cross-platform dependency that
removes more code than it adds and preserves the required mode behavior.

### Simplify ordinary reads, writes, and copies

Use ordinary standard-library path operations for trusted workspace data.
Preserve contextual errors and the existing centralized safety limits, but do
not route every access through an `os.Root`, reopen files to prove identity, or
reject symlinks merely because the current facade does so.

Callers that consume a defined external format may still validate type, size,
checksum, structure, or placement because their own contract requires it. Those
checks should live with the consumer and must not reconstitute a universal
FileOps policy indirectly.

### Replace file updates without power-loss machinery

Use a minimal same-directory temporary-file flow for corruption-sensitive file
updates: create, write, close, apply the intended mode, and replace the canonical
path. Ensure temporary files are removed after pre-commit failures. Preserve
context cancellation checks where a caller can still abandon an update before
replacement.

Remove file and parent-directory syncing, sync-related platform probes and
sentinel errors, and tests that exist only to assert power-loss guarantees. If a
small operating-system-specific replace shim is still required, keep it private
to this helper and test its observable replacement behavior rather than its
internal syscall sequence.

### Simplify download installation

Keep remote downloads in a temporary file until transfer, size, checksum, and
format validation succeeds. Install the validated file using the same minimal
replacement helper or a direct rename when the destination is known absent.
Remove standalone download-install APIs that have no production caller and
avoid duplicating replacement policy between download and ordinary writes.

### Replace Notarius directory promotion

Have Notarius write its output bundle to a unique staging directory within the
invocation workspace. Validate that the reported bundle is inside the requested
output root and satisfies the Notarius integration contract. Normalize required
workspace modes, confirm that the unique durable destination is absent, and use
`os.Rename` to move the validated directory directly to that destination.

Delete the production and test-only directory tree-copy implementations,
promotion hooks, handle-relative no-replace code, directory syncing, and their
platform files. A destination collision must return a clear ordinary error; it
does not require adversarial race recovery.

The successfully renamed bundle becomes the sole retained artifact tree. The
pre-promotion Notarius tree is not duplicated for diagnostics. Existing receipt
and log records remain the troubleshooting evidence, and failed staging trees
are cleaned up only after their scope is verified.

### Keep destructive and restore operations root-scoped

Retain `os.Root` at boundaries where containment is the requirement. Use the
maintained Go version's root-relative recursive removal for cleanup. Continue to
require proper-descendant relationships and application-owned eligibility
before destructive operations.

Rework restore classification and installation around a session root. Inspect
targets without following leaf or ancestor symlinks, classify such targets as
non-regular conflicts in both force and non-force plans, and hash only a target
verified within that root. Do not add inode-stability or hostile-race rechecks.

### Preserve group-owned workspace operation and protect secrets

Centralize the intended workspace modes in one discoverable location and apply
them consistently to created directories, ordinary files, atomic replacements,
downloads, and renamed Notarius bundles. Do not tighten general workspace
permissions on the assumption that transcripts or artifacts are sensitive.

Audit API-key ingress separately from general workspace operations. Secret
values must remain out of files unless an existing explicit credential contract
requires a file, must never enter logs or errors, and must not inherit permissive
workspace modes. The simplification must not weaken existing credential
redaction or environment handling.

### Align documentation and tests with outcomes

Revise the canonical architecture, operations, troubleshooting, and focused
internal documents to remove power-loss durability, universal confinement,
universal no-follow, and atomic directory-promotion claims. Keep volatile modes,
paths, limits, and operational behavior in their existing canonical owners
rather than duplicating them across documents.

Delete tests for removed algorithms, syscall ordering, race hooks, directory-
sync capability behavior, and guarantees no longer offered. Retain or add
focused tests for:

- successful and failed atomic file replacement on every supported platform;
- cleanup of temporary files before replacement;
- bounded input and clear adapter-specific limit errors;
- group-owned file and directory modes;
- validated download installation;
- validated Notarius bundle rename, collision, and cleanup behavior;
- root-scoped cleanup and refusal of escaped destructive targets;
- restore handling of leaf and ancestor symlinks; and
- API-key redaction and any credential-file permission contract.

Tests should assert externally meaningful results and failure boundaries, not
recreate the removed implementation through new private hooks.

## Compatibility and Migration

This work changes internal guarantees rather than CLI, configuration, manifest,
or external artifact schemas. Existing valid workspaces should remain usable.
Users may observe fewer failures on filesystems that cannot sync directories,
faster extract installation, less temporary disk consumption, and ordinary
symlink behavior outside explicitly scoped restore and destructive operations.

The removal of the duplicate pre-promotion Notarius output tree is an intentional
operational change and must be reflected in the operations and troubleshooting
documentation. No compatibility layer should preserve the old FileOps API solely
for internal callers; migrate callers and delete obsolete surfaces together.

## Target End State

At completion:

- ordinary workspace file operations are clear standard-library code located
  near their domain owners;
- shared filesystem code is small and limited to atomic file replacement,
  bounded I/O and modes, plus explicitly root-scoped destructive or restore
  helpers;
- directory promotion, tree-copy duplication, adversarial race defenses,
  directory syncing, and unsupported-durability machinery no longer exist;
- extract validates and renames one Notarius staging bundle without copying the
  complete tree or retaining a duplicate source tree;
- manifests and other corruption-sensitive files are never intentionally
  exposed partially, without claiming power-loss durability;
- accidental out-of-root deletion and restore remain prevented;
- group-owned workspaces remain first-class, and API keys remain the only data
  requiring secret-specific filesystem treatment;
- tests cover the reduced observable contract rather than removed mechanisms;
  and
- canonical documentation accurately distinguishes persistent application state,
  atomic visibility, trusted-workspace assumptions, and destructive-operation
  confinement.

## Out of Scope

- Changing CLI commands, configuration schemas, artifact schemas, or manifest
  formats.
- Adding multi-process writes to one session or removing the session lock.
- Defending trusted workspaces against malicious concurrent mutation.
- Adding encryption or restrictive permissions for shareable campaign data.
- Redesigning remote publish commit ordering or object-store concurrency.
- Adding user-configurable size limits in this feature; constants should remain
  easy to promote to configuration later.