narratio/docs/internal/artifacts.md

Internal: Artifacts

Purpose

Explain the artifact registry, runtime catalog, resolver, previous-input requirements, and shared remote current-state mechanics implemented by internal/artifacts. Configuration fields that accept source IDs belong in Configuration; physical placement belongs in Operations.

Built-in Source IDs

The internal registry recognizes these stable built-in source IDs:

• narratio.transcript.base
• narratio.transcript.polished
• narratio.transcript.final
• narratio.transcript.final_trimmed
• narratio.transcript.final_markdown
• narratio.transcript.final_trimmed_markdown
• narratio.bounds.session

Registry entries bind each ID to its producer, output kind, canonical fallback, and content validator. The focused stage documents own their input/output flow; Configuration owns where operators may select these IDs.

Configured, Extraction, And Previous-Session Sources

• configured source ID format: narratio.artifact.<artifact_key>
• extraction source ID format: narratio.extraction.<output_key>
• previous-session source ID format: narratio.previous_session.artifact.<artifact_key>

All formats are validated by strict source-policy rules. Configured artifact and extraction keys use ^[a-z][a-z0-9_]*$; source parsers never normalize an unrecognized token into a valid source. Extraction sources are registered only from pipeline.notarius.outputs; the Notarius index has no selectable source ID.

Prepared stable source IDs are narratio.input.players, narratio.input.party, narratio.input.glossary, and narratio.input.spell_catalog. Artifact policy owns their canonical manifest kind and prepared filename vocabulary. Canonical party mode preserves the party source bytes in the party record and supplies the players record from the deterministic derived_from_party projection; both remain ordinary prepared source IDs for consumers.

Runtime Catalog

ArtifactCatalog tracks:

• planned: source registered for run context;
• executable: included in the effective analyze artifact set;
• available: the source's canonical evidence owner validates its current manifest record and durable bytes;
• provenance: availability source.

Configured definitions are always registered. Without an explicit selection, the effective analyze set contains enabled definitions. With --artifacts, the exact named configured definitions become the effective set for that invocation, regardless of their enabled value. The effective-set resolver itself does not expand dependencies; the analyze work planner closes those targets over their configured prerequisite graph. Availability is separate from executability. Configuration may normalize a family selection into its concrete generated members before this resolver runs. The effective set retains optional family and character origin metadata, but its keys, catalog sources, and runtime lookups remain concrete configured-artifact identities. Family publish policies are likewise expanded into ordinary configured-source publish rules during configuration resolution. Configured outputs, including non-executable prerequisites, become available only when the versioned analyze state identifies a current result whose source, contract, canonical configured path, size, and checksum match a confined no-follow regular file. An incidental canonical file and a legacy aggregate analyze output are unavailable. Extraction entries are registered from configuration and become available only after compatible extraction evidence is hydrated.

During an analyze invocation, a newly validated and atomically materialized configured output is marked available with its producer run ID, contract, checksum, and size. Later scheduled dependents therefore observe the same semantic identity whether their prerequisite was reused from current manifest evidence or produced earlier in the invocation.

Current provenance values:

• generated.current_analyze_run
• manifest.current_analyze_artifact
• manifest.inputs.previous_cache
• current_session.previous_cache

Resolution Rules

Built-ins:

1. manifest producer outputs (when present)
2. canonical session-path fallback

Configured sources (narratio.artifact.*):

• resolve only through runtime catalog availability;
• use the shared typed analyze-evidence inspection in analyze_evidence.go for prior current-session results;
• require the supported analyze-state and fingerprint versions, a current record for the exact configured key and source ID, a complete contract, the configured canonical relative path, positive stored size, and stored checksum matching bytes read from a confined no-follow regular file; and
• treat non-current statuses, legacy or malformed records, removed keys, unsafe or missing files, and size/checksum mismatches as unavailable without rewriting manifest state. Catalog construction iterates current configuration, so removed or renamed records are not advertised.

narratio.member_artifact.* is not a runtime source family. Configuration resolution accepts it only in an artifact-family declaration and rewrites it to the corresponding configured source before this catalog is built.

Prepared stable sources (narratio.input.*):

• resolve only from the current manifest's exact prepared-input record;
• require the policy-owned canonical path below the session root, a confined non-symlink regular file, a non-empty payload, and a matching SHA-256 checksum; and
• return an immutable source/path/checksum/size identity shared by extract and analyze rather than falling back to campaign/session source paths.

Extraction sources (narratio.extraction.*):

• use the shared typed bundle evidence inspection in extraction_evidence.go;
• require a current successful extract record with the exact configured source, compatible contract and Notarius provenance, a confined regular durable payload, matching checksum, and the current resolved trimmed-transcript identity;
• remain unavailable unless catalog hydration receives valid evidence. Resume treats absent or obsolete evidence as a rerun decision and unsafe evidence as an error; and
• are never inferred by scanning the Notarius bundle directory.

Previous-session sources (narratio.previous_session.artifact.*):

• resolve only from local previous/ cache state;
• prefer manifest-backed previous-input paths;
• fallback to existing previous-cache filesystem paths.

Source absence is evaluated by the consuming artifact input. An optional input is omitted from that invocation; a required input fails resolution. This is separate from a stage's lifecycle outcome.

Validation by content type:

• transcript JSON built-ins: JSON with top-level segments array;
• transcript Markdown built-ins: non-empty text file;
• bounds built-in: valid JSON;
• configured/previous-session artifact files: non-empty text file.

Previous Requirement Collection

CollectPreviousArtifactRequirements:

• scans the effective configured artifact set;
• extracts only canonical previous-session sources;
• deduplicates by artifact key;
• merges required and optional references (required wins);
• returns deterministic ordering and source locations.

Current-State Helpers

Artifacts package owns shared remote current-state loading mechanics used by restore, status and validation checks, and previous-cache planning.

For a new-protocol current state, the pointer-selected immutable commit is the complete restore authority. Callers receive its declared object identities and must not supplement them by listing mutable session prefixes. The legacy reader is intentionally separate and remains migration-only support.

The reader opens each small control object directly and enforces owner-specific limits before decoding: 64 KiB for the mutable commit pointer, 4 MiB for the immutable commit manifest, and 8 MiB for the selected session manifest. Legacy compatibility applies a 4 KiB limit to current/run_id.txt and the same 8 MiB manifest limit to current/manifest.json. These are exposed as MaxCurrentCommitPointerBytes, MaxRemoteCommitManifestBytes, MaxRemoteSessionManifestBytes, MaxLegacyCurrentRunPointerBytes, and MaxLegacyCurrentManifestBytes.

Each read uses the generation and size metadata returned with its opened body. Actual bytes remain subject to a limit-plus-one read even if size metadata is absent or inaccurate. Immutable selections then retain their declared-size, checksum, generation, and identity checks. No current-state control object is downloaded through a temporary file.

Core helpers:

• LoadCurrentState
• ValidateCurrentStateIdentity
• RemoteCommitManifest and CurrentCommitPointer

Typed missing-state errors:

• CurrentRunPointerMissingError (ErrCurrentRunPointerMissing)
• CurrentManifestMissingError (ErrCurrentManifestMissing)

Identity validation supports caller-provided expectations:

• expected campaign;
• expected session ID;
• expected run ID, or pointer/manifest run-ID consistency check.

Caller policy is intentionally outside artifacts helpers:

• some callers fail on missing current state;
• some callers downgrade missing state to status/findings;
• some callers skip optional behavior when state is missing.

Key Path Helpers

internal/artifacts/paths.go and S3-key helpers define canonical helpers for:

• session/work/run paths;
• previous-cache paths;
• spool/cache paths;
• S3 session/run/current-state key layout.

New publication creates run-scoped immutable objects, including runs/{run_id}/commit.json and runs/{run_id}/session-manifest.json. The sole mutable selector is current/commit-pointer.json; readers verify its selected commit and declared object generations/checksums. Legacy current-pair loading is confined to current_state_legacy.go for migration only.

Campaign, session, and Narratio run IDs are validated as portable opaque segments at configuration and artifact boundaries before they can be used in a workspace or S3 namespace. Previous-artifact destinations remain typed, multi-segment relative paths and are confined beneath previous/artifacts; they are not treated as opaque identifiers.

See Workspace Internals for how callers consume local helpers and Operations for the authoritative physical layout.

Invariants

• source ID formats are stable contracts;
• artifact resolution is deterministic and manifest-aware;
• extraction sources are available only from a compatible successful manifest record;
• previous-session source resolution in analyze is local-only;
• remote current-state key construction remains centralized in artifacts helpers.

Implementation And Tests

• Registry and resolution: internal/artifacts/artifact_resolver.go, internal/artifacts/catalog.go, internal/artifacts/transcripts.go, internal/artifacts/extraction_catalog.go, internal/artifacts/extraction_evidence.go, internal/artifacts/extraction_input.go, internal/artifacts/prepared_input.go
• Current state: internal/artifacts/current_state.go, internal/artifacts/current_state_commit.go, internal/artifacts/current_state_legacy.go
• Paths and keys: internal/artifacts/paths.go, internal/artifacts/s3_keys.go
• Previous requirements: internal/artifacts/previous_requirements.go
• Tests: internal/artifacts/artifact_resolver_test.go, internal/artifacts/catalog_test.go, internal/artifacts/extraction_catalog_test.go, internal/artifacts/current_state_test.go, internal/artifacts/paths_model_test.go, internal/artifacts/previous_requirements_test.go