notarius/docs/roadmap/assets.md

# Centralized LLM Assets

## Purpose

Centralize Notarius's embedded LLM-facing assets in a top-level `assets/`
package so prompt declarations, prompt fragments, private structured-response
schemas, shared prompt text, and built-in LLM profiles are easy to find,
inspect, compare, and revise.

This is an ownership and repository-layout change, not a prompt redesign or a
change to any durable artifact contract. Modules continue to own all behavior
associated with their assets.

## Motivation

The D&D module family currently distributes its LLM assets across the packages
that consume them. Twelve D&D prompt consumers collectively embed 46 prompt
files. Their private response schemas, shared entity-reconciliation schema,
shared prompt fragments, and built-in `dnd-extraction` profile are likewise
spread across chunk, extract, normalize, shared, and registrar packages.

Co-location made each original module self-contained, but the growing family
now makes prompt inspection and cross-module editing unnecessarily difficult.
Prompt authors commonly need to:

- review all sibling prompts together;
- preserve exact shared bytes and cache-compatible message prefixes;
- compare declarations, instructions, tasks, and response schemas;
- identify duplicated or drifting language; and
- iterate on prompt content without navigating implementation packages.

A centralized, predictable asset tree makes those operations straightforward
while leaving executable behavior with the module packages.

## Decisions

### Top-Level Asset Package

All production LLM-facing assets will live beneath a repository-root
`assets/` directory. That directory will also be a Go package named `assets`
because Go embed patterns cannot embed files outside the embedding package's
directory tree.

The package will contain one Go source file, `assets/package.go`, unless a
future Go toolchain constraint makes that impractical. Its entire production
API will expose the embedded, read-only filesystem. The intended shape is
equivalent to:

```go
package assets

import (
	"embed"
	"io/fs"
)

//go:embed dnd generic
var embedded embed.FS

func FS() fs.FS { return embedded }
```

The exact embed patterns may reflect the final asset tree, but the package must
retain this minimal character. Adding a new asset domain may require extending
the embed directive; it must not require adding domain behavior to the package.

### Strict Data-Only Boundary

The `assets` package is a content container and dependency leaf. No business,
domain, framework, or PromptKit behavior may live there.

In particular, the package must not own:

- prompt IDs, schema IDs, schema names, or schema versions;
- prompt manifests, message order, cache controls, or input construction;
- module, schema, profile, or PromptKit registration;
- asset selection rules or module-to-path catalogs;
- parsing, validation, normalization, transformation, or hashing;
- checkpoint or fingerprint policy;
- fallback-profile precedence or model defaults; or
- imports of Notarius internal packages, PromptKit, or module packages.

It should expose no path-aware lookup helpers. Consumers obtain the root
read-only filesystem and use the standard library's `fs.Sub` to select the
smallest subtree they own. A module must not depend on an unrelated module's
asset subtree merely because both are available from the same embedded
filesystem.

The package's top-level location and exported accessor are an intentional
tradeoff in favor of human discoverability. They do not establish a supported
public extension API or transfer semantic ownership away from modules.

### Behavioral Ownership Remains With Modules

Each LLM-backed module continues to own:

- its prompt and response-schema identity constants;
- its ordered prompt-asset manifest;
- prompt input preparation and rendered message order;
- its response-schema definition and loading;
- prompt and schema registration with the shared LLM asset registry;
- output decoding and interpretation;
- its component fingerprint composition; and
- tests of its observable prompt and schema behavior.

The D&D shared package continues to own the manifest mechanism and the declared
set of reusable D&D prompt fragments. The D&D registrar continues to own family
composition and registration of the built-in fallback profile. The generic LLM
framework continues to own filesystem flattening, duplicate detection, schema
loading, PromptKit construction, and content hashing.

Physical storage is therefore centralized without creating a central package
that understands or composes every module.

### Asset Scope

The centralized tree includes embedded content whose direct purpose is to
configure or constrain an LLM interaction:

- PromptKit prompt declarations;
- module-owned prompt fragments;
- byte-identical shared prompt fragments;
- private JSON Schemas for structured LLM response envelopes;
- shared private response schemas such as D&D entity reconciliation; and
- built-in fallback LLM profiles.

The migration also includes the generic private LLM response schemas currently
embedded by the LLM framework. The framework may depend on the content-only
asset package for those generic files, but it must refer only to its own scoped
subtree and must not acquire D&D knowledge.

The following embedded files remain with their current owners:

- durable artifact schemas owned by D&D codec packages;
- framework-owned durable schemas such as chunk-map and evidence-context
  contracts;
- the D&D spell catalog and other domain data that is not itself a prompt,
  private LLM response schema, or built-in LLM profile;
- maintained configuration examples; and
- operator-supplied PromptKit profiles and other runtime files.

Private LLM response schemas must remain clearly distinguishable from durable
artifact schemas. Centralization must not turn a model transport envelope into
an external Notarius contract.

## Target Asset Tree

The target tree groups assets first by domain and then by conceptual module.
Directory names use readable kebab case; Go package names, module keys, prompt
IDs, and schema identities do not change to match the directory spelling.

```text
assets/
  package.go
  generic/
    schemas/
  dnd/
    shared/
      prompts/
    profiles/
    entity-reconciliation/
      schemas/
    scenes/
      prompts/
      schemas/
    spells/
      prompts/
      schemas/
    npcs/
      extract/
        prompts/
        schemas/
      normalize/
        prompts/
    combat-turns/
      prompts/
      schemas/
    enemy-events/
      prompts/
      schemas/
    item-events/
      prompts/
      schemas/
    npc-interactions/
      prompts/
      schemas/
    scene-descriptions/
      prompts/
      schemas/
    locations/
      extract/
        prompts/
        schemas/
      normalize/
        prompts/
    location-occurrences/
      prompts/
      schemas/
```

Each prompt-bearing leaf retains predictable filenames for its declaration,
task, instructions, and any lane-specific fragments. Existing filenames should
be preserved where practical so this migration does not create gratuitous
content churn. Shared content must have one physical source file rather than a
copy in each consuming subtree.

Future LLM-backed domains and modules follow the same taxonomy. A future module
with both extraction and LLM-backed normalization assets uses explicit
`extract/` and `normalize/` children; a module with only one LLM-backed
operation may keep its prompt and schema directories directly beneath the
module directory.

## Filesystem And Registration Model

Consumers use `assets.FS()` plus `fs.Sub` to obtain a domain-, module-, or
asset-class-specific filesystem. Existing manifests continue to enumerate the
exact files used by each prompt. Registration continues through
`llm.AssetRegistry`, which presents PromptKit with its existing flattened
prompt, schema, and fallback-profile filesystems.

The virtual PromptKit layout is stable even though repository source paths
move. In particular:

- prompt IDs and declaration names remain unchanged;
- schema keys, IDs, versions, names, and registered filenames remain unchanged;
- fallback profile IDs, contents, and precedence remain unchanged;
- shared fragments remain mounted at the paths expected by existing prompt
  declarations; and
- duplicate-path and missing-file failures continue to occur during
  preparation.

No consumer receives a precomposed global prompt catalog from the `assets`
package. Module manifests and the D&D registrar remain the explicit composition
points.

## Prompt Caching And Fingerprints

The migration must preserve every prompt's rendered roles, bytes, input
placement, message order, and cache-control metadata. Merely moving source
files must not reduce backend prompt-cache compatibility or alter model-visible
content.

Each component fingerprint must remain scoped to the exact assets declared by
that component. An edit to one lane must not invalidate checkpoints belonging
to unrelated lanes. The implementation must not replace manifest-scoped
hashing with a digest of the complete root asset filesystem.

Current asset hashes include source paths as well as bytes, so moving the files
may produce a one-time checkpoint fingerprint change for affected components.
That safe invalidation is acceptable. Preserving old internal fingerprints is
not a reason to retain duplicate files, compatibility shims, or legacy asset
paths. After migration, changing an unrelated asset must not affect a
component's fingerprint.

## Architectural Documentation

Central storage refines the accepted domain-packaging decision in ADR-0004,
which currently says domain-specific prompt fragments and schemas live within
the domain implementation tree. Implementation must add a new ADR recording
the content-only root package, module ownership rules, alternatives, and
consequences, and mark only the conflicting asset-co-location portion of
ADR-0004 as superseded. Its remaining domain-first module packaging decision
continues to apply.

Once implemented, the canonical architecture and internal documentation must
describe current behavior:

- architecture documents the data-only asset-package boundary and dependency
  direction;
- LLM internals document embedding, scoped filesystem consumption,
  registration, and private-schema ownership;
- D&D internals document the domain asset taxonomy, shared-fragment ownership,
  manifest-scoped fingerprints, and the convention for new modules; and
- the internal component overview links to the focused owner without
  duplicating volatile paths.

The roadmap remains the only documentation of this unimplemented target until
the migration lands.

## Work Scope

Completing this feature requires:

- creating the top-level asset package and target directory structure;
- moving all in-scope files without editing their semantic content;
- replacing package-local `embed.FS` declarations for migrated assets with
  scoped views of the central filesystem;
- updating module manifests, response-schema loaders, shared-fragment
  resolution, generic schema loading, fallback-profile registration, and asset
  hashes to use those scoped views;
- removing obsolete module-local prompt, private-schema, and profile asset
  directories and their embedding files;
- preserving module-local behavioral declarations and registration methods;
- updating tests and documentation at their durable ownership boundaries; and
- recording the architectural decision described above.

## Verification Expectations

Verification should protect behavior and architectural boundaries rather than
freeze prompt prose or count files. The completed migration must demonstrate
that:

- all registered prompts and private response schemas prepare successfully;
- all built-in fallback profiles remain available with unchanged effective
  configuration;
- rendered prompt messages and cache controls are unchanged for representative
  extraction, chunking, and normalization prompts;
- shared prompt fragments come from one physical source and render identically
  wherever reused;
- schema loading retains the existing identities and content digests;
- duplicate, missing, or malformed assets still fail at the established
  preparation boundary;
- component fingerprints use only their declared assets;
- D&D family and production composition tests pass;
- no package-local migrated copies remain; and
- `assets/` contains only the single minimal Go source file and in-scope data
  files, with no business logic or internal/PromptKit dependencies.

Tests must follow the testing policy: do not add exact prompt-prose snapshots,
asset-count assertions, source-path snapshots, or other change-detector tests.
Use existing preparation, rendering, registration, schema, fingerprint, and
composition boundaries to protect meaningful behavior.

Repository-wide validation includes:

```sh
go test ./...
go vet ./...
go build ./cmd/notarius
```

## Non-Goals

This work does not:

- rewrite, consolidate, or otherwise tune prompt content;
- change prompt message order, caching policy, or profile defaults;
- change private or durable schema shapes;
- move durable schemas or the spell catalog into the root package;
- add runtime prompt overrides or live filesystem reloading;
- expose asset paths as configuration or a supported public API;
- replace the existing LLM asset registry or PromptKit filesystem boundary;
- centralize module behavior, registration, or identity constants; or
- preserve reuse of checkpoints created with the pre-migration internal asset
  paths.

An optional development-time filesystem overlay may be considered separately
if prompt iteration without rebuilding becomes valuable. It requires distinct
precedence, provenance, fingerprint, reproducibility, and security decisions
and is not implied by this repository-layout change.

## Target End State

Prompt authors can inspect every embedded LLM interaction asset from one
predictable top-level tree. Reusable prompt content has one physical owner, and
private LLM schemas are visibly separated from durable artifact contracts.

The `assets` Go package consists only of `package.go`, embeds the content, and
exposes one read-only filesystem accessor. It contains no application logic and
has no Notarius-internal or PromptKit dependencies. Modules select scoped
subtrees and retain complete ownership of prompt semantics, schema identity,
registration, interpretation, and fingerprints. The framework remains
domain-neutral, the D&D registrar remains the family composition root, and all
observable prompt, schema, profile, and pipeline behavior is unchanged apart
from the accepted one-time invalidation of checkpoints whose fingerprints
include moved source paths.