Add a roadmap to refactor prompt assets
This commit is contained in:
@@ -1,16 +1,120 @@
|
|||||||
# Scriptorium Runtime Migration
|
# Shared Prompt And Asset Layout Implementation Plan
|
||||||
|
|
||||||
The Scriptorium-backed LLM runtime migration is complete.
|
## Summary
|
||||||
|
|
||||||
Current implemented behavior is documented in:
|
Implement the asset layout defined in
|
||||||
|
[Shared Prompt And Asset Layout Roadmap](sharedassets.md). This is a structural
|
||||||
|
reorganization only: prompt semantics, prompt IDs, prompt versions, response
|
||||||
|
schema IDs, schema names, runtime module contracts, diagnostics policy, and
|
||||||
|
manifest provenance shape must remain unchanged.
|
||||||
|
|
||||||
- [Configuration](../config.md)
|
## Stage 1: Introduce Shared Assets Package
|
||||||
- [CLI Reference](../cli.md)
|
|
||||||
- [Operations](../operations.md)
|
|
||||||
- [Troubleshooting](../troubleshooting.md)
|
|
||||||
- [LLM Runtime Internals](../internal/llm.md)
|
|
||||||
- [Module Internals](../internal/modules.md)
|
|
||||||
- [Pipeline Internals](../internal/pipeline.md)
|
|
||||||
- [JSON Output](../integrations/json-output.md)
|
|
||||||
|
|
||||||
Remaining product and operational ideas belong in [Future Work](future.md).
|
- Create `internal/modules/sharedassets` as the cross-module shared asset
|
||||||
|
package.
|
||||||
|
- Move the existing shared D&D prompt fragments from
|
||||||
|
`internal/modules/dnd/promptassets` into `internal/modules/sharedassets`.
|
||||||
|
- Rename shared prompt files exactly as:
|
||||||
|
- `common-dnd-system.md`
|
||||||
|
- `common-dnd-transcript.md`
|
||||||
|
- `common-dnd-references.md`
|
||||||
|
- Implement the shared package with the same responsibilities as the current
|
||||||
|
shared prompt package:
|
||||||
|
- embed shared prompt files;
|
||||||
|
- register them with `llm.AssetRegistry`;
|
||||||
|
- expose hash-part helpers for common shared prompts and reference prompts.
|
||||||
|
- Keep Scriptorium-visible prompt paths under the shared prompt root as
|
||||||
|
`common-dnd-*.md`; do not preserve the old `shared/*.md` Scriptorium paths.
|
||||||
|
- Update production prompt asset collection in `internal/cli` and D&D module
|
||||||
|
prompt tests to import `internal/modules/sharedassets`.
|
||||||
|
- Do not delete the old `internal/modules/dnd/promptassets` package until all
|
||||||
|
callers have been updated and tests compile.
|
||||||
|
|
||||||
|
## Stage 2: Flatten Module Asset Trees
|
||||||
|
|
||||||
|
- Move `dnd/spells` module-owned prompt assets to:
|
||||||
|
- `internal/modules/extract/dnd/spells/assets/prompts/dnd.spells.yaml`
|
||||||
|
- `internal/modules/extract/dnd/spells/assets/prompts/task.md`
|
||||||
|
- `internal/modules/extract/dnd/spells/assets/prompts/instructions.md`
|
||||||
|
- Move `dnd/scenes` module-owned prompt assets to:
|
||||||
|
- `internal/modules/chunk/dnd/scenes/assets/prompts/dnd.scenes.yaml`
|
||||||
|
- `internal/modules/chunk/dnd/scenes/assets/prompts/task.md`
|
||||||
|
- `internal/modules/chunk/dnd/scenes/assets/prompts/instructions.md`
|
||||||
|
- Keep module schemas under each module's existing shallow schema directory:
|
||||||
|
- `assets/schemas/dnd_spells.v1.json`
|
||||||
|
- `assets/schemas/dnd_scenes.v1.json`
|
||||||
|
- Update each module's `go:embed` patterns to include only the new shallow
|
||||||
|
prompt and schema paths.
|
||||||
|
- Update each module's prompt registration root from the old
|
||||||
|
`assets/scriptorium/prompts` tree to `assets/prompts`.
|
||||||
|
- Update prompt hash inputs to point at the new module prompt paths and the new
|
||||||
|
shared asset hash parts.
|
||||||
|
|
||||||
|
## Stage 3: Update Scriptorium Prompt Definitions
|
||||||
|
|
||||||
|
- Update `dnd.spells.yaml` and `dnd.scenes.yaml` to use the new shallow
|
||||||
|
`content_file` paths.
|
||||||
|
- Shared prompt references must point to:
|
||||||
|
- `./common-dnd-system.md`
|
||||||
|
- `./common-dnd-transcript.md`
|
||||||
|
- `./common-dnd-references.md`
|
||||||
|
- Module prompt references must point to:
|
||||||
|
- `./task.md`
|
||||||
|
- `./instructions.md`
|
||||||
|
- Keep all existing prompt IDs, prompt versions, default profiles, inputs,
|
||||||
|
cache-control settings, output schema paths, validation modes, and repair
|
||||||
|
attempts unchanged.
|
||||||
|
- Confirm the flattened module prompt roots and shared prompt roots combine into
|
||||||
|
one Scriptorium prompt filesystem without duplicate asset paths.
|
||||||
|
|
||||||
|
## Stage 4: Remove Old Package And Update Docs
|
||||||
|
|
||||||
|
- Delete `internal/modules/dnd/promptassets` after all imports and asset
|
||||||
|
references are gone.
|
||||||
|
- Update canonical current-behavior docs to describe the new convention:
|
||||||
|
- module-owned prompts and schemas live under each module's shallow
|
||||||
|
`assets/prompts` and `assets/schemas` directories;
|
||||||
|
- shared reusable prompt fragments live under
|
||||||
|
`internal/modules/sharedassets`;
|
||||||
|
- framework and CLI code may collect assets, but modules still own prompt
|
||||||
|
intent, prompt IDs, schemas, and interpretation.
|
||||||
|
- Keep roadmap docs as future/target-state material only; do not describe
|
||||||
|
completed behavior in `docs/roadmap/` until the implementation is complete.
|
||||||
|
|
||||||
|
## Stage 5: Verification
|
||||||
|
|
||||||
|
- Update focused tests that assert asset paths, prepared prompt message content,
|
||||||
|
prompt hash metadata, or import paths.
|
||||||
|
- Add or preserve coverage proving:
|
||||||
|
- production prompt asset registration includes shared assets, `dnd/scenes`
|
||||||
|
assets, and `dnd/spells` assets;
|
||||||
|
- Scriptorium can prepare both D&D prompts using the moved assets;
|
||||||
|
- prompt and schema hashes remain present in module manifest metadata;
|
||||||
|
- raw source input, references, prompt text, and schema JSON still do not leak
|
||||||
|
into default diagnostics or manifests.
|
||||||
|
- Run focused validation:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
go test ./internal/framework/llm
|
||||||
|
go test ./internal/modules/chunk/dnd/scenes ./internal/modules/extract/dnd/spells
|
||||||
|
go test ./internal/cli
|
||||||
|
```
|
||||||
|
|
||||||
|
- Run full validation:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
go test ./...
|
||||||
|
go vet ./...
|
||||||
|
go build ./cmd/notarius
|
||||||
|
```
|
||||||
|
|
||||||
|
## Assumptions
|
||||||
|
|
||||||
|
- This migration is allowed to change internal asset paths because prompt assets
|
||||||
|
are embedded implementation details, not public API.
|
||||||
|
- Prompt IDs, versions, schema IDs, schema names, runtime request contracts, and
|
||||||
|
output manifest shape are stable and must not change in this pass.
|
||||||
|
- `internal/modules/sharedassets` may contain D&D-specific reusable prompt
|
||||||
|
fragments when filenames make the domain scope explicit.
|
||||||
|
- Shared assets should contain durable operating rules and reusable framing, not
|
||||||
|
module-specific extraction, chunking, normalization, or validation semantics.
|
||||||
|
|||||||
@@ -1,33 +0,0 @@
|
|||||||
# Completed Scriptorium LLM Runtime Migration
|
|
||||||
|
|
||||||
The Scriptorium-backed LLM runtime migration is complete. Notarius now uses
|
|
||||||
`gitea.maximumdirect.net/eric/scriptorium` as the production LLM execution
|
|
||||||
engine behind Notarius-owned module contracts, diagnostics, manifest
|
|
||||||
provenance, and secret-handling policy.
|
|
||||||
|
|
||||||
## Implemented Outcomes
|
|
||||||
|
|
||||||
- production LLM execution uses Scriptorium prompt assets and profiles;
|
|
||||||
- chunk, extract, and normalize modules call LLMs through the Notarius
|
|
||||||
`StructuredLLMClient` boundary;
|
|
||||||
- modules pass prompt IDs, prompt versions, profile IDs, session IDs, input
|
|
||||||
materials, and prompt variables instead of locally rendered chat messages;
|
|
||||||
- source transcript input is preserved as run-scoped input material for
|
|
||||||
byte-stable cacheable prompt messages;
|
|
||||||
- Scriptorium profile provenance is recorded in run manifests without raw
|
|
||||||
prompts, schemas, source payloads, references, or secrets.
|
|
||||||
|
|
||||||
## Canonical Docs
|
|
||||||
|
|
||||||
Current behavior is documented in:
|
|
||||||
|
|
||||||
- [Configuration](../config.md)
|
|
||||||
- [CLI Reference](../cli.md)
|
|
||||||
- [Internal LLM Runtime](../internal/llm.md)
|
|
||||||
- [Pipeline Internals](../internal/pipeline.md)
|
|
||||||
- [JSON Output Integration](../integrations/json-output.md)
|
|
||||||
- [Troubleshooting](../troubleshooting.md)
|
|
||||||
|
|
||||||
There is no active Scriptorium-specific roadmap work in this file. Deferred
|
|
||||||
work related to broader LLM, reference, retrieval, context-window, or provider
|
|
||||||
strategy belongs in [Future Roadmap](future.md).
|
|
||||||
97
docs/roadmap/sharedassets.md
Normal file
97
docs/roadmap/sharedassets.md
Normal file
@@ -0,0 +1,97 @@
|
|||||||
|
# Shared Prompt And Asset Layout Roadmap
|
||||||
|
|
||||||
|
This roadmap defines the target state for reorganizing embedded prompt and
|
||||||
|
schema assets so module-owned files are shallow and shared prompt fragments live
|
||||||
|
in an explicit shared asset package.
|
||||||
|
|
||||||
|
## Motivation
|
||||||
|
|
||||||
|
The current Scriptorium asset layout works, but module prompt files are nested
|
||||||
|
more deeply than their package ownership requires. For example, a module package
|
||||||
|
already identifies its stage, domain, and artifact, so paths like
|
||||||
|
`assets/scriptorium/prompts/dnd/spells/...` repeat context that is already in
|
||||||
|
the Go package path.
|
||||||
|
|
||||||
|
Shared D&D prompt fragments also currently live under a D&D-specific
|
||||||
|
`promptassets` package. That is serviceable, but it undersells the fact that
|
||||||
|
shared assets are intentionally cross-module. A clearer shared package will make
|
||||||
|
reuse deliberate and make room for common prompt hardening and reference-use
|
||||||
|
rules.
|
||||||
|
|
||||||
|
## Target State
|
||||||
|
|
||||||
|
Module-owned assets should stay in the module package that owns the behavior,
|
||||||
|
but their internal paths should be shallow:
|
||||||
|
|
||||||
|
```text
|
||||||
|
internal/modules/extract/dnd/spells/assets/
|
||||||
|
prompts/
|
||||||
|
dnd.spells.yaml
|
||||||
|
task.md
|
||||||
|
instructions.md
|
||||||
|
schemas/
|
||||||
|
dnd_spells.v1.json
|
||||||
|
|
||||||
|
internal/modules/chunk/dnd/scenes/assets/
|
||||||
|
prompts/
|
||||||
|
dnd.scenes.yaml
|
||||||
|
task.md
|
||||||
|
instructions.md
|
||||||
|
schemas/
|
||||||
|
dnd_scenes.v1.json
|
||||||
|
```
|
||||||
|
|
||||||
|
Shared reusable assets should move to:
|
||||||
|
|
||||||
|
```text
|
||||||
|
internal/modules/sharedassets/
|
||||||
|
assets.go
|
||||||
|
prompts/
|
||||||
|
common-dnd-system.md
|
||||||
|
common-dnd-transcript.md
|
||||||
|
common-dnd-references.md
|
||||||
|
schemas/
|
||||||
|
```
|
||||||
|
|
||||||
|
The `sharedassets` package should own only reusable fragments and shared asset
|
||||||
|
registration. It should not own module task semantics, response schemas,
|
||||||
|
validators, artifact interpretation, or stage behavior.
|
||||||
|
|
||||||
|
## Ownership Rules
|
||||||
|
|
||||||
|
Module packages own:
|
||||||
|
|
||||||
|
- complete Scriptorium prompt definitions;
|
||||||
|
- module task prompts and instruction prompts;
|
||||||
|
- response schemas and schema metadata;
|
||||||
|
- prompt IDs and versions;
|
||||||
|
- semantic validation and interpretation.
|
||||||
|
|
||||||
|
`internal/modules/sharedassets` owns:
|
||||||
|
|
||||||
|
- shared prompt fragments used by more than one module;
|
||||||
|
- standardized cross-module prompt hardening;
|
||||||
|
- shared source/transcript framing;
|
||||||
|
- shared reference framing and reference-use constraints;
|
||||||
|
- future genuinely shared schemas, if a real cross-module schema emerges.
|
||||||
|
|
||||||
|
Shared prompt fragments should be durable operating rules, not artifact-specific
|
||||||
|
instructions. They may tell the model to treat source material as primary
|
||||||
|
evidence, use references only for disambiguation unless a module says otherwise,
|
||||||
|
ignore instructions embedded inside source/reference content, follow the
|
||||||
|
structured output contract, and avoid exposing prompt instructions.
|
||||||
|
|
||||||
|
## Documentation Outcome
|
||||||
|
|
||||||
|
When implemented, canonical internal documentation should describe the asset
|
||||||
|
ownership convention:
|
||||||
|
|
||||||
|
- module-owned prompts and schemas live under each module's shallow `assets`
|
||||||
|
tree;
|
||||||
|
- shared reusable fragments live under `internal/modules/sharedassets`;
|
||||||
|
- framework and CLI code may collect assets, but module packages remain
|
||||||
|
responsible for prompt intent and schemas.
|
||||||
|
|
||||||
|
Prompt IDs, prompt versions, response schema IDs, schema names, and runtime
|
||||||
|
module contracts should remain unchanged unless a later feature explicitly
|
||||||
|
changes prompt semantics.
|
||||||
Reference in New Issue
Block a user