Added shared helper for module prompt FS
This commit is contained in:
@@ -1,120 +1,19 @@
|
|||||||
# Shared Prompt And Asset Layout Implementation Plan
|
# Completed Shared Asset Layout Follow-Up
|
||||||
|
|
||||||
## Summary
|
The shared prompt and asset layout reorganization is complete.
|
||||||
|
|
||||||
Implement the asset layout defined in
|
Implemented behavior is documented in canonical current-behavior docs:
|
||||||
[Shared Prompt And Asset Layout Roadmap](sharedassets.md). This is a structural
|
|
||||||
reorganization only: prompt semantics, prompt IDs, prompt versions, response
|
|
||||||
schema IDs, schema names, runtime module contracts, diagnostics policy, and
|
|
||||||
manifest provenance shape must remain unchanged.
|
|
||||||
|
|
||||||
## Stage 1: Introduce Shared Assets Package
|
- [Internal Overview](../internal/overview.md)
|
||||||
|
- [Internal LLM Runtime](../internal/llm.md)
|
||||||
|
- [Internal Modules](../internal/modules.md)
|
||||||
|
|
||||||
- Create `internal/modules/sharedassets` as the cross-module shared asset
|
## Implemented Outcomes
|
||||||
package.
|
|
||||||
- Move the existing shared D&D prompt fragments from
|
|
||||||
`internal/modules/dnd/promptassets` into `internal/modules/sharedassets`.
|
|
||||||
- Rename shared prompt files exactly as:
|
|
||||||
- `common-dnd-system.md`
|
|
||||||
- `common-dnd-transcript.md`
|
|
||||||
- `common-dnd-references.md`
|
|
||||||
- Implement the shared package with the same responsibilities as the current
|
|
||||||
shared prompt package:
|
|
||||||
- embed shared prompt files;
|
|
||||||
- register them with `llm.AssetRegistry`;
|
|
||||||
- expose hash-part helpers for common shared prompts and reference prompts.
|
|
||||||
- Keep Scriptorium-visible prompt paths under the shared prompt root as
|
|
||||||
`common-dnd-*.md`; do not preserve the old `shared/*.md` Scriptorium paths.
|
|
||||||
- Update production prompt asset collection in `internal/cli` and D&D module
|
|
||||||
prompt tests to import `internal/modules/sharedassets`.
|
|
||||||
- Do not delete the old `internal/modules/dnd/promptassets` package until all
|
|
||||||
callers have been updated and tests compile.
|
|
||||||
|
|
||||||
## Stage 2: Flatten Module Asset Trees
|
- module-owned prompts and schemas use shallow module-local asset trees;
|
||||||
|
- reusable D&D prompt fragments live under `internal/modules/sharedassets`;
|
||||||
|
- shared prompt filesystem composition is centralized in `sharedassets`;
|
||||||
|
- D&D scene and spell modules no longer carry duplicated prompt filesystem
|
||||||
|
helper implementations.
|
||||||
|
|
||||||
- Move `dnd/spells` module-owned prompt assets to:
|
Remaining future work belongs in [Future Work](future.md).
|
||||||
- `internal/modules/extract/dnd/spells/assets/prompts/dnd.spells.yaml`
|
|
||||||
- `internal/modules/extract/dnd/spells/assets/prompts/task.md`
|
|
||||||
- `internal/modules/extract/dnd/spells/assets/prompts/instructions.md`
|
|
||||||
- Move `dnd/scenes` module-owned prompt assets to:
|
|
||||||
- `internal/modules/chunk/dnd/scenes/assets/prompts/dnd.scenes.yaml`
|
|
||||||
- `internal/modules/chunk/dnd/scenes/assets/prompts/task.md`
|
|
||||||
- `internal/modules/chunk/dnd/scenes/assets/prompts/instructions.md`
|
|
||||||
- Keep module schemas under each module's existing shallow schema directory:
|
|
||||||
- `assets/schemas/dnd_spells.v1.json`
|
|
||||||
- `assets/schemas/dnd_scenes.v1.json`
|
|
||||||
- Update each module's `go:embed` patterns to include only the new shallow
|
|
||||||
prompt and schema paths.
|
|
||||||
- Update each module's prompt registration root from the old
|
|
||||||
`assets/scriptorium/prompts` tree to `assets/prompts`.
|
|
||||||
- Update prompt hash inputs to point at the new module prompt paths and the new
|
|
||||||
shared asset hash parts.
|
|
||||||
|
|
||||||
## Stage 3: Update Scriptorium Prompt Definitions
|
|
||||||
|
|
||||||
- Update `dnd.spells.yaml` and `dnd.scenes.yaml` to use the new shallow
|
|
||||||
`content_file` paths.
|
|
||||||
- Shared prompt references must point to:
|
|
||||||
- `./common-dnd-system.md`
|
|
||||||
- `./common-dnd-transcript.md`
|
|
||||||
- `./common-dnd-references.md`
|
|
||||||
- Module prompt references must point to:
|
|
||||||
- `./task.md`
|
|
||||||
- `./instructions.md`
|
|
||||||
- Keep all existing prompt IDs, prompt versions, default profiles, inputs,
|
|
||||||
cache-control settings, output schema paths, validation modes, and repair
|
|
||||||
attempts unchanged.
|
|
||||||
- Confirm the flattened module prompt roots and shared prompt roots combine into
|
|
||||||
one Scriptorium prompt filesystem without duplicate asset paths.
|
|
||||||
|
|
||||||
## Stage 4: Remove Old Package And Update Docs
|
|
||||||
|
|
||||||
- Delete `internal/modules/dnd/promptassets` after all imports and asset
|
|
||||||
references are gone.
|
|
||||||
- Update canonical current-behavior docs to describe the new convention:
|
|
||||||
- module-owned prompts and schemas live under each module's shallow
|
|
||||||
`assets/prompts` and `assets/schemas` directories;
|
|
||||||
- shared reusable prompt fragments live under
|
|
||||||
`internal/modules/sharedassets`;
|
|
||||||
- framework and CLI code may collect assets, but modules still own prompt
|
|
||||||
intent, prompt IDs, schemas, and interpretation.
|
|
||||||
- Keep roadmap docs as future/target-state material only; do not describe
|
|
||||||
completed behavior in `docs/roadmap/` until the implementation is complete.
|
|
||||||
|
|
||||||
## Stage 5: Verification
|
|
||||||
|
|
||||||
- Update focused tests that assert asset paths, prepared prompt message content,
|
|
||||||
prompt hash metadata, or import paths.
|
|
||||||
- Add or preserve coverage proving:
|
|
||||||
- production prompt asset registration includes shared assets, `dnd/scenes`
|
|
||||||
assets, and `dnd/spells` assets;
|
|
||||||
- Scriptorium can prepare both D&D prompts using the moved assets;
|
|
||||||
- prompt and schema hashes remain present in module manifest metadata;
|
|
||||||
- raw source input, references, prompt text, and schema JSON still do not leak
|
|
||||||
into default diagnostics or manifests.
|
|
||||||
- Run focused validation:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
go test ./internal/framework/llm
|
|
||||||
go test ./internal/modules/chunk/dnd/scenes ./internal/modules/extract/dnd/spells
|
|
||||||
go test ./internal/cli
|
|
||||||
```
|
|
||||||
|
|
||||||
- Run full validation:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
go test ./...
|
|
||||||
go vet ./...
|
|
||||||
go build ./cmd/notarius
|
|
||||||
```
|
|
||||||
|
|
||||||
## Assumptions
|
|
||||||
|
|
||||||
- This migration is allowed to change internal asset paths because prompt assets
|
|
||||||
are embedded implementation details, not public API.
|
|
||||||
- Prompt IDs, versions, schema IDs, schema names, runtime request contracts, and
|
|
||||||
output manifest shape are stable and must not change in this pass.
|
|
||||||
- `internal/modules/sharedassets` may contain D&D-specific reusable prompt
|
|
||||||
fragments when filenames make the domain scope explicit.
|
|
||||||
- Shared assets should contain durable operating rules and reusable framing, not
|
|
||||||
module-specific extraction, chunking, normalization, or validation semantics.
|
|
||||||
|
|||||||
@@ -1,47 +1,15 @@
|
|||||||
# Shared Prompt And Asset Layout Roadmap
|
# Completed Shared Prompt And Asset Layout
|
||||||
|
|
||||||
This roadmap defines the target state for reorganizing embedded prompt and
|
The embedded prompt and schema asset layout has been reorganized so module-owned
|
||||||
schema assets so module-owned files are shallow and shared prompt fragments live
|
files are shallow and shared prompt fragments live in an explicit shared asset
|
||||||
in an explicit shared asset package.
|
package.
|
||||||
|
|
||||||
## Motivation
|
## Outcome
|
||||||
|
|
||||||
The current Scriptorium asset layout works, but module prompt files are nested
|
Module-owned assets stay in the module package that owns the behavior, with
|
||||||
more deeply than their package ownership requires. For example, a module package
|
shallow internal paths:
|
||||||
already identifies its stage, domain, and artifact, so paths like
|
|
||||||
`assets/scriptorium/prompts/dnd/spells/...` repeat context that is already in
|
|
||||||
the Go package path.
|
|
||||||
|
|
||||||
Shared D&D prompt fragments also currently live under a D&D-specific
|
Shared reusable assets live under:
|
||||||
`promptassets` package. That is serviceable, but it undersells the fact that
|
|
||||||
shared assets are intentionally cross-module. A clearer shared package will make
|
|
||||||
reuse deliberate and make room for common prompt hardening and reference-use
|
|
||||||
rules.
|
|
||||||
|
|
||||||
## Target State
|
|
||||||
|
|
||||||
Module-owned assets should stay in the module package that owns the behavior,
|
|
||||||
but their internal paths should be shallow:
|
|
||||||
|
|
||||||
```text
|
|
||||||
internal/modules/extract/dnd/spells/assets/
|
|
||||||
prompts/
|
|
||||||
dnd.spells.yaml
|
|
||||||
task.md
|
|
||||||
instructions.md
|
|
||||||
schemas/
|
|
||||||
dnd_spells.v1.json
|
|
||||||
|
|
||||||
internal/modules/chunk/dnd/scenes/assets/
|
|
||||||
prompts/
|
|
||||||
dnd.scenes.yaml
|
|
||||||
task.md
|
|
||||||
instructions.md
|
|
||||||
schemas/
|
|
||||||
dnd_scenes.v1.json
|
|
||||||
```
|
|
||||||
|
|
||||||
Shared reusable assets should move to:
|
|
||||||
|
|
||||||
```text
|
```text
|
||||||
internal/modules/sharedassets/
|
internal/modules/sharedassets/
|
||||||
@@ -81,17 +49,13 @@ evidence, use references only for disambiguation unless a module says otherwise,
|
|||||||
ignore instructions embedded inside source/reference content, follow the
|
ignore instructions embedded inside source/reference content, follow the
|
||||||
structured output contract, and avoid exposing prompt instructions.
|
structured output contract, and avoid exposing prompt instructions.
|
||||||
|
|
||||||
## Documentation Outcome
|
## Canonical Docs
|
||||||
|
|
||||||
When implemented, canonical internal documentation should describe the asset
|
Current behavior is documented in:
|
||||||
ownership convention:
|
|
||||||
|
|
||||||
- module-owned prompts and schemas live under each module's shallow `assets`
|
- [Internal Overview](../internal/overview.md)
|
||||||
tree;
|
- [Internal LLM Runtime](../internal/llm.md)
|
||||||
- shared reusable fragments live under `internal/modules/sharedassets`;
|
- [Internal Modules](../internal/modules.md)
|
||||||
- framework and CLI code may collect assets, but module packages remain
|
|
||||||
responsible for prompt intent and schemas.
|
|
||||||
|
|
||||||
Prompt IDs, prompt versions, response schema IDs, schema names, and runtime
|
Prompt IDs, prompt versions, response schema IDs, schema names, and runtime
|
||||||
module contracts should remain unchanged unless a later feature explicitly
|
module contracts remain unchanged by this reorganization.
|
||||||
changes prompt semantics.
|
|
||||||
|
|||||||
@@ -3,7 +3,6 @@ package scenes
|
|||||||
import (
|
import (
|
||||||
"bytes"
|
"bytes"
|
||||||
"fmt"
|
"fmt"
|
||||||
"io/fs"
|
|
||||||
"sort"
|
"sort"
|
||||||
"sync"
|
"sync"
|
||||||
|
|
||||||
@@ -15,11 +14,11 @@ import (
|
|||||||
const scriptoriumPromptRoot = "assets/prompts"
|
const scriptoriumPromptRoot = "assets/prompts"
|
||||||
|
|
||||||
func RegisterPromptAssets(registry *llm.AssetRegistry) error {
|
func RegisterPromptAssets(registry *llm.AssetRegistry) error {
|
||||||
sharedPromptFS, err := promptSharedFS()
|
promptFS, err := sharedassets.ModulePromptFS("dnd.scenes", embeddedAssets, []sharedassets.ModulePromptFile{
|
||||||
if err != nil {
|
{Name: "dnd.scenes.yaml", Path: "assets/prompts/dnd.scenes.yaml"},
|
||||||
return fmt.Errorf("prepare shared prompt assets: %w", err)
|
{Name: "task.md", Path: "assets/prompts/task.md"},
|
||||||
}
|
{Name: "instructions.md", Path: "assets/prompts/instructions.md"},
|
||||||
promptFS, err := modulePromptFS(sharedPromptFS)
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return fmt.Errorf("prepare scene prompt assets: %w", err)
|
return fmt.Errorf("prepare scene prompt assets: %w", err)
|
||||||
}
|
}
|
||||||
@@ -29,14 +28,6 @@ func RegisterPromptAssets(registry *llm.AssetRegistry) error {
|
|||||||
return registry.RegisterSchemaFS(embeddedAssets, "assets/schemas")
|
return registry.RegisterSchemaFS(embeddedAssets, "assets/schemas")
|
||||||
}
|
}
|
||||||
|
|
||||||
func promptSharedFS() (fs.FS, error) {
|
|
||||||
registry := llm.NewAssetRegistry()
|
|
||||||
if err := sharedassets.Register(registry); err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
return registry.PromptFS()
|
|
||||||
}
|
|
||||||
|
|
||||||
func promptInputs(req contracts.ChunkRequest) contracts.LLMInputSet {
|
func promptInputs(req contracts.ChunkRequest) contracts.LLMInputSet {
|
||||||
return contracts.LLMInputSet{
|
return contracts.LLMInputSet{
|
||||||
"transcript": transcriptPromptInput(req.SourceInput),
|
"transcript": transcriptPromptInput(req.SourceInput),
|
||||||
|
|||||||
@@ -1,194 +0,0 @@
|
|||||||
package spells
|
|
||||||
|
|
||||||
import (
|
|
||||||
"bytes"
|
|
||||||
"fmt"
|
|
||||||
"io"
|
|
||||||
"io/fs"
|
|
||||||
"path"
|
|
||||||
"sort"
|
|
||||||
"strings"
|
|
||||||
"time"
|
|
||||||
)
|
|
||||||
|
|
||||||
func modulePromptFS(shared fs.FS) (fs.FS, error) {
|
|
||||||
assetFS, err := promptMapFSFromEmbedded(map[string]string{
|
|
||||||
"assets/prompts/dnd.spells/dnd.spells.yaml": "assets/prompts/dnd.spells.yaml",
|
|
||||||
"assets/prompts/dnd.spells/task.md": "assets/prompts/task.md",
|
|
||||||
"assets/prompts/dnd.spells/instructions.md": "assets/prompts/instructions.md",
|
|
||||||
})
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
assets := assetFS.(promptMapFS)
|
|
||||||
if shared == nil {
|
|
||||||
return assets, nil
|
|
||||||
}
|
|
||||||
for _, name := range []string{
|
|
||||||
"common-dnd-system.md",
|
|
||||||
"common-dnd-transcript.md",
|
|
||||||
"common-dnd-references.md",
|
|
||||||
} {
|
|
||||||
data, err := fs.ReadFile(shared, name)
|
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf("read shared prompt asset %s: %w", name, err)
|
|
||||||
}
|
|
||||||
assets["assets/prompts/dnd.spells/"+name] = append([]byte(nil), data...)
|
|
||||||
}
|
|
||||||
return assets, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func promptMapFSFromEmbedded(files map[string]string) (fs.FS, error) {
|
|
||||||
assets := make(promptMapFS, len(files))
|
|
||||||
for virtualPath, embeddedPath := range files {
|
|
||||||
data, err := fs.ReadFile(embeddedAssets, embeddedPath)
|
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf("read prompt asset %s: %w", embeddedPath, err)
|
|
||||||
}
|
|
||||||
assets[virtualPath] = append([]byte(nil), data...)
|
|
||||||
}
|
|
||||||
return assets, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
type promptMapFS map[string][]byte
|
|
||||||
|
|
||||||
func (m promptMapFS) Open(name string) (fs.File, error) {
|
|
||||||
cleaned, err := cleanPromptPath(name)
|
|
||||||
if err != nil {
|
|
||||||
return nil, &fs.PathError{Op: "open", Path: name, Err: err}
|
|
||||||
}
|
|
||||||
if data, ok := m[cleaned]; ok {
|
|
||||||
return &promptFile{
|
|
||||||
reader: bytes.NewReader(data),
|
|
||||||
info: promptFileInfo{name: path.Base(cleaned), size: int64(len(data))},
|
|
||||||
}, nil
|
|
||||||
}
|
|
||||||
entries := m.dirEntries(cleaned)
|
|
||||||
if entries != nil {
|
|
||||||
return &promptDir{name: path.Base(cleaned), entries: entries}, nil
|
|
||||||
}
|
|
||||||
return nil, &fs.PathError{Op: "open", Path: name, Err: fs.ErrNotExist}
|
|
||||||
}
|
|
||||||
|
|
||||||
func (m promptMapFS) ReadDir(name string) ([]fs.DirEntry, error) {
|
|
||||||
cleaned, err := cleanPromptPath(name)
|
|
||||||
if err != nil {
|
|
||||||
return nil, &fs.PathError{Op: "readdir", Path: name, Err: err}
|
|
||||||
}
|
|
||||||
entries := m.dirEntries(cleaned)
|
|
||||||
if entries == nil {
|
|
||||||
return nil, &fs.PathError{Op: "readdir", Path: name, Err: fs.ErrNotExist}
|
|
||||||
}
|
|
||||||
return entries, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func (m promptMapFS) dirEntries(dir string) []fs.DirEntry {
|
|
||||||
children := map[string]promptDirEntry{}
|
|
||||||
prefix := ""
|
|
||||||
if dir != "." {
|
|
||||||
prefix = dir + "/"
|
|
||||||
}
|
|
||||||
for name, data := range m {
|
|
||||||
if !strings.HasPrefix(name, prefix) {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
rest := strings.TrimPrefix(name, prefix)
|
|
||||||
if rest == "" {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
childName, _, hasSlash := strings.Cut(rest, "/")
|
|
||||||
entry := promptDirEntry{name: childName, dir: hasSlash}
|
|
||||||
if !hasSlash {
|
|
||||||
entry.size = int64(len(data))
|
|
||||||
}
|
|
||||||
children[childName] = entry
|
|
||||||
}
|
|
||||||
if len(children) == 0 {
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
names := make([]string, 0, len(children))
|
|
||||||
for name := range children {
|
|
||||||
names = append(names, name)
|
|
||||||
}
|
|
||||||
sort.Strings(names)
|
|
||||||
entries := make([]fs.DirEntry, 0, len(names))
|
|
||||||
for _, name := range names {
|
|
||||||
entries = append(entries, children[name])
|
|
||||||
}
|
|
||||||
return entries
|
|
||||||
}
|
|
||||||
|
|
||||||
func cleanPromptPath(name string) (string, error) {
|
|
||||||
trimmed := strings.TrimSpace(name)
|
|
||||||
if trimmed == "" {
|
|
||||||
return "", fmt.Errorf("path must not be empty")
|
|
||||||
}
|
|
||||||
cleaned := path.Clean(strings.TrimPrefix(trimmed, "/"))
|
|
||||||
if cleaned == "." || !fs.ValidPath(cleaned) {
|
|
||||||
return "", fmt.Errorf("invalid path %q", name)
|
|
||||||
}
|
|
||||||
return cleaned, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
type promptFile struct {
|
|
||||||
reader *bytes.Reader
|
|
||||||
info promptFileInfo
|
|
||||||
}
|
|
||||||
|
|
||||||
func (f *promptFile) Stat() (fs.FileInfo, error) { return f.info, nil }
|
|
||||||
func (f *promptFile) Read(p []byte) (int, error) { return f.reader.Read(p) }
|
|
||||||
func (f *promptFile) Close() error { return nil }
|
|
||||||
|
|
||||||
type promptDir struct {
|
|
||||||
name string
|
|
||||||
offset int
|
|
||||||
entries []fs.DirEntry
|
|
||||||
}
|
|
||||||
|
|
||||||
func (d *promptDir) Stat() (fs.FileInfo, error) { return promptFileInfo{name: d.name, dir: true}, nil }
|
|
||||||
func (d *promptDir) Read([]byte) (int, error) { return 0, io.EOF }
|
|
||||||
func (d *promptDir) Close() error { return nil }
|
|
||||||
func (d *promptDir) ReadDir(n int) ([]fs.DirEntry, error) {
|
|
||||||
if d.offset >= len(d.entries) {
|
|
||||||
return nil, io.EOF
|
|
||||||
}
|
|
||||||
end := len(d.entries)
|
|
||||||
if n > 0 && d.offset+n < end {
|
|
||||||
end = d.offset + n
|
|
||||||
}
|
|
||||||
out := append([]fs.DirEntry(nil), d.entries[d.offset:end]...)
|
|
||||||
d.offset = end
|
|
||||||
return out, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
type promptDirEntry struct {
|
|
||||||
name string
|
|
||||||
dir bool
|
|
||||||
size int64
|
|
||||||
}
|
|
||||||
|
|
||||||
func (e promptDirEntry) Name() string { return e.name }
|
|
||||||
func (e promptDirEntry) IsDir() bool { return e.dir }
|
|
||||||
func (e promptDirEntry) Type() fs.FileMode { return e.fileInfoMode().Type() }
|
|
||||||
func (e promptDirEntry) Info() (fs.FileInfo, error) {
|
|
||||||
return promptFileInfo{name: e.name, dir: e.dir, size: e.size}, nil
|
|
||||||
}
|
|
||||||
func (e promptDirEntry) fileInfoMode() fs.FileMode {
|
|
||||||
if e.dir {
|
|
||||||
return fs.ModeDir | 0o555
|
|
||||||
}
|
|
||||||
return 0o444
|
|
||||||
}
|
|
||||||
|
|
||||||
type promptFileInfo struct {
|
|
||||||
name string
|
|
||||||
dir bool
|
|
||||||
size int64
|
|
||||||
}
|
|
||||||
|
|
||||||
func (i promptFileInfo) Name() string { return i.name }
|
|
||||||
func (i promptFileInfo) Size() int64 { return i.size }
|
|
||||||
func (i promptFileInfo) Mode() fs.FileMode { return promptDirEntry{dir: i.dir}.fileInfoMode() }
|
|
||||||
func (i promptFileInfo) ModTime() time.Time { return time.Time{} }
|
|
||||||
func (i promptFileInfo) IsDir() bool { return i.dir }
|
|
||||||
func (i promptFileInfo) Sys() any { return nil }
|
|
||||||
@@ -3,7 +3,6 @@ package spells
|
|||||||
import (
|
import (
|
||||||
"bytes"
|
"bytes"
|
||||||
"fmt"
|
"fmt"
|
||||||
"io/fs"
|
|
||||||
"sort"
|
"sort"
|
||||||
"sync"
|
"sync"
|
||||||
|
|
||||||
@@ -15,11 +14,11 @@ import (
|
|||||||
const scriptoriumPromptRoot = "assets/prompts"
|
const scriptoriumPromptRoot = "assets/prompts"
|
||||||
|
|
||||||
func RegisterPromptAssets(registry *llm.AssetRegistry) error {
|
func RegisterPromptAssets(registry *llm.AssetRegistry) error {
|
||||||
sharedPromptFS, err := promptSharedFS()
|
promptFS, err := sharedassets.ModulePromptFS("dnd.spells", embeddedAssets, []sharedassets.ModulePromptFile{
|
||||||
if err != nil {
|
{Name: "dnd.spells.yaml", Path: "assets/prompts/dnd.spells.yaml"},
|
||||||
return fmt.Errorf("prepare shared prompt assets: %w", err)
|
{Name: "task.md", Path: "assets/prompts/task.md"},
|
||||||
}
|
{Name: "instructions.md", Path: "assets/prompts/instructions.md"},
|
||||||
promptFS, err := modulePromptFS(sharedPromptFS)
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return fmt.Errorf("prepare spell prompt assets: %w", err)
|
return fmt.Errorf("prepare spell prompt assets: %w", err)
|
||||||
}
|
}
|
||||||
@@ -29,14 +28,6 @@ func RegisterPromptAssets(registry *llm.AssetRegistry) error {
|
|||||||
return registry.RegisterSchemaFS(embeddedAssets, "assets/schemas")
|
return registry.RegisterSchemaFS(embeddedAssets, "assets/schemas")
|
||||||
}
|
}
|
||||||
|
|
||||||
func promptSharedFS() (fs.FS, error) {
|
|
||||||
registry := llm.NewAssetRegistry()
|
|
||||||
if err := sharedassets.Register(registry); err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
return registry.PromptFS()
|
|
||||||
}
|
|
||||||
|
|
||||||
func promptInputs(req contracts.ExtractionRequest) contracts.LLMInputSet {
|
func promptInputs(req contracts.ExtractionRequest) contracts.LLMInputSet {
|
||||||
return contracts.LLMInputSet{
|
return contracts.LLMInputSet{
|
||||||
"transcript": transcriptPromptInput(req.SourceInput),
|
"transcript": transcriptPromptInput(req.SourceInput),
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
package scenes
|
package sharedassets
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"bytes"
|
"bytes"
|
||||||
@@ -11,49 +11,70 @@ import (
|
|||||||
"time"
|
"time"
|
||||||
)
|
)
|
||||||
|
|
||||||
func modulePromptFS(shared fs.FS) (fs.FS, error) {
|
// ModulePromptFile maps a module-owned embedded prompt file into the
|
||||||
assetFS, err := promptMapFSFromEmbedded(map[string]string{
|
// Scriptorium-visible module prompt directory.
|
||||||
"assets/prompts/dnd.scenes/dnd.scenes.yaml": "assets/prompts/dnd.scenes.yaml",
|
type ModulePromptFile struct {
|
||||||
"assets/prompts/dnd.scenes/task.md": "assets/prompts/task.md",
|
Name string
|
||||||
"assets/prompts/dnd.scenes/instructions.md": "assets/prompts/instructions.md",
|
Path string
|
||||||
})
|
}
|
||||||
|
|
||||||
|
// ModulePromptFS builds a prompt filesystem for a module directory from
|
||||||
|
// module-owned prompt files plus the common D&D shared prompt files.
|
||||||
|
func ModulePromptFS(moduleDir string, moduleFS fs.FS, files []ModulePromptFile) (fs.FS, error) {
|
||||||
|
return modulePromptFS(moduleDir, moduleFS, files, embeddedAssets)
|
||||||
|
}
|
||||||
|
|
||||||
|
func modulePromptFS(moduleDir string, moduleFS fs.FS, files []ModulePromptFile, sharedFS fs.FS) (fs.FS, error) {
|
||||||
|
cleanModuleDir, err := cleanPromptPath(moduleDir)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, fmt.Errorf("module prompt directory: %w", err)
|
||||||
}
|
}
|
||||||
assets := assetFS.(promptMapFS)
|
if moduleFS == nil {
|
||||||
if shared == nil {
|
return nil, fmt.Errorf("module prompt filesystem must not be nil")
|
||||||
return assets, nil
|
|
||||||
}
|
}
|
||||||
for _, name := range []string{
|
if sharedFS == nil {
|
||||||
"common-dnd-system.md",
|
return nil, fmt.Errorf("shared prompt filesystem must not be nil")
|
||||||
"common-dnd-transcript.md",
|
}
|
||||||
"common-dnd-references.md",
|
assets := make(promptMapFS, len(files)+len(commonDNDPromptFiles))
|
||||||
} {
|
for _, file := range files {
|
||||||
data, err := fs.ReadFile(shared, name)
|
name, err := cleanPromptPath(file.Name)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("module prompt file name %q: %w", file.Name, err)
|
||||||
|
}
|
||||||
|
if strings.Contains(name, "/") {
|
||||||
|
return nil, fmt.Errorf("module prompt file name %q must not contain path separators", file.Name)
|
||||||
|
}
|
||||||
|
filePath, err := cleanPromptPath(file.Path)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("module prompt file path %q: %w", file.Path, err)
|
||||||
|
}
|
||||||
|
data, err := fs.ReadFile(moduleFS, filePath)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("read module prompt asset %s: %w", filePath, err)
|
||||||
|
}
|
||||||
|
assets["assets/prompts/"+cleanModuleDir+"/"+name] = append([]byte(nil), data...)
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, name := range commonDNDPromptFiles {
|
||||||
|
data, err := fs.ReadFile(sharedFS, "assets/prompts/"+name)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, fmt.Errorf("read shared prompt asset %s: %w", name, err)
|
return nil, fmt.Errorf("read shared prompt asset %s: %w", name, err)
|
||||||
}
|
}
|
||||||
assets["assets/prompts/dnd.scenes/"+name] = append([]byte(nil), data...)
|
assets["assets/prompts/"+cleanModuleDir+"/"+name] = append([]byte(nil), data...)
|
||||||
}
|
}
|
||||||
return assets, nil
|
return assets, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func promptMapFSFromEmbedded(files map[string]string) (fs.FS, error) {
|
var commonDNDPromptFiles = []string{
|
||||||
assets := make(promptMapFS, len(files))
|
"common-dnd-system.md",
|
||||||
for virtualPath, embeddedPath := range files {
|
"common-dnd-transcript.md",
|
||||||
data, err := fs.ReadFile(embeddedAssets, embeddedPath)
|
"common-dnd-references.md",
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf("read prompt asset %s: %w", embeddedPath, err)
|
|
||||||
}
|
|
||||||
assets[virtualPath] = append([]byte(nil), data...)
|
|
||||||
}
|
|
||||||
return assets, nil
|
|
||||||
}
|
}
|
||||||
|
|
||||||
type promptMapFS map[string][]byte
|
type promptMapFS map[string][]byte
|
||||||
|
|
||||||
func (m promptMapFS) Open(name string) (fs.File, error) {
|
func (m promptMapFS) Open(name string) (fs.File, error) {
|
||||||
cleaned, err := cleanPromptPath(name)
|
cleaned, err := cleanPromptFSPath(name)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, &fs.PathError{Op: "open", Path: name, Err: err}
|
return nil, &fs.PathError{Op: "open", Path: name, Err: err}
|
||||||
}
|
}
|
||||||
@@ -71,7 +92,7 @@ func (m promptMapFS) Open(name string) (fs.File, error) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
func (m promptMapFS) ReadDir(name string) ([]fs.DirEntry, error) {
|
func (m promptMapFS) ReadDir(name string) ([]fs.DirEntry, error) {
|
||||||
cleaned, err := cleanPromptPath(name)
|
cleaned, err := cleanPromptFSPath(name)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, &fs.PathError{Op: "readdir", Path: name, Err: err}
|
return nil, &fs.PathError{Op: "readdir", Path: name, Err: err}
|
||||||
}
|
}
|
||||||
@@ -130,6 +151,21 @@ func cleanPromptPath(name string) (string, error) {
|
|||||||
return cleaned, nil
|
return cleaned, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func cleanPromptFSPath(name string) (string, error) {
|
||||||
|
trimmed := strings.TrimSpace(name)
|
||||||
|
if trimmed == "" {
|
||||||
|
return "", fmt.Errorf("path must not be empty")
|
||||||
|
}
|
||||||
|
cleaned := path.Clean(strings.TrimPrefix(trimmed, "/"))
|
||||||
|
if cleaned == "." {
|
||||||
|
return cleaned, nil
|
||||||
|
}
|
||||||
|
if !fs.ValidPath(cleaned) {
|
||||||
|
return "", fmt.Errorf("invalid path %q", name)
|
||||||
|
}
|
||||||
|
return cleaned, nil
|
||||||
|
}
|
||||||
|
|
||||||
type promptFile struct {
|
type promptFile struct {
|
||||||
reader *bytes.Reader
|
reader *bytes.Reader
|
||||||
info promptFileInfo
|
info promptFileInfo
|
||||||
77
internal/modules/sharedassets/prompt_fs_test.go
Normal file
77
internal/modules/sharedassets/prompt_fs_test.go
Normal file
@@ -0,0 +1,77 @@
|
|||||||
|
package sharedassets
|
||||||
|
|
||||||
|
import (
|
||||||
|
"io/fs"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"testing/fstest"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestModulePromptFSCombinesModuleAndSharedPrompts(t *testing.T) {
|
||||||
|
fsys, err := ModulePromptFS("dnd.test", fstest.MapFS{
|
||||||
|
"assets/prompts/dnd.test.yaml": {Data: []byte("id: dnd.test")},
|
||||||
|
"assets/prompts/task.md": {Data: []byte("task")},
|
||||||
|
}, []ModulePromptFile{
|
||||||
|
{Name: "dnd.test.yaml", Path: "assets/prompts/dnd.test.yaml"},
|
||||||
|
{Name: "task.md", Path: "assets/prompts/task.md"},
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("ModulePromptFS() error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
tests := map[string]string{
|
||||||
|
"assets/prompts/dnd.test/dnd.test.yaml": "id: dnd.test",
|
||||||
|
"assets/prompts/dnd.test/task.md": "task",
|
||||||
|
"assets/prompts/dnd.test/common-dnd-system.md": "You work with Dungeons & Dragons gameplay transcripts.",
|
||||||
|
"assets/prompts/dnd.test/common-dnd-transcript.md": "{{ input \"transcript\" }}",
|
||||||
|
"assets/prompts/dnd.test/common-dnd-references.md": "Roster reference:",
|
||||||
|
}
|
||||||
|
for path, want := range tests {
|
||||||
|
data, err := fs.ReadFile(fsys, path)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("ReadFile(%q) error = %v, want nil", path, err)
|
||||||
|
}
|
||||||
|
if !strings.Contains(string(data), want) {
|
||||||
|
t.Fatalf("ReadFile(%q) = %q, want substring %q", path, data, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
entries, err := fs.ReadDir(fsys, ".")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("ReadDir(.) error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if len(entries) != 1 || entries[0].Name() != "assets" || !entries[0].IsDir() {
|
||||||
|
t.Fatalf("ReadDir(.) = %#v, want assets directory", entries)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestModulePromptFSRejectsMissingModuleFile(t *testing.T) {
|
||||||
|
_, err := ModulePromptFS("dnd.test", fstest.MapFS{}, []ModulePromptFile{
|
||||||
|
{Name: "task.md", Path: "assets/prompts/task.md"},
|
||||||
|
})
|
||||||
|
if err == nil || !strings.Contains(err.Error(), "read module prompt asset assets/prompts/task.md") {
|
||||||
|
t.Fatalf("ModulePromptFS() error = %v, want missing module asset context", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestModulePromptFSRejectsMissingSharedFile(t *testing.T) {
|
||||||
|
_, err := modulePromptFS("dnd.test", fstest.MapFS{
|
||||||
|
"assets/prompts/task.md": {Data: []byte("task")},
|
||||||
|
}, []ModulePromptFile{
|
||||||
|
{Name: "task.md", Path: "assets/prompts/task.md"},
|
||||||
|
}, fstest.MapFS{})
|
||||||
|
if err == nil || !strings.Contains(err.Error(), "read shared prompt asset common-dnd-system.md") {
|
||||||
|
t.Fatalf("modulePromptFS() error = %v, want missing shared asset context", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestModulePromptFSRejectsNestedVirtualFileName(t *testing.T) {
|
||||||
|
_, err := ModulePromptFS("dnd.test", fstest.MapFS{
|
||||||
|
"assets/prompts/task.md": {Data: []byte("task")},
|
||||||
|
}, []ModulePromptFile{
|
||||||
|
{Name: "nested/task.md", Path: "assets/prompts/task.md"},
|
||||||
|
})
|
||||||
|
if err == nil || !strings.Contains(err.Error(), "must not contain path separators") {
|
||||||
|
t.Fatalf("ModulePromptFS() error = %v, want nested file name error", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
Reference in New Issue
Block a user