170 lines
6.3 KiB
Markdown
170 lines
6.3 KiB
Markdown
# Source Internals
|
|
|
|
## Purpose
|
|
|
|
This document covers implemented prompt, profile, schema, artifact, and catalog source behavior. It is for developers changing loaders or source wiring.
|
|
|
|
Full user-facing YAML and config reference material belongs in `docs/config.md`.
|
|
|
|
## Prompt Definition Sources
|
|
|
|
`internal/promptdef` provides directory-backed and `fs.FS` repositories.
|
|
|
|
Behavior:
|
|
|
|
- recursively scans `.yaml` and `.yml` files.
|
|
- decodes YAML with known-fields checking.
|
|
- looks up prompts by YAML `id`, not by path.
|
|
- optionally filters by prompt `version`.
|
|
- rejects duplicate matching prompt IDs.
|
|
- requires `id`, `version`, and at least one message.
|
|
- requires each message to set exactly one of `content` or `content_file`.
|
|
- resolves filesystem `content_file` values relative to the prompt YAML file.
|
|
- resolves `fs.FS` `content_file` values inside the configured source root.
|
|
- permits prompt subdirectories only as organization; they are not part of prompt identity.
|
|
|
|
For `fs.FS` roots, absolute paths and relative traversal outside the source root are rejected by catalog path helpers.
|
|
|
|
## Profile Sources
|
|
|
|
`internal/profile` provides directory-backed, `fs.FS`, and overlay repositories. `internal/profile/builtin` embeds built-in profile YAML assets and exposes them through the same repository interface.
|
|
|
|
Behavior:
|
|
|
|
- recursively scans `.yaml` and `.yml` files.
|
|
- decodes YAML with known-fields checking.
|
|
- looks up profiles by YAML `id`, not by path.
|
|
- rejects duplicate IDs inside the same source.
|
|
- rejects raw `api_key` fields in YAML; file-backed profiles must use `api_key_env`.
|
|
- validates required `endpoint` and `model` values.
|
|
- validates numeric profile ranges.
|
|
|
|
Overlay behavior:
|
|
|
|
- custom profiles are primary.
|
|
- built-in profiles are fallback.
|
|
- fallback occurs only after a primary `ErrProfileNotFound`.
|
|
- primary validation, YAML, duplicate, and raw-key errors are returned directly.
|
|
- duplicate IDs across custom and built-in sources are allowed because the custom profile overrides the built-in one.
|
|
|
|
The public Go facade can add in-memory profiles ahead of file-backed and built-in profiles.
|
|
|
|
## Schema Sources
|
|
|
|
`internal/validate` provides:
|
|
|
|
- `StandardValidator` for filesystem paths.
|
|
- `FSValidator` for `fs.FS` roots and single-file public schema sources.
|
|
|
|
Behavior:
|
|
|
|
- `json_schema` validation requires a non-empty `schema_path`.
|
|
- filesystem schema paths resolve relative to `schema_dir` unless absolute.
|
|
- directory-backed schema lookup uses the explicit `schema_path`; it does not search recursively by basename.
|
|
- `fs.FS` schema paths must remain inside the configured source root.
|
|
- single-file schema sources match by the configured file base name.
|
|
- schema documents are loaded before the LLM call for structured output.
|
|
- JSON parse failures are validation content failures.
|
|
- schema access, decode, registration, and compile failures are runtime validation errors.
|
|
|
|
## Artifact Sources
|
|
|
|
`internal/artifact` supports two input artifact reference types:
|
|
|
|
- `inline`
|
|
- `file`
|
|
|
|
Inline behavior:
|
|
|
|
- requires a non-empty body.
|
|
- produces text/plain artifacts.
|
|
- hashes the body bytes.
|
|
|
|
Direct file behavior:
|
|
|
|
- used by CLI `run`, CLI `render`, and the public Go facade.
|
|
- requires a non-empty URI.
|
|
- reads from the process filesystem without HTTP artifact-root restrictions.
|
|
- infers content type from file extension, defaulting to text/plain.
|
|
|
|
Restricted file behavior:
|
|
|
|
- used by HTTP `serve`.
|
|
- allows inline artifacts even when no artifact root is configured.
|
|
- denies file artifacts when no artifact root is configured.
|
|
- resolves relative file URIs against `server.artifact_root`.
|
|
- accepts absolute file URIs only when they pass containment checks.
|
|
- applies `server.max_artifact_bytes` when configured.
|
|
|
|
Restricted containment is lexical. It cleans paths and checks the relative path against the configured root; it does not resolve symlinks. Symlinks inside the root are followed by the operating system, including symlinks that target files outside the root.
|
|
|
|
## Catalog Helpers
|
|
|
|
`internal/filecatalog` centralizes shared source helpers:
|
|
|
|
- recursive YAML discovery for filesystem and `fs.FS` roots.
|
|
- deterministic sorting.
|
|
- `.yaml` and `.yml` filtering.
|
|
- display paths for diagnostics.
|
|
- YAML file stems.
|
|
- `fs.FS` root cleaning and containment checks.
|
|
|
|
Repository code should use these helpers instead of reimplementing path traversal and containment rules.
|
|
|
|
## Failure Behavior
|
|
|
|
Common source failures:
|
|
|
|
- missing prompt/profile/schema/artifact files.
|
|
- invalid YAML or JSON.
|
|
- unknown YAML fields.
|
|
- duplicate prompt or profile IDs.
|
|
- prompt/profile validation errors.
|
|
- raw API key fields in profile YAML.
|
|
- unsupported artifact reference type.
|
|
- missing inline body or file URI.
|
|
- artifact outside HTTP root.
|
|
- artifact exceeding HTTP size limit.
|
|
- schema load or compile failure.
|
|
|
|
Prompt/profile repository lookup errors are mapped by adapters separately from runtime runner errors. Validation content failures remain result state; source and schema runtime failures return errors.
|
|
|
|
## State And Manifests
|
|
|
|
Source packages do not persist run state.
|
|
|
|
- No manifests are read or written.
|
|
- No source package implements skip or resume behavior.
|
|
- Source reads reflect the current filesystem or `fs.FS` state for each request.
|
|
|
|
## Tests To Inspect
|
|
|
|
- `internal/promptdef/repository_test.go`
|
|
- `internal/profile/repository_test.go`
|
|
- `internal/profile/builtin/repository_test.go`
|
|
- `internal/artifact/reader_test.go`
|
|
- `internal/validate/standard_validator_test.go`
|
|
- `internal/usecase/integration_test.go`
|
|
- `engine_test.go`
|
|
|
|
## Change Recipe
|
|
|
|
When updating prompt, profile, schema, or built-in-profile assets:
|
|
|
|
1. Keep files valid for their strict loader and source boundary.
|
|
2. Keep maintained examples and fixtures secret-free.
|
|
3. Run focused prompt, profile, schema, or validation tests for the changed
|
|
source.
|
|
4. Update the [configuration contract](../config.md) and every affected
|
|
external contract; update this document when loading or precedence mechanics
|
|
change.
|
|
|
|
## Architectural Invariants
|
|
|
|
- Prompt/profile identity comes from YAML `id`.
|
|
- External YAML decoding remains strict.
|
|
- File-backed profile YAML never accepts raw API key values.
|
|
- Built-in profiles are fallback, not a replacement for custom source validation.
|
|
- HTTP file artifacts remain rooted by lexical containment.
|
|
- Schema runtime failures remain errors, while JSON/schema content mismatches remain validation results.
|