Refocus internal component documentation
This commit is contained in:
@@ -2,142 +2,79 @@
|
||||
|
||||
## Purpose
|
||||
|
||||
This document covers implemented prompt, profile, schema, artifact, and catalog source behavior. It is for developers changing loaders or source wiring.
|
||||
This document describes how source packages load prompt definitions, profiles,
|
||||
schemas, and artifacts. The [configuration reference](../config.md) owns their
|
||||
user-facing formats and settings. The [HTTP API reference](../api.md) owns
|
||||
HTTP-visible artifact outcomes; [operations](../operations.md) owns deployment
|
||||
handling.
|
||||
|
||||
Full user-facing YAML and config reference material belongs in `docs/config.md`.
|
||||
## Prompt Definitions
|
||||
|
||||
## Prompt Definition Sources
|
||||
`internal/promptdef` provides filesystem and `fs.FS` repositories. Both use
|
||||
`internal/filecatalog` for recursive YAML discovery, deterministic ordering,
|
||||
display paths, and root cleaning.
|
||||
|
||||
`internal/promptdef` provides directory-backed and `fs.FS` repositories.
|
||||
Repositories select a prompt by YAML ID and optional version rather than by
|
||||
path. They decode through strict YAML handling, reject duplicate matching
|
||||
definitions, and resolve `content_file` relative to the definition. The `fs.FS`
|
||||
implementation resolves content paths inside its source root; absolute paths and
|
||||
traversal outside that root are rejected before file access.
|
||||
|
||||
Behavior:
|
||||
## Profiles And Built-Ins
|
||||
|
||||
- recursively scans `.yaml` and `.yml` files.
|
||||
- decodes YAML with known-fields checking.
|
||||
- looks up prompts by YAML `id`, not by path.
|
||||
- optionally filters by prompt `version`.
|
||||
- rejects duplicate matching prompt IDs.
|
||||
- requires `id`, `version`, and at least one message.
|
||||
- requires each message to set exactly one of `content` or `content_file`.
|
||||
- resolves filesystem `content_file` values relative to the prompt YAML file.
|
||||
- resolves `fs.FS` `content_file` values inside the configured source root.
|
||||
- permits prompt subdirectories only as organization; they are not part of prompt identity.
|
||||
`internal/profile` provides filesystem, `fs.FS`, and overlay repositories.
|
||||
`internal/profile/builtin` exposes embedded assets through the same repository
|
||||
interface.
|
||||
|
||||
For `fs.FS` roots, absolute paths and relative traversal outside the source root are rejected by catalog path helpers.
|
||||
An overlay asks its primary source first. It falls back only when the primary
|
||||
reports `ErrProfileNotFound`; invalid YAML, duplicate IDs, validation failures,
|
||||
and raw-key failures are returned rather than hidden by fallback. This makes a
|
||||
custom ID override a built-in ID while retaining errors in the custom source.
|
||||
|
||||
## Profile Sources
|
||||
The public engine can overlay in-memory profiles ahead of both file-backed and
|
||||
built-in repositories. Profile field definitions, validation ranges, and the
|
||||
built-in catalog remain in the [configuration reference](../config.md).
|
||||
|
||||
`internal/profile` provides directory-backed, `fs.FS`, and overlay repositories. `internal/profile/builtin` embeds built-in profile YAML assets and exposes them through the same repository interface.
|
||||
## Schemas
|
||||
|
||||
Behavior:
|
||||
`internal/validate` supplies `StandardValidator` for filesystem sources and
|
||||
`FSValidator` for `fs.FS` sources. Directory-backed validation loads the named
|
||||
schema path; it does not search directories by basename. `fs.FS` schema paths
|
||||
are cleaned and checked against their configured root, while a single-file
|
||||
source matches its file base name.
|
||||
|
||||
- recursively scans `.yaml` and `.yml` files.
|
||||
- decodes YAML with known-fields checking.
|
||||
- looks up profiles by YAML `id`, not by path.
|
||||
- rejects duplicate IDs inside the same source.
|
||||
- rejects raw `api_key` fields in YAML; file-backed profiles must use `api_key_env`.
|
||||
- validates required `endpoint` and `model` values.
|
||||
- validates numeric profile ranges.
|
||||
The runner requests a schema document before generation when it needs
|
||||
structured output. JSON and schema mismatches in generated content are
|
||||
validation results; source access, decoding, registration, and compilation
|
||||
failures are operational errors.
|
||||
|
||||
Overlay behavior:
|
||||
## Artifacts
|
||||
|
||||
- custom profiles are primary.
|
||||
- built-in profiles are fallback.
|
||||
- fallback occurs only after a primary `ErrProfileNotFound`.
|
||||
- primary validation, YAML, duplicate, and raw-key errors are returned directly.
|
||||
- duplicate IDs across custom and built-in sources are allowed because the custom profile overrides the built-in one.
|
||||
`internal/artifact` composes inline and file readers. The ordinary composite
|
||||
reader used by CLI and the public engine reads file references from the process
|
||||
filesystem. The restricted composite reader used by the HTTP adapter combines
|
||||
inline reading with a rooted file reader and optional byte limit.
|
||||
|
||||
The public Go facade can add in-memory profiles ahead of file-backed and built-in profiles.
|
||||
The rooted reader cleans paths and applies lexical containment without resolving
|
||||
symlinks. It checks relative references against the configured root and accepts
|
||||
absolute references only when they remain inside that lexical root. The OS still
|
||||
follows symlinks after that check. The public containment outcome is documented
|
||||
by the [HTTP API reference](../api.md); deployment permissions belong in
|
||||
[operations](../operations.md).
|
||||
|
||||
## Schema Sources
|
||||
## Failure Boundaries
|
||||
|
||||
`internal/validate` provides:
|
||||
Source packages report repository, decoding, duplicate, validation, and read
|
||||
failures to their callers. They do not select public status codes or response
|
||||
schemas. The runner wraps source failures with use-case categories; adapters map
|
||||
them to their own external contract.
|
||||
|
||||
- `StandardValidator` for filesystem paths.
|
||||
- `FSValidator` for `fs.FS` roots and single-file public schema sources.
|
||||
Source reads use current filesystem or `fs.FS` content for each request. These
|
||||
packages create no manifests, checkpoints, or durable run state.
|
||||
|
||||
Behavior:
|
||||
## Verification And Change Recipe
|
||||
|
||||
- `json_schema` validation requires a non-empty `schema_path`.
|
||||
- filesystem schema paths resolve relative to `schema_dir` unless absolute.
|
||||
- directory-backed schema lookup uses the explicit `schema_path`; it does not search recursively by basename.
|
||||
- `fs.FS` schema paths must remain inside the configured source root.
|
||||
- single-file schema sources match by the configured file base name.
|
||||
- schema documents are loaded before the LLM call for structured output.
|
||||
- JSON parse failures are validation content failures.
|
||||
- schema access, decode, registration, and compile failures are runtime validation errors.
|
||||
|
||||
## Artifact Sources
|
||||
|
||||
`internal/artifact` supports two input artifact reference types:
|
||||
|
||||
- `inline`
|
||||
- `file`
|
||||
|
||||
Inline behavior:
|
||||
|
||||
- requires a non-empty body.
|
||||
- produces text/plain artifacts.
|
||||
- hashes the body bytes.
|
||||
|
||||
Direct file behavior:
|
||||
|
||||
- used by CLI `run`, CLI `render`, and the public Go facade.
|
||||
- requires a non-empty URI.
|
||||
- reads from the process filesystem without HTTP artifact-root restrictions.
|
||||
- infers content type from file extension, defaulting to text/plain.
|
||||
|
||||
Restricted file behavior:
|
||||
|
||||
- used by HTTP `serve`.
|
||||
- allows inline artifacts even when no artifact root is configured.
|
||||
- denies file artifacts when no artifact root is configured.
|
||||
- resolves relative file URIs against `server.artifact_root`.
|
||||
- accepts absolute file URIs only when they pass containment checks.
|
||||
- applies `server.max_artifact_bytes` when configured.
|
||||
|
||||
Restricted containment is lexical. It cleans paths and checks the relative path against the configured root; it does not resolve symlinks. Symlinks inside the root are followed by the operating system, including symlinks that target files outside the root.
|
||||
|
||||
## Catalog Helpers
|
||||
|
||||
`internal/filecatalog` centralizes shared source helpers:
|
||||
|
||||
- recursive YAML discovery for filesystem and `fs.FS` roots.
|
||||
- deterministic sorting.
|
||||
- `.yaml` and `.yml` filtering.
|
||||
- display paths for diagnostics.
|
||||
- YAML file stems.
|
||||
- `fs.FS` root cleaning and containment checks.
|
||||
|
||||
Repository code should use these helpers instead of reimplementing path traversal and containment rules.
|
||||
|
||||
## Failure Behavior
|
||||
|
||||
Common source failures:
|
||||
|
||||
- missing prompt/profile/schema/artifact files.
|
||||
- invalid YAML or JSON.
|
||||
- unknown YAML fields.
|
||||
- duplicate prompt or profile IDs.
|
||||
- prompt/profile validation errors.
|
||||
- raw API key fields in profile YAML.
|
||||
- unsupported artifact reference type.
|
||||
- missing inline body or file URI.
|
||||
- artifact outside HTTP root.
|
||||
- artifact exceeding HTTP size limit.
|
||||
- schema load or compile failure.
|
||||
|
||||
Prompt/profile repository lookup errors are mapped by adapters separately from runtime runner errors. Validation content failures remain result state; source and schema runtime failures return errors.
|
||||
|
||||
## State And Manifests
|
||||
|
||||
Source packages do not persist run state.
|
||||
|
||||
- No manifests are read or written.
|
||||
- No source package implements skip or resume behavior.
|
||||
- Source reads reflect the current filesystem or `fs.FS` state for each request.
|
||||
|
||||
## Tests To Inspect
|
||||
Inspect:
|
||||
|
||||
- `internal/promptdef/repository_test.go`
|
||||
- `internal/profile/repository_test.go`
|
||||
@@ -147,23 +84,14 @@ Source packages do not persist run state.
|
||||
- `internal/usecase/integration_test.go`
|
||||
- `engine_test.go`
|
||||
|
||||
## Change Recipe
|
||||
When updating prompt, profile, schema, or built-in assets:
|
||||
|
||||
When updating prompt, profile, schema, or built-in-profile assets:
|
||||
1. keep assets valid for the strict loader and the relevant source boundary;
|
||||
2. update the [configuration reference](../config.md) when a file-format,
|
||||
catalog, or default changes;
|
||||
3. run focused source and integration tests, including the built-in repository
|
||||
test when embedded assets change; and
|
||||
4. update this document when discovery, precedence, containment, or failure
|
||||
mechanics change.
|
||||
|
||||
1. Keep files valid for their strict loader and source boundary.
|
||||
2. Keep maintained examples and fixtures secret-free.
|
||||
3. Run focused prompt, profile, schema, or validation tests for the changed
|
||||
source.
|
||||
4. Update the [configuration contract](../config.md) and every affected
|
||||
external contract; update this document when loading or precedence mechanics
|
||||
change.
|
||||
|
||||
## Architectural Invariants
|
||||
|
||||
- Prompt/profile identity comes from YAML `id`.
|
||||
- External YAML decoding remains strict.
|
||||
- File-backed profile YAML never accepts raw API key values.
|
||||
- Built-in profiles are fallback, not a replacement for custom source validation.
|
||||
- HTTP file artifacts remain rooted by lexical containment.
|
||||
- Schema runtime failures remain errors, while JSON/schema content mismatches remain validation results.
|
||||
The [testing policy](../policy/testing.md) owns global test sufficiency.
|
||||
|
||||
Reference in New Issue
Block a user