From f1a657401329d9cab4831457b61e6921d9590db1 Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Tue, 28 Jul 2026 16:52:49 +0000 Subject: [PATCH] Document the PromptKit integration --- docs/config.md | 32 +++-- docs/development.md | 2 +- docs/integrations/pkg-promptkit.md | 176 +++++------------------ docs/internal/cli.md | 21 +-- docs/internal/llm.md | 32 ++++- docs/operations.md | 16 ++- docs/roadmap/future.md | 13 +- docs/roadmap/implementation.md | 6 +- internal/framework/promptfs/prompt_fs.go | 4 +- 9 files changed, 121 insertions(+), 181 deletions(-) diff --git a/docs/config.md b/docs/config.md index 1b804d4..7920740 100644 --- a/docs/config.md +++ b/docs/config.md @@ -1,7 +1,7 @@ # Configuration This is the canonical reference for Notarius configuration. Configuration files -are YAML and must declare version 3. They select pipelines and their modules; +are YAML and must declare version 4. They select pipelines and their modules; the [CLI reference](cli.md) owns invocation syntax, and [Operations](operations.md) owns run-state procedures. @@ -45,8 +45,8 @@ other than **version** is optional. | Field | Type | Default | Rules | | --- | --- | --- | --- | -| **version** | integer | none | Required; must be 3. | -| **scriptorium** | object | none | Profile source configuration. | +| **version** | integer | none | Required; must be 4. | +| **promptkit** | object | none | Profile source configuration. | | **pipelines** | map | empty | Maps pipeline IDs to pipeline definitions. | | **concurrency** | object | see below | Global LLM and extraction limits. | | **output** | object | see below | Published output settings. | @@ -69,9 +69,16 @@ Built-in defaults are: An empty cache directory in YAML deliberately selects the corresponding per-user root. An explicit empty output or debug directory is invalid. -## Scriptorium Profiles +## PromptKit Profiles -The optional **scriptorium** object selects one source of profile definitions: +The optional **promptkit** object selects one source of profile definitions: + +~~~yaml +version: 4 +promptkit: + profile_dir: /path/to/profiles + # profile_file: /path/to/profiles.yml +~~~ | Field | Type | Rules | | --- | --- | --- | @@ -79,10 +86,19 @@ The optional **scriptorium** object selects one source of profile definitions: | **profile_file** | string | Non-empty profile file. | Set at most one of these fields. Profile IDs used by a binding must be available -from the selected Scriptorium profile source when the pipeline is resolved. +from the selected PromptKit profile source when the pipeline is resolved. Keep credentials out of this file: configure a profile to read its credential from an environment variable, then set that environment variable only in the -run environment. +run environment. PromptKit owns the profile-file format; see the +[PromptKit upstream boundary](integrations/pkg-promptkit.md) for the pinned +package and canonical format reference. + +## Migrating Version 3 Configuration + +Version 3 files are not decoded or rewritten. Change **version: 3** to +**version: 4** and rename the top-level **scriptorium:** section to +**promptkit:**. Version 4 decoding is strict, so a remaining **scriptorium** +field is rejected as unknown. ## Operational Environment Variables @@ -199,7 +215,7 @@ extract: | Binding field | Type | Default | Rules | | --- | --- | --- | --- | | **module** | string | none | Required for an object binding. Must be a registered compatible key. | -| **llm_profile** | string | none | Optional non-empty Scriptorium profile ID. | +| **llm_profile** | string | none | Optional non-empty PromptKit profile ID. | | **retries** | integer | 0 | Non-negative additional attempts for chunk, extract, merge, and normalize bindings. | | **options** | object | none | Must satisfy the selected module. | | **references** | map | none | Valid only on chunk, extract, merge, and normalize bindings. | diff --git a/docs/development.md b/docs/development.md index 03296b6..a6def97 100644 --- a/docs/development.md +++ b/docs/development.md @@ -22,7 +22,7 @@ implemented component map. | Configuration loading, resolution, or user-visible configuration behavior | [Configuration Internals](internal/configuration.md) and [Configuration](config.md) | The internal guide owns loading and resolution mechanics; the reference owns the configuration contract. | | Pipeline resolution or execution | [Pipeline Internals](internal/pipeline.md) | It documents profiles, references, validation, retries, checkpoints, and runner behavior. | | Production modules or validators | [Module Internals](internal/modules.md), [D&D Module Internals](internal/dnd.md), and [D&D integration contracts](integrations/) | The generic guide owns extension mechanics, the D&D guide owns shared family conventions, and the contracts own durable output shapes. | -| LLM clients, prompts, schemas, profiles, or scheduling | [LLM Runtime](internal/llm.md) | It documents the transport boundary and Scriptorium integration. | +| LLM clients, prompts, schemas, profiles, or scheduling | [LLM Runtime](internal/llm.md) | It documents the transport boundary and PromptKit integration. | | Output, cache, resume, or debug artifacts | [Run State Internals](internal/state.md), [Operations](operations.md), and [Configuration](config.md) | These separate implementation details, operator behavior, and configuration contracts. | | External input formats, artifact schemas, or durable output files | [Integration Contracts](integrations/) | Integration documents define external and durable data contracts. | | Proposed or unimplemented behavior | [Roadmap](roadmap/) | Future work belongs only in roadmap documentation until implemented. | diff --git a/docs/integrations/pkg-promptkit.md b/docs/integrations/pkg-promptkit.md index 989e887..582f26c 100644 --- a/docs/integrations/pkg-promptkit.md +++ b/docs/integrations/pkg-promptkit.md @@ -1,151 +1,41 @@ -# Package `promptkit` +# PromptKit Integration -Import path: +Notarius pins +[`gitea.maximumdirect.net/eric/promptkit` v0.1.0](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0) +as its in-process prompt engine. The upstream +[Go package consumer guide](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/docs/consumers/pkg-promptkit.md) +owns the public engine API, and the upstream +[format reference](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/docs/formats.md) +owns prompt, profile, and schema file contracts. -```go -import "gitea.maximumdirect.net/eric/promptkit" -``` +## Supported Boundary -Package `promptkit` is the supported Go contract for in-process prompt -preparation and execution. The declarations and their GoDoc in the -[root package](../../doc.go) own the exact API; this guide explains how the -pieces are used together. The [framework format reference](../formats.md) owns -prompt, profile, and schema file contracts. +Notarius relies on the root `promptkit` package to: -## Engine Construction And Sources +- construct an `Engine` with filesystem-backed prompt, schema, and optional + profile sources; +- prepare and run a `RunRequest` with named inline artifacts, variables, + metadata, prompt identity, and profile selection; +- return rendered debug material, validated structured output, selected + profile and model metadata, and token usage; +- distinguish structured-output validation failure from execution failure; and +- identify a missing explicit profile through `ErrProfileNotFound`. -Construct an engine with [`NewEngine`, `Config`, and -`Option`](../../engine.go). `PromptDir` is required unless a prompt source -option is supplied. `ProfileDir` optionally overlays built-in profiles, and an -empty `SchemaDir` uses the current directory. `Timeout` is the transport-wide -safety cap for the built-in OpenAI-compatible client. An optional `HTTPClient` -is cloned; its positive timeout takes precedence. +Notarius does not use PromptKit's optional `ArtifactReader`. It materializes +source and reference content itself and supplies owned inline artifacts at the +adapter boundary. It also retains responsibility for pipeline retries, +scheduling, debug persistence, redaction, profile provenance, and conversion +from private model responses into durable domain artifacts. -Nil options are ignored. Invalid construction, including a nil injected client -or artifact reader, returns an error matching `ErrInvalidConfig`. +## Notarius Ownership -The [source options](../../engine.go) replace their matching directory source: +[LLM Runtime Internals](../internal/llm.md) describes how Notarius mounts +module assets, maps its transport-neutral completion contract, prepares and +executes requests, validates output, records provenance, captures debug +material, redacts errors, and preserves timeout ownership. +[Configuration](../config.md#promptkit-profiles) defines how a Notarius +configuration selects one PromptKit profile source. -- `WithPromptFS` and `WithPromptFile` select prompt definitions; -- `WithProfileFS` and `WithProfileFile` overlay built-in profiles; -- `WithProfiles` adds in-memory profiles ahead of file and built-in profiles; -- `WithSchemaFS` and `WithSchemaFile` select JSON Schema documents; -- `WithLLMClient` replaces the built-in model client; and -- `WithArtifactReader` replaces the default reader for every input. - -Source selection, path resolution, strict decoding, profile overlays, and -file-to-request precedence are defined in the -[framework format reference](../formats.md). - -Per-generation timeout values from profiles or requests are independent of -the transport cap and caller context. An explicit request value of zero -disables only the per-generation deadline. The -[outbound integration contract](../integrations/openai-compatible-chat.md#timeout-and-cancellation) -defines the complete timeout layering. - -## Preparation And Execution - -[`Engine.Prepare` and `Engine.Run`](../../engine.go) accept the public -[`RunRequest`](../../types.go). `Prepare` resolves the prompt, profile, input -artifacts, validation contract, and rendered messages without calling an LLM. -`Run` performs the same preparation, calls the configured client, and validates -the generated content. The maintained -[offline preparation example](../../examples/go-library/prepare/main.go) -provides a complete runnable workflow using a prompt file, in-memory profile, -and inline input. - -[`PreparedRun` and `RunResult`](../../types.go) expose copied public values. -Preparation returns effective settings, hashes, rendered messages, selected -profile, structured-output information, and timing without resolved secrets or -model output. Execution adds the generated artifact and raw output, validation -state, model metadata, usage, run ID, and duration. - -A generated-content validation failure returns a result with -`Validation.Status == ValidationFailed`. An inability to perform validation -returns an error matching `ErrValidation`. - -## Requests, Inputs, And Overrides - -The [request and value declarations](../../types.go) own the available fields, -serialized constants, and result shapes. Use `File`, `Inline`, or -`InlineWithURI` to construct artifact references. The -[framework format reference](../formats.md) defines declared inputs, template -references, output contracts, and the relationship between file values and -request overrides. - -`ExecutionTargetOverride` uses pointers for numeric settings so an explicit -zero remains distinct from no override. `ExtraParams` accepts JSON-compatible -strings, booleans, finite numbers, string-keyed objects, arrays or slices, and -nil. Unsupported values, non-string map keys, non-finite numbers, and cycles -match `ErrInvalidConfig` in profiles or `ErrInvalidRequest` in request -overrides. - -Returned requests, profiles, prepared values, results, artifacts, maps, and -slices are isolated from internal engine state. Consumers and injected -extensions should not retain or mutate values owned by another caller. - -## Profiles And Credentials - -[`OpenAICompatibleProfile`](../../profiles.go) constructs an ordinary -in-memory profile for an OpenAI-compatible chat-completions endpoint. -`WithProfiles` rejects duplicate IDs in one call and gives in-memory profiles -precedence over explicit file sources and built-ins. - -Raw API keys do not belong in profiles. File-backed profiles may name an -environment variable, while an in-memory profile can require a request key. -A direct `RunRequest.APIKey` is request-scoped and takes precedence over an -environment lookup for the built-in client. Profile fields, ranges, built-ins, -precedence, and credential rules are owned by the -[framework format reference](../formats.md). - -API keys are excluded from JSON, prepared values, and results. The public -`String` and `GoString` methods report only whether a direct key is present. -Avoid reflection-based dumps of request structs, which can bypass that -redaction. - -## Extension Interfaces - -The [`LLMClient`, `GenerateRequest`, and -`GenerateResponse`](../../types.go) boundary lets a consumer replace model -generation. Injected clients receive copied rendered messages, effective -settings, explicit numeric-setting presence, structured-output constraints, -and the request-scoped key. They return generated content and token usage. - -The [`ArtifactReader`](../../types.go) boundary replaces the default inline and -file reader for every input. Readers provide artifact content and metadata; the -engine fills an empty artifact name from the input-map key. A reader error -matches `ErrArtifactLoad` while preserving the original identity for -`errors.Is`. A nil artifact with a nil error is also an artifact-load failure. - -Extensions should honor context cancellation and avoid logging raw prompts, -artifacts, or credentials. - -## Errors - -The [public error declarations](../../engine.go) and -[mapping](../../errors.go) preserve these sentinel checks through `errors.Is`: - -- `ErrInvalidConfig` -- `ErrInvalidRequest` -- `ErrPromptNotFound` -- `ErrProfileNotFound` -- `ErrProfileRequired` -- `ErrPromptLoad` -- `ErrProfileLoad` -- `ErrAPIKeyEnvMissing` -- `ErrArtifactLoad` -- `ErrPromptRender` -- `ErrLLMGenerate` -- `ErrValidation` - -`ErrProfileRequired` and `ErrAPIKeyEnvMissing` also match -`ErrInvalidRequest`, allowing either broad request handling or a specific -condition. Wrapped collaborator errors retain their identity where the public -contract promises it. - -## Consumer Boundary - -Promptkit is an importable library. It does not own a command, inbound HTTP -API, process configuration, or deployment policy. Scriptorium is one -downstream application that maps this root package contract into those -application concerns. +PromptKit API or format changes outside this boundary are not implicitly +supported. Updating the pinned version requires reviewing the adapter and +profile/configuration contracts against the upstream documentation. diff --git a/docs/internal/cli.md b/docs/internal/cli.md index 423508d..5a4a718 100644 --- a/docs/internal/cli.md +++ b/docs/internal/cli.md @@ -39,8 +39,10 @@ in [Configuration Internals](configuration.md). Configuration validation without a selected pipeline checks structural configuration only. Validation with a selected pipeline also builds the effective catalog, resolves the pipeline, and verifies explicitly selected -Scriptorium profiles. Pipeline listing validates configuration before returning -normalized, sorted identifiers. +PromptKit profiles. Each explicit binding or validator profile is prepared +against the configured PromptKit source without performing generation, so an +unknown profile fails before pipeline preparation. Pipeline listing validates +configuration before returning normalized, sorted identifiers. ## Production Composition @@ -51,12 +53,13 @@ catalog used for resolution and the concrete constructors used for preparation. Tests may provide a catalog or registries instead; production code must not silently merge an injected partial catalog with production registrations. -The production LLM factory builds the PromptKit-backed client from resolved -configuration, creates one scheduler from the effective global LLM limit, and -wraps the client before it reaches modules. Registration and LLM construction -errors are returned before a pipeline is prepared. Concrete module keys and -validator chains are public configuration choices and remain documented in -[Configuration](../config.md). +The production LLM factory builds one PromptKit-backed client from the resolved +**promptkit.profile_dir** or **promptkit.profile_file** source, attaches the +profile-provenance recorder, creates one scheduler from the effective global +LLM limit, and wraps the client before it reaches modules. Registration and LLM +construction errors are returned before a pipeline is prepared. Configuration +field definitions remain in [Configuration](../config.md#promptkit-profiles); +the adapter mechanics remain in [LLM Runtime](llm.md). ## Run Orchestration @@ -68,7 +71,7 @@ handoff: 2. create and validate a safe run identity, then allocate a debug bundle only when requested; 3. build the effective catalog, resolve requested reference changes, resolve - the effective pipeline, and verify explicit Scriptorium profiles; + the effective pipeline, and verify explicit PromptKit profiles; 4. materialize external or generated references and record redacted invocation and resolution provenance when debug capture is enabled; 5. construct registries, the scheduled LLM client, prepared modules, and the diff --git a/docs/internal/llm.md b/docs/internal/llm.md index 47afdde..1799c35 100644 --- a/docs/internal/llm.md +++ b/docs/internal/llm.md @@ -6,7 +6,7 @@ provider calls, assembles registered prompt and schema assets, records selected profiles, and redacts provider errors. The architectural boundary is defined in [Architecture](../policy/architecture.md#llm-boundary); profile sources, credentials, and concurrency settings belong in -[Configuration](../config.md#scriptorium-profiles) and +[Configuration](../config.md#promptkit-profiles) and [Configuration](../config.md#concurrency-output-cache-and-debug). ## Structured Completion Boundary @@ -26,16 +26,20 @@ durable schemas. Those responsibilities remain with the module and its `PromptKitClient` validates the request target and prompt identity, maps each named material to a PromptKit inline artifact while preserving its origin URI, -forwards session and profile selection, then prepares and runs the prompt. It -returns PromptKit’s validated raw bytes rather than re-encoding the decoded -target. An empty optional material is represented as one space so its named -input is retained by PromptKit. +maps the request session to the existing `session_id` prompt variable, forwards +profile selection, then prepares and runs the prompt. PromptKit v0.1.0 has no +direct request-level session field. The adapter returns PromptKit’s validated +raw bytes rather than re-encoding the decoded target. An empty optional +material is represented as one space so its named input is retained by +PromptKit. An empty request profile lets the prompt select its configured default. The CLI prepares every explicitly selected binding profile before a run begins, so a missing explicit profile fails before stage execution. Calls record the profile actually selected by PromptKit; the recorder deduplicates non-secret profile -identity, provider, and model values for manifest use. +identity, provider, and model values for manifest use. Successful completion +responses and recorded profile manifests identify the adapter provider as +`promptkit`. ## Shared Provider-Call Limit @@ -111,6 +115,22 @@ attempt lifecycle, validation chains, and retry diagnostics; see [Pipeline Internals](pipeline.md#validation-retries-and-output) and the [binding reference](../config.md#module-bindings-and-validators). +## Timeout Ownership + +The caller context remains the outer cancellation authority. PromptKit applies +a positive effective generation timeout as an inner request deadline; an +explicit zero disables only that generation deadline. The HTTP client timeout +is a separate transport-wide cap. Notarius forwards the caller context and +does not install another timeout wrapper around PromptKit. + +The selected PromptKit profile owns generation settings. Notarius binding +retries remain outside the adapter and repeat the complete module operation +and validation chain. PromptKit v0.1.0 does not add a provider retry loop. +Operator-facing behavior is summarized in +[Operations](../operations.md#operational-limits), and the pinned upstream +contract is identified in +[PromptKit Integration](../integrations/pkg-promptkit.md). + ## Observability And Redaction When debug recording is enabled, the pipeline decorates the shared client. The diff --git a/docs/operations.md b/docs/operations.md index eede0c7..fd29516 100644 --- a/docs/operations.md +++ b/docs/operations.md @@ -207,9 +207,19 @@ or automatic cleanup command. ## Operational Limits -Provider retries and timeouts are supplied by the selected Scriptorium profile. -Module retry settings and concurrency limits are configuration contracts; see -[module bindings](config.md#module-bindings-and-validators) and +Provider execution settings and the generation timeout come from the selected +PromptKit profile. PromptKit v0.1.0 does not add a provider retry loop; +Notarius binding retries rerun the complete module operation and validation +chain as defined by [module bindings](config.md#module-bindings-and-validators). + +Timeouts are layered. Caller cancellation is the outer authority. A positive +effective generation timeout adds an inner request deadline, while zero +disables only that generation deadline. The HTTP client timeout remains a +transport-wide cap. Notarius does not add another timeout around PromptKit. +The pinned upstream boundary and profile-format links are in +[PromptKit Integration](integrations/pkg-promptkit.md). + +Concurrency limits are configuration contracts; see [concurrency](config.md#concurrency-output-cache-and-debug). Extract-worker limits and actual provider-call limits are independent. Notarius writes local filesystem state only; remote storage, archival, and retention automation are diff --git a/docs/roadmap/future.md b/docs/roadmap/future.md index 24a043c..27df403 100644 --- a/docs/roadmap/future.md +++ b/docs/roadmap/future.md @@ -95,9 +95,9 @@ safety checks, and deterministic application of accepted changes. ### Native Session Propagation -- Once upstream Scriptorium exposes a session identifier on its run request, +- Once upstream PromptKit exposes a direct request-level session identifier, propagate the existing `StructuredCompletionRequest.SessionID` through the - Scriptorium adapter's native session field. + PromptKit adapter's native session field. - Preserve the current `--session-id` invocation contract and its run-wide propagation to every prompt-facing module and validator. Do not introduce a second session configuration surface. @@ -110,10 +110,11 @@ safety checks, and deterministic application of accepted changes. concurrent-run isolation, and unsupported-provider behavior once the upstream contract is available. -This work is blocked on native session support in the upstream Scriptorium -package. Notarius already carries a run-scoped session ID through its CLI, -pipeline requests, checkpoint identity, and prompt variables; the missing -capability is native propagation across the LLM adapter boundary. +This work is blocked because PromptKit v0.1.0 does not expose the required +direct request-level session field. Notarius already carries a run-scoped +session ID through its CLI, pipeline requests, checkpoint identity, and a +`session_id` prompt variable; that prompt-variable propagation is not native +provider session support. ## Further Reference Evolution diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md index be22283..4fb66ae 100644 --- a/docs/roadmap/implementation.md +++ b/docs/roadmap/implementation.md @@ -3,9 +3,9 @@ ## Status In progress. The dependency, framework adapter, version 4 PromptKit -configuration migration, provider-neutral module prompt-asset support, and -provenance alignment are implemented; canonical documentation and final -repository verification are still planned. +configuration migration, provider-neutral module prompt-asset support, +provenance alignment, and canonical documentation are implemented; final +repository verification is still planned. ## Objective diff --git a/internal/framework/promptfs/prompt_fs.go b/internal/framework/promptfs/prompt_fs.go index eb6f6c5..feccd7b 100644 --- a/internal/framework/promptfs/prompt_fs.go +++ b/internal/framework/promptfs/prompt_fs.go @@ -12,14 +12,14 @@ import ( ) // ModulePromptFile maps a module-owned embedded prompt file into the -// Scriptorium-visible module prompt directory. +// PromptKit-visible module prompt directory. type ModulePromptFile struct { Name string Path string } // SharedPromptFile maps a caller-owned shared prompt file into a module's -// Scriptorium-visible sharedassets prompt subdirectory. +// PromptKit-visible sharedassets prompt subdirectory. type SharedPromptFile struct { Name string FS fs.FS