Compare commits

...

5 Commits

66 changed files with 878 additions and 688 deletions

View File

@@ -1,7 +1,7 @@
# Configuration # Configuration
This is the canonical reference for Notarius configuration. Configuration files This is the canonical reference for Notarius configuration. Configuration files
are YAML and must declare version 3. They select pipelines and their modules; are YAML and must declare version 4. They select pipelines and their modules;
the [CLI reference](cli.md) owns invocation syntax, and the [CLI reference](cli.md) owns invocation syntax, and
[Operations](operations.md) owns run-state procedures. [Operations](operations.md) owns run-state procedures.
@@ -45,8 +45,8 @@ other than **version** is optional.
| Field | Type | Default | Rules | | Field | Type | Default | Rules |
| --- | --- | --- | --- | | --- | --- | --- | --- |
| **version** | integer | none | Required; must be 3. | | **version** | integer | none | Required; must be 4. |
| **scriptorium** | object | none | Profile source configuration. | | **promptkit** | object | none | Profile source configuration. |
| **pipelines** | map | empty | Maps pipeline IDs to pipeline definitions. | | **pipelines** | map | empty | Maps pipeline IDs to pipeline definitions. |
| **concurrency** | object | see below | Global LLM and extraction limits. | | **concurrency** | object | see below | Global LLM and extraction limits. |
| **output** | object | see below | Published output settings. | | **output** | object | see below | Published output settings. |
@@ -69,9 +69,16 @@ Built-in defaults are:
An empty cache directory in YAML deliberately selects the corresponding An empty cache directory in YAML deliberately selects the corresponding
per-user root. An explicit empty output or debug directory is invalid. per-user root. An explicit empty output or debug directory is invalid.
## Scriptorium Profiles ## PromptKit Profiles
The optional **scriptorium** object selects one source of profile definitions: The optional **promptkit** object selects one source of profile definitions:
~~~yaml
version: 4
promptkit:
profile_dir: /path/to/profiles
# profile_file: /path/to/profiles.yml
~~~
| Field | Type | Rules | | Field | Type | Rules |
| --- | --- | --- | | --- | --- | --- |
@@ -79,10 +86,19 @@ The optional **scriptorium** object selects one source of profile definitions:
| **profile_file** | string | Non-empty profile file. | | **profile_file** | string | Non-empty profile file. |
Set at most one of these fields. Profile IDs used by a binding must be available Set at most one of these fields. Profile IDs used by a binding must be available
from the selected Scriptorium profile source when the pipeline is resolved. from the selected PromptKit profile source when the pipeline is resolved.
Keep credentials out of this file: configure a profile to read its credential Keep credentials out of this file: configure a profile to read its credential
from an environment variable, then set that environment variable only in the from an environment variable, then set that environment variable only in the
run environment. run environment. PromptKit owns the profile-file format; see the
[PromptKit upstream boundary](integrations/pkg-promptkit.md) for the pinned
package and canonical format reference.
## Migrating Version 3 Configuration
Version 3 files are not decoded or rewritten. Change **version: 3** to
**version: 4** and rename the top-level **scriptorium:** section to
**promptkit:**. Version 4 decoding is strict, so a remaining **scriptorium**
field is rejected as unknown.
## Operational Environment Variables ## Operational Environment Variables
@@ -199,7 +215,7 @@ extract:
| Binding field | Type | Default | Rules | | Binding field | Type | Default | Rules |
| --- | --- | --- | --- | | --- | --- | --- | --- |
| **module** | string | none | Required for an object binding. Must be a registered compatible key. | | **module** | string | none | Required for an object binding. Must be a registered compatible key. |
| **llm_profile** | string | none | Optional non-empty Scriptorium profile ID. | | **llm_profile** | string | none | Optional non-empty PromptKit profile ID. |
| **retries** | integer | 0 | Non-negative additional attempts for chunk, extract, merge, and normalize bindings. | | **retries** | integer | 0 | Non-negative additional attempts for chunk, extract, merge, and normalize bindings. |
| **options** | object | none | Must satisfy the selected module. | | **options** | object | none | Must satisfy the selected module. |
| **references** | map | none | Valid only on chunk, extract, merge, and normalize bindings. | | **references** | map | none | Valid only on chunk, extract, merge, and normalize bindings. |

View File

@@ -22,7 +22,7 @@ implemented component map.
| Configuration loading, resolution, or user-visible configuration behavior | [Configuration Internals](internal/configuration.md) and [Configuration](config.md) | The internal guide owns loading and resolution mechanics; the reference owns the configuration contract. | | Configuration loading, resolution, or user-visible configuration behavior | [Configuration Internals](internal/configuration.md) and [Configuration](config.md) | The internal guide owns loading and resolution mechanics; the reference owns the configuration contract. |
| Pipeline resolution or execution | [Pipeline Internals](internal/pipeline.md) | It documents profiles, references, validation, retries, checkpoints, and runner behavior. | | Pipeline resolution or execution | [Pipeline Internals](internal/pipeline.md) | It documents profiles, references, validation, retries, checkpoints, and runner behavior. |
| Production modules or validators | [Module Internals](internal/modules.md), [D&D Module Internals](internal/dnd.md), and [D&D integration contracts](integrations/) | The generic guide owns extension mechanics, the D&D guide owns shared family conventions, and the contracts own durable output shapes. | | Production modules or validators | [Module Internals](internal/modules.md), [D&D Module Internals](internal/dnd.md), and [D&D integration contracts](integrations/) | The generic guide owns extension mechanics, the D&D guide owns shared family conventions, and the contracts own durable output shapes. |
| LLM clients, prompts, schemas, profiles, or scheduling | [LLM Runtime](internal/llm.md) | It documents the transport boundary and Scriptorium integration. | | LLM clients, prompts, schemas, profiles, or scheduling | [LLM Runtime](internal/llm.md) | It documents the transport boundary and PromptKit integration. |
| Output, cache, resume, or debug artifacts | [Run State Internals](internal/state.md), [Operations](operations.md), and [Configuration](config.md) | These separate implementation details, operator behavior, and configuration contracts. | | Output, cache, resume, or debug artifacts | [Run State Internals](internal/state.md), [Operations](operations.md), and [Configuration](config.md) | These separate implementation details, operator behavior, and configuration contracts. |
| External input formats, artifact schemas, or durable output files | [Integration Contracts](integrations/) | Integration documents define external and durable data contracts. | | External input formats, artifact schemas, or durable output files | [Integration Contracts](integrations/) | Integration documents define external and durable data contracts. |
| Proposed or unimplemented behavior | [Roadmap](roadmap/) | Future work belongs only in roadmap documentation until implemented. | | Proposed or unimplemented behavior | [Roadmap](roadmap/) | Future work belongs only in roadmap documentation until implemented. |

View File

@@ -1,151 +1,41 @@
# Package `promptkit` # PromptKit Integration
Import path: Notarius pins
[`gitea.maximumdirect.net/eric/promptkit` v0.1.0](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0)
as its in-process prompt engine. The upstream
[Go package consumer guide](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/docs/consumers/pkg-promptkit.md)
owns the public engine API, and the upstream
[format reference](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/docs/formats.md)
owns prompt, profile, and schema file contracts.
```go ## Supported Boundary
import "gitea.maximumdirect.net/eric/promptkit"
```
Package `promptkit` is the supported Go contract for in-process prompt Notarius relies on the root `promptkit` package to:
preparation and execution. The declarations and their GoDoc in the
[root package](../../doc.go) own the exact API; this guide explains how the
pieces are used together. The [framework format reference](../formats.md) owns
prompt, profile, and schema file contracts.
## Engine Construction And Sources - construct an `Engine` with filesystem-backed prompt, schema, and optional
profile sources;
- prepare and run a `RunRequest` with named inline artifacts, variables,
metadata, prompt identity, and profile selection;
- return rendered debug material, validated structured output, selected
profile and model metadata, and token usage;
- distinguish structured-output validation failure from execution failure; and
- identify a missing explicit profile through `ErrProfileNotFound`.
Construct an engine with [`NewEngine`, `Config`, and Notarius does not use PromptKit's optional `ArtifactReader`. It materializes
`Option`](../../engine.go). `PromptDir` is required unless a prompt source source and reference content itself and supplies owned inline artifacts at the
option is supplied. `ProfileDir` optionally overlays built-in profiles, and an adapter boundary. It also retains responsibility for pipeline retries,
empty `SchemaDir` uses the current directory. `Timeout` is the transport-wide scheduling, debug persistence, redaction, profile provenance, and conversion
safety cap for the built-in OpenAI-compatible client. An optional `HTTPClient` from private model responses into durable domain artifacts.
is cloned; its positive timeout takes precedence.
Nil options are ignored. Invalid construction, including a nil injected client ## Notarius Ownership
or artifact reader, returns an error matching `ErrInvalidConfig`.
The [source options](../../engine.go) replace their matching directory source: [LLM Runtime Internals](../internal/llm.md) describes how Notarius mounts
module assets, maps its transport-neutral completion contract, prepares and
executes requests, validates output, records provenance, captures debug
material, redacts errors, and preserves timeout ownership.
[Configuration](../config.md#promptkit-profiles) defines how a Notarius
configuration selects one PromptKit profile source.
- `WithPromptFS` and `WithPromptFile` select prompt definitions; PromptKit API or format changes outside this boundary are not implicitly
- `WithProfileFS` and `WithProfileFile` overlay built-in profiles; supported. Updating the pinned version requires reviewing the adapter and
- `WithProfiles` adds in-memory profiles ahead of file and built-in profiles; profile/configuration contracts against the upstream documentation.
- `WithSchemaFS` and `WithSchemaFile` select JSON Schema documents;
- `WithLLMClient` replaces the built-in model client; and
- `WithArtifactReader` replaces the default reader for every input.
Source selection, path resolution, strict decoding, profile overlays, and
file-to-request precedence are defined in the
[framework format reference](../formats.md).
Per-generation timeout values from profiles or requests are independent of
the transport cap and caller context. An explicit request value of zero
disables only the per-generation deadline. The
[outbound integration contract](../integrations/openai-compatible-chat.md#timeout-and-cancellation)
defines the complete timeout layering.
## Preparation And Execution
[`Engine.Prepare` and `Engine.Run`](../../engine.go) accept the public
[`RunRequest`](../../types.go). `Prepare` resolves the prompt, profile, input
artifacts, validation contract, and rendered messages without calling an LLM.
`Run` performs the same preparation, calls the configured client, and validates
the generated content. The maintained
[offline preparation example](../../examples/go-library/prepare/main.go)
provides a complete runnable workflow using a prompt file, in-memory profile,
and inline input.
[`PreparedRun` and `RunResult`](../../types.go) expose copied public values.
Preparation returns effective settings, hashes, rendered messages, selected
profile, structured-output information, and timing without resolved secrets or
model output. Execution adds the generated artifact and raw output, validation
state, model metadata, usage, run ID, and duration.
A generated-content validation failure returns a result with
`Validation.Status == ValidationFailed`. An inability to perform validation
returns an error matching `ErrValidation`.
## Requests, Inputs, And Overrides
The [request and value declarations](../../types.go) own the available fields,
serialized constants, and result shapes. Use `File`, `Inline`, or
`InlineWithURI` to construct artifact references. The
[framework format reference](../formats.md) defines declared inputs, template
references, output contracts, and the relationship between file values and
request overrides.
`ExecutionTargetOverride` uses pointers for numeric settings so an explicit
zero remains distinct from no override. `ExtraParams` accepts JSON-compatible
strings, booleans, finite numbers, string-keyed objects, arrays or slices, and
nil. Unsupported values, non-string map keys, non-finite numbers, and cycles
match `ErrInvalidConfig` in profiles or `ErrInvalidRequest` in request
overrides.
Returned requests, profiles, prepared values, results, artifacts, maps, and
slices are isolated from internal engine state. Consumers and injected
extensions should not retain or mutate values owned by another caller.
## Profiles And Credentials
[`OpenAICompatibleProfile`](../../profiles.go) constructs an ordinary
in-memory profile for an OpenAI-compatible chat-completions endpoint.
`WithProfiles` rejects duplicate IDs in one call and gives in-memory profiles
precedence over explicit file sources and built-ins.
Raw API keys do not belong in profiles. File-backed profiles may name an
environment variable, while an in-memory profile can require a request key.
A direct `RunRequest.APIKey` is request-scoped and takes precedence over an
environment lookup for the built-in client. Profile fields, ranges, built-ins,
precedence, and credential rules are owned by the
[framework format reference](../formats.md).
API keys are excluded from JSON, prepared values, and results. The public
`String` and `GoString` methods report only whether a direct key is present.
Avoid reflection-based dumps of request structs, which can bypass that
redaction.
## Extension Interfaces
The [`LLMClient`, `GenerateRequest`, and
`GenerateResponse`](../../types.go) boundary lets a consumer replace model
generation. Injected clients receive copied rendered messages, effective
settings, explicit numeric-setting presence, structured-output constraints,
and the request-scoped key. They return generated content and token usage.
The [`ArtifactReader`](../../types.go) boundary replaces the default inline and
file reader for every input. Readers provide artifact content and metadata; the
engine fills an empty artifact name from the input-map key. A reader error
matches `ErrArtifactLoad` while preserving the original identity for
`errors.Is`. A nil artifact with a nil error is also an artifact-load failure.
Extensions should honor context cancellation and avoid logging raw prompts,
artifacts, or credentials.
## Errors
The [public error declarations](../../engine.go) and
[mapping](../../errors.go) preserve these sentinel checks through `errors.Is`:
- `ErrInvalidConfig`
- `ErrInvalidRequest`
- `ErrPromptNotFound`
- `ErrProfileNotFound`
- `ErrProfileRequired`
- `ErrPromptLoad`
- `ErrProfileLoad`
- `ErrAPIKeyEnvMissing`
- `ErrArtifactLoad`
- `ErrPromptRender`
- `ErrLLMGenerate`
- `ErrValidation`
`ErrProfileRequired` and `ErrAPIKeyEnvMissing` also match
`ErrInvalidRequest`, allowing either broad request handling or a specific
condition. Wrapped collaborator errors retain their identity where the public
contract promises it.
## Consumer Boundary
Promptkit is an importable library. It does not own a command, inbound HTTP
API, process configuration, or deployment policy. Scriptorium is one
downstream application that maps this root package contract into those
application concerns.

View File

@@ -39,8 +39,10 @@ in [Configuration Internals](configuration.md).
Configuration validation without a selected pipeline checks structural Configuration validation without a selected pipeline checks structural
configuration only. Validation with a selected pipeline also builds the configuration only. Validation with a selected pipeline also builds the
effective catalog, resolves the pipeline, and verifies explicitly selected effective catalog, resolves the pipeline, and verifies explicitly selected
Scriptorium profiles. Pipeline listing validates configuration before returning PromptKit profiles. Each explicit binding or validator profile is prepared
normalized, sorted identifiers. against the configured PromptKit source without performing generation, so an
unknown profile fails before pipeline preparation. Pipeline listing validates
configuration before returning normalized, sorted identifiers.
## Production Composition ## Production Composition
@@ -51,12 +53,13 @@ catalog used for resolution and the concrete constructors used for preparation.
Tests may provide a catalog or registries instead; production code must not Tests may provide a catalog or registries instead; production code must not
silently merge an injected partial catalog with production registrations. silently merge an injected partial catalog with production registrations.
The production LLM factory builds the Scriptorium-backed client from resolved The production LLM factory builds one PromptKit-backed client from the resolved
configuration, creates one scheduler from the effective global LLM limit, and **promptkit.profile_dir** or **promptkit.profile_file** source, attaches the
wraps the client before it reaches modules. Registration and LLM construction profile-provenance recorder, creates one scheduler from the effective global
errors are returned before a pipeline is prepared. Concrete module keys and LLM limit, and wraps the client before it reaches modules. Registration and LLM
validator chains are public configuration choices and remain documented in construction errors are returned before a pipeline is prepared. Configuration
[Configuration](../config.md). field definitions remain in [Configuration](../config.md#promptkit-profiles);
the adapter mechanics remain in [LLM Runtime](llm.md).
## Run Orchestration ## Run Orchestration
@@ -68,7 +71,7 @@ handoff:
2. create and validate a safe run identity, then allocate a debug bundle only 2. create and validate a safe run identity, then allocate a debug bundle only
when requested; when requested;
3. build the effective catalog, resolve requested reference changes, resolve 3. build the effective catalog, resolve requested reference changes, resolve
the effective pipeline, and verify explicit Scriptorium profiles; the effective pipeline, and verify explicit PromptKit profiles;
4. materialize external or generated references and record redacted invocation 4. materialize external or generated references and record redacted invocation
and resolution provenance when debug capture is enabled; and resolution provenance when debug capture is enabled;
5. construct registries, the scheduled LLM client, prepared modules, and the 5. construct registries, the scheduled LLM client, prepared modules, and the

View File

@@ -1,12 +1,12 @@
# LLM Runtime Internals # LLM Runtime Internals
`internal/framework/llm` is Notariuss provider-independent structured `internal/framework/llm` is Notariuss provider-independent structured
completion boundary. It adapts framework requests to Scriptorium, bounds completion boundary. It adapts framework requests to PromptKit, bounds
provider calls, assembles registered prompt and schema assets, records selected provider calls, assembles registered prompt and schema assets, records selected
profiles, and redacts provider errors. The architectural boundary is defined in profiles, and redacts provider errors. The architectural boundary is defined in
[Architecture](../policy/architecture.md#llm-boundary); profile sources, [Architecture](../policy/architecture.md#llm-boundary); profile sources,
credentials, and concurrency settings belong in credentials, and concurrency settings belong in
[Configuration](../config.md#scriptorium-profiles) and [Configuration](../config.md#promptkit-profiles) and
[Configuration](../config.md#concurrency-output-cache-and-debug). [Configuration](../config.md#concurrency-output-cache-and-debug).
## Structured Completion Boundary ## Structured Completion Boundary
@@ -24,22 +24,26 @@ adapter does not own source evidence, artifact conversion, normalization, or
durable schemas. Those responsibilities remain with the module and its durable schemas. Those responsibilities remain with the module and its
[integration contract](../integrations/). [integration contract](../integrations/).
`ScriptoriumClient` validates the request target and prompt identity, maps each `PromptKitClient` validates the request target and prompt identity, maps each
named material to a Scriptorium inline artifact while preserving its origin URI, named material to a PromptKit inline artifact while preserving its origin URI,
forwards session and profile selection, then prepares and runs the prompt. It maps the request session to the existing `session_id` prompt variable, forwards
returns Scriptoriums validated raw bytes rather than re-encoding the decoded profile selection, then prepares and runs the prompt. PromptKit v0.1.0 has no
target. An empty optional material is represented as one space so its named direct request-level session field. The adapter returns PromptKits validated
input is retained by Scriptorium. raw bytes rather than re-encoding the decoded target. An empty optional
material is represented as one space so its named input is retained by
PromptKit.
An empty request profile lets the prompt select its configured default. The CLI An empty request profile lets the prompt select its configured default. The CLI
prepares every explicitly selected binding profile before a run begins, so a prepares every explicitly selected binding profile before a run begins, so a
missing explicit profile fails before stage execution. Calls record the profile missing explicit profile fails before stage execution. Calls record the profile
actually selected by Scriptorium; the recorder deduplicates non-secret profile actually selected by PromptKit; the recorder deduplicates non-secret profile
identity, provider, and model values for manifest use. identity, provider, and model values for manifest use. Successful completion
responses and recorded profile manifests identify the adapter provider as
`promptkit`.
## Shared Provider-Call Limit ## Shared Provider-Call Limit
Production construction creates one Scriptorium client and wraps it in one Production construction creates one PromptKit client and wraps it in one
scheduled client. The scheduler has a fixed, positive permit limit, serves scheduled client. The scheduler has a fixed, positive permit limit, serves
queued calls in FIFO order, and removes a queued call when its context is queued calls in FIFO order, and removes a queued call when its context is
cancelled. A granted permit is released exactly once on every completion path. cancelled. A granted permit is released exactly once on every completion path.
@@ -54,7 +58,7 @@ and its effective default are owned by
## Prompt And Schema Assets ## Prompt And Schema Assets
An `AssetRegistry` collects prompt and schema filesystems from production module An `AssetRegistry` collects prompt and schema filesystems from production module
families. It flattens registered roots into the Scriptorium filesystems and families. It flattens registered roots into the PromptKit filesystems and
rejects invalid roots, unreadable assets, duplicate paths, and missing prompt rejects invalid roots, unreadable assets, duplicate paths, and missing prompt
or schema files during preparation. The frameworks `promptfs` helper combines or schema files during preparation. The frameworks `promptfs` helper combines
module-owned prompt files with reusable domain fragments without making the module-owned prompt files with reusable domain fragments without making the
@@ -95,14 +99,14 @@ meaningful input placement and cache controls of the prompt being changed.
## Validation, Repair, And Retries ## Validation, Repair, And Retries
Scriptorium performs prompt rendering, provider execution, and the prompts PromptKit performs prompt rendering, provider execution, and the prompts
structured-output validation. The adapter reports an empty result, validation structured-output validation. The adapter reports an empty result, validation
failure, empty structured body, or decode failure as failure, empty structured body, or decode failure as
`ErrInvalidStructuredOutput`, while retaining the returned raw bytes and debug `ErrInvalidStructuredOutput`, while retaining the returned raw bytes and debug
material when they exist. Provider failures remain operational errors rather material when they exist. Provider failures remain operational errors rather
than output-validation failures. than output-validation failures.
Prompt-declared repair is executed within Scriptoriums structured-output flow. Prompt-declared repair is executed within PromptKits structured-output flow.
The current production D&D prompt manifests set repair attempts to zero. That The current production D&D prompt manifests set repair attempts to zero. That
setting does not replace pipeline retry behavior: a bindings configured retry setting does not replace pipeline retry behavior: a bindings configured retry
count reruns its stage attempt after an error or rejection, and an exhausted count reruns its stage attempt after an error or rejection, and an exhausted
@@ -111,6 +115,22 @@ attempt lifecycle, validation chains, and retry diagnostics; see
[Pipeline Internals](pipeline.md#validation-retries-and-output) and the [Pipeline Internals](pipeline.md#validation-retries-and-output) and the
[binding reference](../config.md#module-bindings-and-validators). [binding reference](../config.md#module-bindings-and-validators).
## Timeout Ownership
The caller context remains the outer cancellation authority. PromptKit applies
a positive effective generation timeout as an inner request deadline; an
explicit zero disables only that generation deadline. The HTTP client timeout
is a separate transport-wide cap. Notarius forwards the caller context and
does not install another timeout wrapper around PromptKit.
The selected PromptKit profile owns generation settings. Notarius binding
retries remain outside the adapter and repeat the complete module operation
and validation chain. PromptKit v0.1.0 does not add a provider retry loop.
Operator-facing behavior is summarized in
[Operations](../operations.md#operational-limits), and the pinned upstream
contract is identified in
[PromptKit Integration](../integrations/pkg-promptkit.md).
## Observability And Redaction ## Observability And Redaction
When debug recording is enabled, the pipeline decorates the shared client. The When debug recording is enabled, the pipeline decorates the shared client. The

View File

@@ -207,9 +207,19 @@ or automatic cleanup command.
## Operational Limits ## Operational Limits
Provider retries and timeouts are supplied by the selected Scriptorium profile. Provider execution settings and the generation timeout come from the selected
Module retry settings and concurrency limits are configuration contracts; see PromptKit profile. PromptKit v0.1.0 does not add a provider retry loop;
[module bindings](config.md#module-bindings-and-validators) and Notarius binding retries rerun the complete module operation and validation
chain as defined by [module bindings](config.md#module-bindings-and-validators).
Timeouts are layered. Caller cancellation is the outer authority. A positive
effective generation timeout adds an inner request deadline, while zero
disables only that generation deadline. The HTTP client timeout remains a
transport-wide cap. Notarius does not add another timeout around PromptKit.
The pinned upstream boundary and profile-format links are in
[PromptKit Integration](integrations/pkg-promptkit.md).
Concurrency limits are configuration contracts; see
[concurrency](config.md#concurrency-output-cache-and-debug). Extract-worker [concurrency](config.md#concurrency-output-cache-and-debug). Extract-worker
limits and actual provider-call limits are independent. Notarius writes local limits and actual provider-call limits are independent. Notarius writes local
filesystem state only; remote storage, archival, and retention automation are filesystem state only; remote storage, archival, and retention automation are

View File

@@ -95,9 +95,9 @@ safety checks, and deterministic application of accepted changes.
### Native Session Propagation ### Native Session Propagation
- Once upstream Scriptorium exposes a session identifier on its run request, - Once upstream PromptKit exposes a direct request-level session identifier,
propagate the existing `StructuredCompletionRequest.SessionID` through the propagate the existing `StructuredCompletionRequest.SessionID` through the
Scriptorium adapter's native session field. PromptKit adapter's native session field.
- Preserve the current `--session-id` invocation contract and its run-wide - Preserve the current `--session-id` invocation contract and its run-wide
propagation to every prompt-facing module and validator. Do not introduce a propagation to every prompt-facing module and validator. Do not introduce a
second session configuration surface. second session configuration surface.
@@ -110,10 +110,11 @@ safety checks, and deterministic application of accepted changes.
concurrent-run isolation, and unsupported-provider behavior once the concurrent-run isolation, and unsupported-provider behavior once the
upstream contract is available. upstream contract is available.
This work is blocked on native session support in the upstream Scriptorium This work is blocked because PromptKit v0.1.0 does not expose the required
package. Notarius already carries a run-scoped session ID through its CLI, direct request-level session field. Notarius already carries a run-scoped
pipeline requests, checkpoint identity, and prompt variables; the missing session ID through its CLI, pipeline requests, checkpoint identity, and a
capability is native propagation across the LLM adapter boundary. `session_id` prompt variable; that prompt-variable propagation is not native
provider session support.
## Further Reference Evolution ## Further Reference Evolution

View File

@@ -2,7 +2,10 @@
## Status ## Status
Planned. In progress. The dependency, framework adapter, version 4 PromptKit
configuration migration, provider-neutral module prompt-asset support,
provenance alignment, and canonical documentation are implemented; final
repository verification is still planned.
## Objective ## Objective

View File

@@ -1,4 +1,4 @@
version: 3 version: 4
concurrency: concurrency:
total_llm: 2 total_llm: 2
stage_workers: stage_workers:

View File

@@ -1,4 +1,4 @@
version: 3 version: 4
pipelines: pipelines:
dnd-session: dnd-session:
input: seriatim input: seriatim

2
go.mod
View File

@@ -3,7 +3,7 @@ module gitea.maximumdirect.net/eric/notarius
go 1.25.5 go 1.25.5
require ( require (
gitea.maximumdirect.net/eric/scriptorium v0.11.1 gitea.maximumdirect.net/eric/promptkit v0.1.0
github.com/santhosh-tekuri/jsonschema/v6 v6.0.2 github.com/santhosh-tekuri/jsonschema/v6 v6.0.2
gopkg.in/yaml.v3 v3.0.1 gopkg.in/yaml.v3 v3.0.1
) )

8
go.sum
View File

@@ -1,13 +1,9 @@
gitea.maximumdirect.net/eric/scriptorium v0.11.0 h1:rjvbt9FTaWHxYlHq7QlUzmMVUt3QdbTmeCkmH81N//o= gitea.maximumdirect.net/eric/promptkit v0.1.0 h1:vuKeBxkiY8E54LRFbLQFjlJJCiOfMvB1++DYBCrD/ug=
gitea.maximumdirect.net/eric/scriptorium v0.11.0/go.mod h1:FQ5lEuNxmrQyNgIomkpZdxvfTC0jWjbXYuq3tbJWF64= gitea.maximumdirect.net/eric/promptkit v0.1.0/go.mod h1:R95NM6fbMDGDC0/UomgnSBP6ui2ns+8SZb8bESNvrDQ=
gitea.maximumdirect.net/eric/scriptorium v0.11.1 h1:zBKtB3+fP8FcHGI8DJD99CiTL6crAGitBhWtE+xYJHc=
gitea.maximumdirect.net/eric/scriptorium v0.11.1/go.mod h1:FQ5lEuNxmrQyNgIomkpZdxvfTC0jWjbXYuq3tbJWF64=
github.com/dlclark/regexp2 v1.11.0 h1:G/nrcoOa7ZXlpoa/91N3X7mM3r8eIlMBBJZvsz/mxKI= github.com/dlclark/regexp2 v1.11.0 h1:G/nrcoOa7ZXlpoa/91N3X7mM3r8eIlMBBJZvsz/mxKI=
github.com/dlclark/regexp2 v1.11.0/go.mod h1:DHkYz0B9wPfa6wondMfaivmHpzrQ3v9q8cnmRbL6yW8= github.com/dlclark/regexp2 v1.11.0/go.mod h1:DHkYz0B9wPfa6wondMfaivmHpzrQ3v9q8cnmRbL6yW8=
github.com/santhosh-tekuri/jsonschema/v6 v6.0.2 h1:KRzFb2m7YtdldCEkzs6KqmJw4nqEVZGK7IN2kJkjTuQ= github.com/santhosh-tekuri/jsonschema/v6 v6.0.2 h1:KRzFb2m7YtdldCEkzs6KqmJw4nqEVZGK7IN2kJkjTuQ=
github.com/santhosh-tekuri/jsonschema/v6 v6.0.2/go.mod h1:JXeL+ps8p7/KNMjDQk3TCwPpBy0wYklyWTfbkIzdIFU= github.com/santhosh-tekuri/jsonschema/v6 v6.0.2/go.mod h1:JXeL+ps8p7/KNMjDQk3TCwPpBy0wYklyWTfbkIzdIFU=
golang.org/x/text v0.14.0 h1:ScX5w1eTa3QqT8oi6+ziP7dTV1S2+ALU0bI+0zXKWiQ=
golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
golang.org/x/text v0.40.0 h1:Ub2Z6/xjgF1WrYQz2nuITOEegKFtiIy+rieRJ5lHZKs= golang.org/x/text v0.40.0 h1:Ub2Z6/xjgF1WrYQz2nuITOEegKFtiIy+rieRJ5lHZKs=
golang.org/x/text v0.40.0/go.mod h1:hpnzDAfGV753zIKo+wk3u1bVKCGPbrnF7+7LBF/UHVY= golang.org/x/text v0.40.0/go.mod h1:hpnzDAfGV753zIKo+wk3u1bVKCGPbrnF7+7LBF/UHVY=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=

View File

@@ -168,14 +168,14 @@ func buildProductionLLMClient(ctx context.Context, cfg config.Config, profileID
return nil, nil, fmt.Errorf("production asset registry must not be nil") return nil, nil, fmt.Errorf("production asset registry must not be nil")
} }
recorder := llm.NewLLMProfileRecorder() recorder := llm.NewLLMProfileRecorder()
client, err := llm.NewScriptoriumClient(llm.ScriptoriumClientConfig{ client, err := llm.NewPromptKitClient(llm.PromptKitClientConfig{
ProfileDir: cfg.Scriptorium.ProfileDir, ProfileDir: cfg.PromptKit.ProfileDir,
ProfileFile: cfg.Scriptorium.ProfileFile, ProfileFile: cfg.PromptKit.ProfileFile,
Assets: assets, Assets: assets,
Recorder: recorder, Recorder: recorder,
}) })
if err != nil { if err != nil {
return nil, nil, fmt.Errorf("create Scriptorium-backed LLM client: %w", err) return nil, nil, fmt.Errorf("create PromptKit-backed LLM client: %w", err)
} }
scheduler, err := llm.NewScheduler(cfg.Concurrency.TotalLLM) scheduler, err := llm.NewScheduler(cfg.Concurrency.TotalLLM)
if err != nil { if err != nil {

View File

@@ -214,13 +214,13 @@ func commandContractOptionsWithLookup(t *testing.T, lookup func(string) (string,
func writeCommandConfig(t *testing.T, firstID, secondID string) string { func writeCommandConfig(t *testing.T, firstID, secondID string) string {
t.Helper() t.Helper()
content := fmt.Sprintf("version: 3\npipelines:\n %q:\n input: seriatim\n %q:\n input: seriatim\n", firstID, secondID) content := fmt.Sprintf("version: 4\npipelines:\n %q:\n input: seriatim\n %q:\n input: seriatim\n", firstID, secondID)
return writeCommandConfigContent(t, content) return writeCommandConfigContent(t, content)
} }
func writeResolvableCommandConfig(t *testing.T) string { func writeResolvableCommandConfig(t *testing.T) string {
t.Helper() t.Helper()
return writeCommandConfigContent(t, `version: 3 return writeCommandConfigContent(t, `version: 4
pipelines: pipelines:
demo: demo:
input: seriatim input: seriatim

View File

@@ -6,8 +6,10 @@ import (
"fmt" "fmt"
"reflect" "reflect"
"strings" "strings"
"sync"
"testing" "testing"
"gitea.maximumdirect.net/eric/notarius/internal/core/artifacts"
"gitea.maximumdirect.net/eric/notarius/internal/core/config" "gitea.maximumdirect.net/eric/notarius/internal/core/config"
"gitea.maximumdirect.net/eric/notarius/internal/core/source" "gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts" "gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
@@ -28,7 +30,7 @@ func TestProductionSceneDescriptionWorkflow(t *testing.T) {
Output: pipeline.Binding("json"), Output: pipeline.Binding("json"),
Artifacts: map[string]pipeline.ArtifactLaneProfile{ Artifacts: map[string]pipeline.ArtifactLaneProfile{
"scene-descriptions": { "scene-descriptions": {
Extract: pipeline.Binding(sceneextract.Key), Extract: pipeline.ModuleBinding{Module: sceneextract.Key, LLMProfile: "scene-description-profile"},
Normalize: pipeline.Binding(scenenormalize.Key), Normalize: pipeline.Binding(scenenormalize.Key),
}, },
}, },
@@ -45,7 +47,8 @@ func TestProductionSceneDescriptionWorkflow(t *testing.T) {
t.Fatalf("resolved references = %#v / %#v, want no generated or required references", lane.ExtractReferences, lane.NormalizeReferences) t.Fatalf("resolved references = %#v / %#v, want no generated or required references", lane.ExtractReferences, lane.NormalizeReferences)
} }
prepared, err := pipeline.Prepare(effective.ResolvedPipeline, components.registries, pipeline.ModuleDependencies{LLM: sceneDescriptionLLM{}}) llmClient := &sceneDescriptionLLM{}
prepared, err := pipeline.Prepare(effective.ResolvedPipeline, components.registries, pipeline.ModuleDependencies{LLM: llmClient})
if err != nil { if err != nil {
t.Fatalf("Prepare() error = %v", err) t.Fatalf("Prepare() error = %v", err)
} }
@@ -60,6 +63,14 @@ func TestProductionSceneDescriptionWorkflow(t *testing.T) {
if output.Manifest.ValidationStatus != "approved" || len(output.Rejected) != 0 || len(output.NormalizeOutputs) != 1 { if output.Manifest.ValidationStatus != "approved" || len(output.Rejected) != 0 || len(output.NormalizeOutputs) != 1 {
t.Fatalf("run output = %#v, want one approved normalized artifact", output) t.Fatalf("run output = %#v, want one approved normalized artifact", output)
} }
wantProfiles := []artifacts.LLMProfileManifest{{
ID: "scene-description-profile",
Provider: "promptkit",
Model: "deterministic",
}}
if !reflect.DeepEqual(output.Manifest.LLMProfiles, wantProfiles) {
t.Fatalf("manifest LLM profiles = %#v, want %#v", output.Manifest.LLMProfiles, wantProfiles)
}
normalizedOutput := output.NormalizeOutputs[0] normalizedOutput := output.NormalizeOutputs[0]
if normalizedOutput.NormalizerKey != scenenormalize.Key || normalizedOutput.Artifact.Kind != dnd.SceneDescriptionListKind || normalizedOutput.Artifact.Schema.ID != scenecodec.SchemaID || normalizedOutput.Artifact.Schema.Name != scenecodec.SchemaName || normalizedOutput.Artifact.Schema.Version != scenecodec.SchemaVersion { if normalizedOutput.NormalizerKey != scenenormalize.Key || normalizedOutput.Artifact.Kind != dnd.SceneDescriptionListKind || normalizedOutput.Artifact.Schema.ID != scenecodec.SchemaID || normalizedOutput.Artifact.Schema.Name != scenecodec.SchemaName || normalizedOutput.Artifact.Schema.Version != scenecodec.SchemaVersion {
t.Fatalf("normalized output = %#v, want registered durable scene-description schema", normalizedOutput) t.Fatalf("normalized output = %#v, want registered durable scene-description schema", normalizedOutput)
@@ -85,9 +96,12 @@ func TestProductionSceneDescriptionWorkflow(t *testing.T) {
} }
} }
type sceneDescriptionLLM struct{} type sceneDescriptionLLM struct {
mu sync.Mutex
profile *artifacts.LLMProfileManifest
}
func (sceneDescriptionLLM) CompleteStructured(ctx context.Context, req contracts.StructuredCompletionRequest, out any) (contracts.StructuredCompletionResponse, error) { func (client *sceneDescriptionLLM) CompleteStructured(ctx context.Context, req contracts.StructuredCompletionRequest, out any) (contracts.StructuredCompletionResponse, error) {
if err := ctx.Err(); err != nil { if err := ctx.Err(); err != nil {
return contracts.StructuredCompletionResponse{}, err return contracts.StructuredCompletionResponse{}, err
} }
@@ -107,5 +121,27 @@ func (sceneDescriptionLLM) CompleteStructured(ctx context.Context, req contracts
if err := json.Unmarshal([]byte(content), out); err != nil { if err := json.Unmarshal([]byte(content), out); err != nil {
return contracts.StructuredCompletionResponse{}, fmt.Errorf("populate structured response: %w", err) return contracts.StructuredCompletionResponse{}, fmt.Errorf("populate structured response: %w", err)
} }
return contracts.StructuredCompletionResponse{Content: []byte(content), Provider: "test", Model: "deterministic", ProfileID: req.ProfileID}, nil profile := artifacts.LLMProfileManifest{
ID: req.ProfileID,
Provider: "promptkit",
Model: "deterministic",
}
client.mu.Lock()
client.profile = &profile
client.mu.Unlock()
return contracts.StructuredCompletionResponse{
Content: []byte(content),
Provider: profile.Provider,
Model: profile.Model,
ProfileID: profile.ID,
}, nil
}
func (client *sceneDescriptionLLM) LLMProfileManifests() []artifacts.LLMProfileManifest {
client.mu.Lock()
defer client.mu.Unlock()
if client.profile == nil {
return nil
}
return []artifacts.LLMProfileManifest{*client.profile}
} }

View File

@@ -23,7 +23,7 @@ func TestOversizedNPCRegistryFailsBeforeRuntimeAndCheckpointConstruction(t *test
t.Fatal(err) t.Fatal(err)
} }
checkpointRoot := filepath.Join(t.TempDir(), "checkpoints") checkpointRoot := filepath.Join(t.TempDir(), "checkpoints")
content := fmt.Sprintf(`version: 3 content := fmt.Sprintf(`version: 4
cache: cache:
chunk_plans: chunk_plans:
mode: bypass mode: bypass

View File

@@ -583,7 +583,7 @@ func maintainedExampleFiles(t *testing.T) []maintainedExample {
func productionSpellCatalogContractConfig(t *testing.T) string { func productionSpellCatalogContractConfig(t *testing.T) string {
t.Helper() t.Helper()
return fmt.Sprintf(`version: 3 return fmt.Sprintf(`version: 4
cache: cache:
chunk_plans: chunk_plans:
mode: bypass mode: bypass
@@ -680,7 +680,7 @@ func productionRunOptions(t *testing.T, fake *productionFakeLLMClient) Options {
} }
func productionRunConfig(outputRoot, chunkModule string) string { func productionRunConfig(outputRoot, chunkModule string) string {
return fmt.Sprintf(`version: 3 return fmt.Sprintf(`version: 4
output: output:
directory: %q directory: %q
cache: cache:

View File

@@ -0,0 +1,68 @@
package cli
import (
"context"
"errors"
"fmt"
"testing/fstest"
"gitea.maximumdirect.net/eric/notarius/internal/core/config"
"gitea.maximumdirect.net/eric/promptkit"
)
const profileCheckPromptID = "notarius.profile.check"
var profileCheckPromptFS = fstest.MapFS{
"prompts/profile-check.yaml": &fstest.MapFile{Data: []byte(`id: notarius.profile.check
version: "1.0.0"
default_profile: mistral-small-3
inputs:
- name: transcript
required: true
messages:
- role: user
content: "{{input \"transcript\"}}"
output:
format: text
validation_mode: none
repair_attempts: 0
`)},
}
func validateExplicitPromptKitProfiles(ctx context.Context, cfg config.Config, profileIDs []string) error {
if len(profileIDs) == 0 {
return nil
}
engine, err := newProfileValidationEngine(cfg)
if err != nil {
return fmt.Errorf("load PromptKit profiles: %w", err)
}
for _, profileID := range profileIDs {
if _, err := engine.Prepare(ctx, promptkit.RunRequest{
PromptID: profileCheckPromptID,
ProfileID: profileID,
Inputs: map[string]promptkit.ArtifactRef{
"transcript": promptkit.Inline("profile check"),
},
}); err != nil {
if errors.Is(err, promptkit.ErrProfileNotFound) {
return fmt.Errorf("PromptKit profile %q is not configured", profileID)
}
return fmt.Errorf("validate PromptKit profile %q: %w", profileID, err)
}
}
return nil
}
func newProfileValidationEngine(cfg config.Config) (*promptkit.Engine, error) {
opts := []promptkit.Option{
promptkit.WithPromptFS(profileCheckPromptFS, "prompts"),
}
if cfg.PromptKit.ProfileFile != "" {
opts = append(opts, promptkit.WithProfileFile(cfg.PromptKit.ProfileFile))
}
return promptkit.NewEngine(promptkit.Config{
PromptDir: "unused",
ProfileDir: cfg.PromptKit.ProfileDir,
}, opts...)
}

View File

@@ -194,7 +194,7 @@ func (recomputeTestOutput) Encode(_ context.Context, req contracts.OutputRequest
func newRecomputeTestRoots(t *testing.T) stateTestRoots { func newRecomputeTestRoots(t *testing.T) stateTestRoots {
t.Helper() t.Helper()
roots := newStateTestRoots(t) roots := newStateTestRoots(t)
config := fmt.Sprintf(`version: 3 config := fmt.Sprintf(`version: 4
output: output:
directory: %q directory: %q
cache: cache:

View File

@@ -217,7 +217,7 @@ func TestReferenceMaterializationSeparatesCLIAndConfigPathOrigins(t *testing.T)
workingDir := t.TempDir() workingDir := t.TempDir()
cfg := referenceContractConfig() cfg := referenceContractConfig()
configPath := filepath.Join(configDir, "config.yml") configPath := filepath.Join(configDir, "config.yml")
if err := os.WriteFile(configPath, []byte("version: 3\n"), 0o600); err != nil { if err := os.WriteFile(configPath, []byte("version: 4\n"), 0o600); err != nil {
t.Fatal(err) t.Fatal(err)
} }
if err := os.WriteFile(filepath.Join(configDir, "required.txt"), []byte("config reference"), 0o600); err != nil { if err := os.WriteFile(filepath.Join(configDir, "required.txt"), []byte("config reference"), 0o600); err != nil {

View File

@@ -315,7 +315,7 @@ func runPipelineCommand(args []string, stdout, stderr io.Writer, opts Options) i
return failPipelineCommand(stderr, commandState, terminalWriter, err) return failPipelineCommand(stderr, commandState, terminalWriter, err)
} }
profileIDs := effectiveLLMProfileIDs(effective.ResolvedPipeline) profileIDs := effectiveLLMProfileIDs(effective.ResolvedPipeline)
if err := validateExplicitScriptoriumProfiles(context.Background(), effective.Config, profileIDs); err != nil { if err := validateExplicitPromptKitProfiles(context.Background(), effective.Config, profileIDs); err != nil {
return failPipelineCommand(stderr, commandState, terminalWriter, err) return failPipelineCommand(stderr, commandState, terminalWriter, err)
} }
workingDir, err := os.Getwd() workingDir, err := os.Getwd()
@@ -979,7 +979,7 @@ func runConfigValidate(args []string, stdout, stderr io.Writer, opts Options) in
fmt.Fprintf(stderr, "notarius: %v\n", err) fmt.Fprintf(stderr, "notarius: %v\n", err)
return 1 return 1
} }
if err := validateExplicitScriptoriumProfiles(context.Background(), effective.Config, effectiveLLMProfileIDs(effective.ResolvedPipeline)); err != nil { if err := validateExplicitPromptKitProfiles(context.Background(), effective.Config, effectiveLLMProfileIDs(effective.ResolvedPipeline)); err != nil {
fmt.Fprintf(stderr, "notarius: %v\n", err) fmt.Fprintf(stderr, "notarius: %v\n", err)
return 1 return 1
} }

View File

@@ -241,7 +241,7 @@ func TestRunLLMProfileOverrideAndValidationUseInjectedBoundaries(t *testing.T) {
t.Run("one effective profile reaches the factory and modules", func(t *testing.T) { t.Run("one effective profile reaches the factory and modules", func(t *testing.T) {
roots := newStateTestRoots(t) roots := newStateTestRoots(t)
profileDir := writeRunContractProfiles(t, "override-profile") profileDir := writeRunContractProfiles(t, "override-profile")
prependRunContractConfig(t, roots, fmt.Sprintf("scriptorium:\n profile_dir: %q\n", profileDir)) prependRunContractConfig(t, roots, fmt.Sprintf("promptkit:\n profile_dir: %q\n", profileDir))
harness := newStateTestHarness() harness := newStateTestHarness()
var factoryProfiles []string var factoryProfiles []string
opts := harness.options() opts := harness.options()
@@ -273,7 +273,7 @@ func TestRunLLMProfileOverrideAndValidationUseInjectedBoundaries(t *testing.T) {
t.Run("validator profile remains distinct", func(t *testing.T) { t.Run("validator profile remains distinct", func(t *testing.T) {
roots := newStateTestRoots(t) roots := newStateTestRoots(t)
profileDir := writeRunContractProfiles(t, "override-profile", "validator-profile") profileDir := writeRunContractProfiles(t, "override-profile", "validator-profile")
prependRunContractConfig(t, roots, fmt.Sprintf("scriptorium:\n profile_dir: %q\n", profileDir)) prependRunContractConfig(t, roots, fmt.Sprintf("promptkit:\n profile_dir: %q\n", profileDir))
harness := newStateTestHarness() harness := newStateTestHarness()
var validatorProfiles []string var validatorProfiles []string
opts := harness.options() opts := harness.options()
@@ -299,7 +299,7 @@ func TestRunLLMProfileOverrideAndValidationUseInjectedBoundaries(t *testing.T) {
t.Run("unknown profile is rejected without factory access", func(t *testing.T) { t.Run("unknown profile is rejected without factory access", func(t *testing.T) {
roots := newStateTestRoots(t) roots := newStateTestRoots(t)
profileDir := writeRunContractProfiles(t, "override-profile") profileDir := writeRunContractProfiles(t, "override-profile")
prependRunContractConfig(t, roots, fmt.Sprintf("scriptorium:\n profile_dir: %q\n", profileDir)) prependRunContractConfig(t, roots, fmt.Sprintf("promptkit:\n profile_dir: %q\n", profileDir))
factoryCalls := 0 factoryCalls := 0
opts := newStateTestHarness().options() opts := newStateTestHarness().options()
opts.LLMClientFactory = func(context.Context, config.Config, string) (contracts.StructuredLLMClient, []artifacts.LLMProfileManifest, error) { opts.LLMClientFactory = func(context.Context, config.Config, string) (contracts.StructuredLLMClient, []artifacts.LLMProfileManifest, error) {

View File

@@ -1,68 +0,0 @@
package cli
import (
"context"
"errors"
"fmt"
"testing/fstest"
"gitea.maximumdirect.net/eric/notarius/internal/core/config"
"gitea.maximumdirect.net/eric/scriptorium"
)
const profileCheckPromptID = "notarius.profile.check"
var profileCheckPromptFS = fstest.MapFS{
"prompts/profile-check.yaml": &fstest.MapFile{Data: []byte(`id: notarius.profile.check
version: "1.0.0"
default_profile: mistral-small-3
inputs:
- name: transcript
required: true
messages:
- role: user
content: "{{input \"transcript\"}}"
output:
format: text
validation_mode: none
repair_attempts: 0
`)},
}
func validateExplicitScriptoriumProfiles(ctx context.Context, cfg config.Config, profileIDs []string) error {
if len(profileIDs) == 0 {
return nil
}
engine, err := newProfileValidationEngine(cfg)
if err != nil {
return fmt.Errorf("load Scriptorium profiles: %w", err)
}
for _, profileID := range profileIDs {
if _, err := engine.Prepare(ctx, scriptorium.RunRequest{
PromptID: profileCheckPromptID,
ProfileID: profileID,
Inputs: map[string]scriptorium.ArtifactRef{
"transcript": scriptorium.Inline("profile check"),
},
}); err != nil {
if errors.Is(err, scriptorium.ErrProfileNotFound) {
return fmt.Errorf("Scriptorium profile %q is not configured", profileID)
}
return fmt.Errorf("validate Scriptorium profile %q: %w", profileID, err)
}
}
return nil
}
func newProfileValidationEngine(cfg config.Config) (*scriptorium.Engine, error) {
opts := []scriptorium.Option{
scriptorium.WithPromptFS(profileCheckPromptFS, "prompts"),
}
if cfg.Scriptorium.ProfileFile != "" {
opts = append(opts, scriptorium.WithProfileFile(cfg.Scriptorium.ProfileFile))
}
return scriptorium.NewEngine(scriptorium.Config{
PromptDir: "unused",
ProfileDir: cfg.Scriptorium.ProfileDir,
}, opts...)
}

View File

@@ -635,7 +635,7 @@ func newStateTestRoots(t *testing.T) stateTestRoots {
t.Fatal(err) t.Fatal(err)
} }
roots.config = filepath.Join(base, "config.yml") roots.config = filepath.Join(base, "config.yml")
config := fmt.Sprintf("version: 3\noutput:\n directory: %q\ncache:\n chunk_plans:\n directory: %q\n mode: auto\n checkpoints:\n enabled: true\n directory: %q\ndebug:\n directory: %q\npipelines:\n sample:\n input: test/input\n chunk: test/chunk\n artifacts:\n items:\n extract: test/extract\n merge: test/merge\n normalize: test/normalize\n output: test/output\n", roots.output, roots.plans, roots.checkpoints, roots.debug) config := fmt.Sprintf("version: 4\noutput:\n directory: %q\ncache:\n chunk_plans:\n directory: %q\n mode: auto\n checkpoints:\n enabled: true\n directory: %q\ndebug:\n directory: %q\npipelines:\n sample:\n input: test/input\n chunk: test/chunk\n artifacts:\n items:\n extract: test/extract\n merge: test/merge\n normalize: test/normalize\n output: test/output\n", roots.output, roots.plans, roots.checkpoints, roots.debug)
if err := os.WriteFile(roots.config, []byte(config), 0o600); err != nil { if err := os.WriteFile(roots.config, []byte(config), 0o600); err != nil {
t.Fatal(err) t.Fatal(err)
} }

View File

@@ -53,7 +53,7 @@ func TestRunManifestIncludesPipelineAndArtifactLaneFields(t *testing.T) {
PipelineID: "pipeline-1", PipelineID: "pipeline-1",
PipelineDigest: "sha256:abc123", PipelineDigest: "sha256:abc123",
LLMProfiles: []LLMProfileManifest{ LLMProfiles: []LLMProfileManifest{
{ID: "default", Provider: "scriptorium", Model: "model-a"}, {ID: "default", Provider: "promptkit", Model: "model-a"},
}, },
ArtifactLanes: []ArtifactLaneManifest{ ArtifactLanes: []ArtifactLaneManifest{
{ {
@@ -102,6 +102,9 @@ func TestRunManifestIncludesPipelineAndArtifactLaneFields(t *testing.T) {
t.Fatalf("llm_profiles[0] = %#v, want object", profiles[0]) t.Fatalf("llm_profiles[0] = %#v, want object", profiles[0])
} }
assertHasKeys(t, profile, "id", "provider", "model") assertHasKeys(t, profile, "id", "provider", "model")
if profile["provider"] != "promptkit" {
t.Fatalf("llm_profiles[0].provider = %#v, want promptkit", profile["provider"])
}
lanes, ok := got["artifact_lanes"].([]any) lanes, ok := got["artifact_lanes"].([]any)
if !ok { if !ok {

View File

@@ -4,10 +4,10 @@ import (
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline" "gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
) )
const SupportedFileConfigVersion = 3 const SupportedFileConfigVersion = 4
type Config struct { type Config struct {
Scriptorium ScriptoriumConfig `json:"scriptorium,omitempty"` PromptKit PromptKitConfig `json:"promptkit,omitempty"`
Pipelines map[string]pipeline.PipelineProfile `json:"pipelines"` Pipelines map[string]pipeline.PipelineProfile `json:"pipelines"`
Concurrency ConcurrencyConfig `json:"concurrency"` Concurrency ConcurrencyConfig `json:"concurrency"`
Output OutputConfig `json:"output"` Output OutputConfig `json:"output"`
@@ -15,7 +15,7 @@ type Config struct {
Debug DebugConfig `json:"debug"` Debug DebugConfig `json:"debug"`
} }
type ScriptoriumConfig struct { type PromptKitConfig struct {
ProfileDir string `json:"profile_dir,omitempty"` ProfileDir string `json:"profile_dir,omitempty"`
ProfileFile string `json:"profile_file,omitempty"` ProfileFile string `json:"profile_file,omitempty"`
} }

View File

@@ -84,6 +84,29 @@ func TestEffectiveConfigMaterializesDefaultBindingsThroughCatalog(t *testing.T)
} }
} }
func TestEffectiveConfigPreservesPromptKitProfileSource(t *testing.T) {
tests := []struct {
name string
profileSource PromptKitConfig
}{
{name: "profile directory", profileSource: PromptKitConfig{ProfileDir: "./profiles"}},
{name: "profile file", profileSource: PromptKitConfig{ProfileFile: "./profiles.yml"}},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
cfg := configForEffectiveTests(t, effectiveProfile())
cfg.PromptKit = tt.profileSource
effective, err := cfg.Resolve(ResolveInput{PipelineID: "main", Catalog: effectiveCatalog(t)})
if err != nil {
t.Fatalf("Resolve() error = %v", err)
}
if effective.Config.PromptKit != cfg.PromptKit {
t.Fatalf("effective PromptKit config = %#v, want %#v", effective.Config.PromptKit, cfg.PromptKit)
}
})
}
}
func TestEffectiveConfigResolutionFailuresRetainContext(t *testing.T) { func TestEffectiveConfigResolutionFailuresRetainContext(t *testing.T) {
tests := []struct { tests := []struct {
name string name string

View File

@@ -10,7 +10,7 @@ import (
) )
func TestPrecedenceFileValuesOverrideBuiltInDefaults(t *testing.T) { func TestPrecedenceFileValuesOverrideBuiltInDefaults(t *testing.T) {
cfg := applyFileConfig(t, `version: 3 cfg := applyFileConfig(t, `version: 4
concurrency: concurrency:
total_llm: 4 total_llm: 4
stage_workers: stage_workers:
@@ -35,7 +35,7 @@ debug:
} }
func TestPrecedenceOperationalEnvironmentOverridesFileValues(t *testing.T) { func TestPrecedenceOperationalEnvironmentOverridesFileValues(t *testing.T) {
cfg := applyFileConfig(t, `version: 3 cfg := applyFileConfig(t, `version: 4
concurrency: concurrency:
total_llm: 2 total_llm: 2
stage_workers: stage_workers:
@@ -81,21 +81,21 @@ func TestPrecedenceExtractWorkersFollowEffectiveConcurrencyUnlessExplicit(t *tes
}{ }{
{ {
name: "default follows environment total", name: "default follows environment total",
file: "version: 3\n", file: "version: 4\n",
env: map[string]string{"NOTARIUS_TOTAL_LLM_CONCURRENCY": "5"}, env: map[string]string{"NOTARIUS_TOTAL_LLM_CONCURRENCY": "5"},
wantTotal: 5, wantTotal: 5,
wantWorker: 5, wantWorker: 5,
}, },
{ {
name: "file worker is retained", name: "file worker is retained",
file: "version: 3\nconcurrency:\n total_llm: 3\n stage_workers:\n extract: 2\n", file: "version: 4\nconcurrency:\n total_llm: 3\n stage_workers:\n extract: 2\n",
env: map[string]string{"NOTARIUS_TOTAL_LLM_CONCURRENCY": "6"}, env: map[string]string{"NOTARIUS_TOTAL_LLM_CONCURRENCY": "6"},
wantTotal: 6, wantTotal: 6,
wantWorker: 2, wantWorker: 2,
}, },
{ {
name: "environment worker is retained", name: "environment worker is retained",
file: "version: 3\nconcurrency:\n total_llm: 2\n", file: "version: 4\nconcurrency:\n total_llm: 2\n",
env: map[string]string{ env: map[string]string{
"NOTARIUS_TOTAL_LLM_CONCURRENCY": "6", "NOTARIUS_TOTAL_LLM_CONCURRENCY": "6",
"NOTARIUS_STAGE_WORKERS_EXTRACT": "4", "NOTARIUS_STAGE_WORKERS_EXTRACT": "4",
@@ -118,7 +118,7 @@ func TestPrecedenceExtractWorkersFollowEffectiveConcurrencyUnlessExplicit(t *tes
} }
func TestPrecedenceEmptyFileCacheDirectoriesDeferPerUserResolution(t *testing.T) { func TestPrecedenceEmptyFileCacheDirectoriesDeferPerUserResolution(t *testing.T) {
cfg := applyFileConfig(t, `version: 3 cfg := applyFileConfig(t, `version: 4
cache: cache:
chunk_plans: chunk_plans:
directory: "" directory: ""

View File

@@ -14,7 +14,7 @@ import (
type FileConfig struct { type FileConfig struct {
Version int `yaml:"version"` Version int `yaml:"version"`
Scriptorium *FileScriptoriumConfig `yaml:"scriptorium,omitempty"` PromptKit *FilePromptKitConfig `yaml:"promptkit,omitempty"`
Pipelines map[string]FilePipelineProfile `yaml:"pipelines,omitempty"` Pipelines map[string]FilePipelineProfile `yaml:"pipelines,omitempty"`
Concurrency *FileConcurrencyConfig `yaml:"concurrency,omitempty"` Concurrency *FileConcurrencyConfig `yaml:"concurrency,omitempty"`
Output *FileOutputConfig `yaml:"output,omitempty"` Output *FileOutputConfig `yaml:"output,omitempty"`
@@ -22,7 +22,7 @@ type FileConfig struct {
Debug *FileDebugConfig `yaml:"debug,omitempty"` Debug *FileDebugConfig `yaml:"debug,omitempty"`
} }
type FileScriptoriumConfig struct { type FilePromptKitConfig struct {
ProfileDir *string `yaml:"profile_dir,omitempty"` ProfileDir *string `yaml:"profile_dir,omitempty"`
ProfileFile *string `yaml:"profile_file,omitempty"` ProfileFile *string `yaml:"profile_file,omitempty"`
} }
@@ -325,6 +325,9 @@ func ParseFileConfigYAML(data []byte) (FileConfig, error) {
if header.Version == 2 { if header.Version == 2 {
return FileConfig{}, fmt.Errorf("config version 2 is no longer supported; migrate the file using the version 2-to-3 migration in docs/config.md") return FileConfig{}, fmt.Errorf("config version 2 is no longer supported; migrate the file using the version 2-to-3 migration in docs/config.md")
} }
if header.Version == 3 {
return FileConfig{}, fmt.Errorf("config version 3 is no longer supported; change \"version: 3\" to \"version: 4\" and rename \"scriptorium:\" to \"promptkit:\"")
}
if header.Version != SupportedFileConfigVersion { if header.Version != SupportedFileConfigVersion {
return FileConfig{}, fmt.Errorf("unsupported config version %d (supported version is %d)", header.Version, SupportedFileConfigVersion) return FileConfig{}, fmt.Errorf("unsupported config version %d (supported version is %d)", header.Version, SupportedFileConfigVersion)
} }
@@ -450,20 +453,20 @@ func (c *Config) applyFileConfigWithLookup(fileCfg FileConfig, lookup func(strin
} }
} }
if fileCfg.Scriptorium != nil { if fileCfg.PromptKit != nil {
if fileCfg.Scriptorium.ProfileDir != nil { if fileCfg.PromptKit.ProfileDir != nil {
value := strings.TrimSpace(*fileCfg.Scriptorium.ProfileDir) value := strings.TrimSpace(*fileCfg.PromptKit.ProfileDir)
if value == "" { if value == "" {
return fmt.Errorf("scriptorium.profile_dir must not be empty when set") return fmt.Errorf("promptkit.profile_dir must not be empty when set")
} }
c.Scriptorium.ProfileDir = value c.PromptKit.ProfileDir = value
} }
if fileCfg.Scriptorium.ProfileFile != nil { if fileCfg.PromptKit.ProfileFile != nil {
value := strings.TrimSpace(*fileCfg.Scriptorium.ProfileFile) value := strings.TrimSpace(*fileCfg.PromptKit.ProfileFile)
if value == "" { if value == "" {
return fmt.Errorf("scriptorium.profile_file must not be empty when set") return fmt.Errorf("promptkit.profile_file must not be empty when set")
} }
c.Scriptorium.ProfileFile = value c.PromptKit.ProfileFile = value
} }
} }

View File

@@ -1,6 +1,7 @@
package config package config
import ( import (
"encoding/json"
"os" "os"
"path/filepath" "path/filepath"
"reflect" "reflect"
@@ -34,8 +35,8 @@ func TestDefaultReturnsDocumentedValuesAndIndependentMaps(t *testing.T) {
} }
} }
func TestFileConfigMinimalVersion3AppliesOverDefaults(t *testing.T) { func TestFileConfigMinimalVersion4AppliesOverDefaults(t *testing.T) {
file := parseFileConfig(t, "version: 3\n") file := parseFileConfig(t, "version: 4\n")
cfg := Default() cfg := Default()
if err := cfg.ApplyFileConfig(file); err != nil { if err := cfg.ApplyFileConfig(file); err != nil {
t.Fatal(err) t.Fatal(err)
@@ -48,6 +49,78 @@ func TestFileConfigMinimalVersion3AppliesOverDefaults(t *testing.T) {
} }
} }
func TestFilePromptKitProfileSourcesSurviveConfigBoundaries(t *testing.T) {
tests := []struct {
name string
yaml string
want PromptKitConfig
}{
{
name: "profile directory",
yaml: "version: 4\npromptkit:\n profile_dir: ' ./profiles '\n",
want: PromptKitConfig{ProfileDir: "./profiles"},
},
{
name: "profile file",
yaml: "version: 4\npromptkit:\n profile_file: ' ./profiles.yml '\n",
want: PromptKitConfig{ProfileFile: "./profiles.yml"},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
cfg := applyFileConfig(t, tt.yaml)
if cfg.PromptKit != tt.want {
t.Fatalf("PromptKit config = %#v, want %#v", cfg.PromptKit, tt.want)
}
if got := cloneConfig(cfg).PromptKit; got != tt.want {
t.Fatalf("cloned PromptKit config = %#v, want %#v", got, tt.want)
}
if got := cfg.Redacted().PromptKit; got != tt.want {
t.Fatalf("redacted PromptKit config = %#v, want %#v", got, tt.want)
}
data, err := json.Marshal(cfg)
if err != nil {
t.Fatalf("json.Marshal() error = %v", err)
}
var payload map[string]json.RawMessage
if err := json.Unmarshal(data, &payload); err != nil {
t.Fatalf("json.Unmarshal() error = %v", err)
}
if _, ok := payload["promptkit"]; !ok {
t.Fatalf("runtime JSON keys = %v, want promptkit", payload)
}
if _, ok := payload["scriptorium"]; ok {
t.Fatalf("runtime JSON keys = %v, must not contain removed section", payload)
}
})
}
}
func TestFilePromptKitExplicitEmptyProfileSourcesAreRejected(t *testing.T) {
for _, field := range []string{"profile_dir", "profile_file"} {
t.Run(field, func(t *testing.T) {
file := parseFileConfig(t, "version: 4\npromptkit:\n "+field+": ''\n")
cfg := Default()
err := cfg.ApplyFileConfig(file)
if err == nil || !strings.Contains(err.Error(), "promptkit."+field+" must not be empty") {
t.Fatalf("ApplyFileConfig() error = %v, want explicit-empty rejection", err)
}
})
}
}
func TestFilePromptKitProfileSourcesRemainMutuallyExclusive(t *testing.T) {
cfg := applyFileConfig(t, `version: 4
promptkit:
profile_dir: ./profiles
profile_file: ./profiles.yml
`)
if err := cfg.Validate(); err == nil || !strings.Contains(err.Error(), "promptkit profile_dir and profile_file are mutually exclusive") {
t.Fatalf("Validate() error = %v, want mutually exclusive profile sources", err)
}
}
func TestFileConfigMissingVersionIsReportedBeforeFieldDecoding(t *testing.T) { func TestFileConfigMissingVersionIsReportedBeforeFieldDecoding(t *testing.T) {
_, err := ParseFileConfigYAML([]byte("workspace:\n directory: /tmp/old\n")) _, err := ParseFileConfigYAML([]byte("workspace:\n directory: /tmp/old\n"))
if err == nil || !strings.Contains(err.Error(), "config version is required") { if err == nil || !strings.Contains(err.Error(), "config version is required") {
@@ -55,6 +128,15 @@ func TestFileConfigMissingVersionIsReportedBeforeFieldDecoding(t *testing.T) {
} }
} }
func TestFileConfigVersion3ReportsPromptKitMigration(t *testing.T) {
_, err := ParseFileConfigYAML([]byte("version: 3\nscriptorium:\n profile_dir: ./profiles\n"))
if err == nil ||
!strings.Contains(err.Error(), `change "version: 3" to "version: 4"`) ||
!strings.Contains(err.Error(), `rename "scriptorium:" to "promptkit:"`) {
t.Fatalf("version 3 error = %v, want actionable version and section migration", err)
}
}
func TestFileConfigRejectsUnknownCurrentAndRemovedFields(t *testing.T) { func TestFileConfigRejectsUnknownCurrentAndRemovedFields(t *testing.T) {
tests := []struct { tests := []struct {
name string name string
@@ -63,14 +145,19 @@ func TestFileConfigRejectsUnknownCurrentAndRemovedFields(t *testing.T) {
}{ }{
{ {
name: "removed diagnostics", name: "removed diagnostics",
yaml: "version: 3\ndiagnostics: {}\n", yaml: "version: 4\ndiagnostics: {}\n",
want: "field diagnostics not found", want: "field diagnostics not found",
}, },
{ {
name: "removed llm profiles", name: "removed llm profiles",
yaml: "version: 3\nllm_profiles: {}\n", yaml: "version: 4\nllm_profiles: {}\n",
want: "field llm_profiles not found", want: "field llm_profiles not found",
}, },
{
name: "removed scriptorium section",
yaml: "version: 4\nscriptorium: {}\n",
want: "field scriptorium not found",
},
{ {
name: "version 2 migration", name: "version 2 migration",
yaml: "version: 2\nworkspace:\n directory: /tmp/old\n", yaml: "version: 2\nworkspace:\n directory: /tmp/old\n",
@@ -78,27 +165,27 @@ func TestFileConfigRejectsUnknownCurrentAndRemovedFields(t *testing.T) {
}, },
{ {
name: "pipeline field", name: "pipeline field",
yaml: "version: 3\npipelines:\n main:\n unknown: true\n", yaml: "version: 4\npipelines:\n main:\n unknown: true\n",
want: "field unknown not found", want: "field unknown not found",
}, },
{ {
name: "lane field", name: "lane field",
yaml: "version: 3\npipelines:\n main:\n artifacts:\n spells:\n unknown: true\n", yaml: "version: 4\npipelines:\n main:\n artifacts:\n spells:\n unknown: true\n",
want: "field unknown not found", want: "field unknown not found",
}, },
{ {
name: "module binding field", name: "module binding field",
yaml: "version: 3\npipelines:\n main:\n input:\n module: seriatim\n unknown: true\n", yaml: "version: 4\npipelines:\n main:\n input:\n module: seriatim\n unknown: true\n",
want: "field unknown not found in module binding", want: "field unknown not found in module binding",
}, },
{ {
name: "checkpoint field", name: "checkpoint field",
yaml: "version: 3\ncache:\n checkpoints:\n unknown: true\n", yaml: "version: 4\ncache:\n checkpoints:\n unknown: true\n",
want: "field unknown not found", want: "field unknown not found",
}, },
{ {
name: "checkpoint enabled type", name: "checkpoint enabled type",
yaml: "version: 3\ncache:\n checkpoints:\n enabled: definitely\n", yaml: "version: 4\ncache:\n checkpoints:\n enabled: definitely\n",
want: "cannot unmarshal", want: "cannot unmarshal",
}, },
} }
@@ -113,7 +200,7 @@ func TestFileConfigRejectsUnknownCurrentAndRemovedFields(t *testing.T) {
} }
func TestFileConfigModuleBindingsPreserveFormsAndValidatorPresence(t *testing.T) { func TestFileConfigModuleBindingsPreserveFormsAndValidatorPresence(t *testing.T) {
cfg := applyFileConfig(t, `version: 3 cfg := applyFileConfig(t, `version: 4
pipelines: pipelines:
main: main:
input: seriatim input: seriatim
@@ -160,7 +247,7 @@ pipelines:
} }
func TestFileConfigReferencePrecedenceIsRetained(t *testing.T) { func TestFileConfigReferencePrecedenceIsRetained(t *testing.T) {
cfg := applyFileConfig(t, `version: 3 cfg := applyFileConfig(t, `version: 4
pipelines: pipelines:
main: main:
input: seriatim input: seriatim
@@ -224,7 +311,7 @@ pipelines:
} }
func TestFileConfigStageLocalValidatorsPreserveOrderAndFields(t *testing.T) { func TestFileConfigStageLocalValidatorsPreserveOrderAndFields(t *testing.T) {
cfg := applyFileConfig(t, `version: 3 cfg := applyFileConfig(t, `version: 4
pipelines: pipelines:
main: main:
input: seriatim input: seriatim
@@ -277,8 +364,8 @@ pipelines:
} }
func TestFileConfigStateSectionsApplyIndependently(t *testing.T) { func TestFileConfigStateSectionsApplyIndependently(t *testing.T) {
cfg := applyFileConfig(t, `version: 3 cfg := applyFileConfig(t, `version: 4
scriptorium: promptkit:
profile_dir: ./profiles profile_dir: ./profiles
concurrency: concurrency:
total_llm: 7 total_llm: 7
@@ -294,15 +381,15 @@ cache:
debug: debug:
directory: ./debug directory: ./debug
`) `)
if cfg.Scriptorium.ProfileDir != "./profiles" || cfg.Scriptorium.ProfileFile != "" { if cfg.PromptKit.ProfileDir != "./profiles" || cfg.PromptKit.ProfileFile != "" {
t.Fatalf("scriptorium = %#v", cfg.Scriptorium) t.Fatalf("promptkit = %#v", cfg.PromptKit)
} }
if cfg.Concurrency.TotalLLM != 7 || cfg.Concurrency.StageWorkers["extract"] != 7 { if cfg.Concurrency.TotalLLM != 7 || cfg.Concurrency.StageWorkers["extract"] != 7 {
t.Fatalf("concurrency = %#v", cfg.Concurrency) t.Fatalf("concurrency = %#v", cfg.Concurrency)
} }
if cfg.Output.Directory != "./output" || cfg.Cache.ChunkPlans.Directory != "plans" || cfg.Cache.ChunkPlans.Mode != pipeline.ChunkCacheBypass || if cfg.Output.Directory != "./output" || cfg.Cache.ChunkPlans.Directory != "plans" || cfg.Cache.ChunkPlans.Mode != pipeline.ChunkCacheBypass ||
!cfg.Cache.Checkpoints.Enabled || cfg.Cache.Checkpoints.Directory != "checkpoints" || cfg.Debug.Directory != "./debug" { !cfg.Cache.Checkpoints.Enabled || cfg.Cache.Checkpoints.Directory != "checkpoints" || cfg.Debug.Directory != "./debug" {
t.Fatalf("state sections = %#v, %#v, %#v, %#v", cfg.Output, cfg.Cache, cfg.Debug, cfg.Scriptorium) t.Fatalf("state sections = %#v, %#v, %#v, %#v", cfg.Output, cfg.Cache, cfg.Debug, cfg.PromptKit)
} }
if cfg.Output.Directory == cfg.Cache.ChunkPlans.Directory || cfg.Cache.ChunkPlans.Directory == cfg.Cache.Checkpoints.Directory || cfg.Cache.Checkpoints.Directory == cfg.Debug.Directory { if cfg.Output.Directory == cfg.Cache.ChunkPlans.Directory || cfg.Cache.ChunkPlans.Directory == cfg.Cache.Checkpoints.Directory || cfg.Cache.Checkpoints.Directory == cfg.Debug.Directory {
t.Fatal("state roots were coupled") t.Fatal("state roots were coupled")
@@ -310,11 +397,11 @@ debug:
} }
func TestFileConfigCheckpointEnabledCanBeExplicitlyDisabled(t *testing.T) { func TestFileConfigCheckpointEnabledCanBeExplicitlyDisabled(t *testing.T) {
cfg := applyFileConfig(t, "version: 3\ncache:\n checkpoints:\n enabled: true\n") cfg := applyFileConfig(t, "version: 4\ncache:\n checkpoints:\n enabled: true\n")
if !cfg.Cache.Checkpoints.Enabled || !cloneConfig(cfg).Cache.Checkpoints.Enabled { if !cfg.Cache.Checkpoints.Enabled || !cloneConfig(cfg).Cache.Checkpoints.Enabled {
t.Fatalf("enabled checkpoint config was not retained: %#v", cfg.Cache.Checkpoints) t.Fatalf("enabled checkpoint config was not retained: %#v", cfg.Cache.Checkpoints)
} }
file := parseFileConfig(t, "version: 3\ncache:\n checkpoints:\n enabled: false\n") file := parseFileConfig(t, "version: 4\ncache:\n checkpoints:\n enabled: false\n")
if err := cfg.ApplyFileConfig(file); err != nil { if err := cfg.ApplyFileConfig(file); err != nil {
t.Fatal(err) t.Fatal(err)
} }
@@ -331,17 +418,17 @@ func TestFileConfigRejectsTrimmedKeyCollisions(t *testing.T) {
}{ }{
{ {
name: "pipeline ids", name: "pipeline ids",
yaml: "version: 3\npipelines:\n main: {}\n ' main ': {}\n", yaml: "version: 4\npipelines:\n main: {}\n ' main ': {}\n",
want: "pipeline id \"main\" is duplicated after trimming", want: "pipeline id \"main\" is duplicated after trimming",
}, },
{ {
name: "lane ids", name: "lane ids",
yaml: "version: 3\npipelines:\n main:\n artifacts:\n spells: {}\n ' spells ': {}\n", yaml: "version: 4\npipelines:\n main:\n artifacts:\n spells: {}\n ' spells ': {}\n",
want: "artifact lane id \"spells\" is duplicated after trimming", want: "artifact lane id \"spells\" is duplicated after trimming",
}, },
{ {
name: "reference slots", name: "reference slots",
yaml: "version: 3\npipelines:\n main:\n references:\n slot: ./one.txt\n ' slot ': ./two.txt\n", yaml: "version: 4\npipelines:\n main:\n references:\n slot: ./one.txt\n ' slot ': ./two.txt\n",
want: "reference slot \"slot\" is duplicated after trimming", want: "reference slot \"slot\" is duplicated after trimming",
}, },
} }
@@ -358,7 +445,7 @@ func TestFileConfigRejectsTrimmedKeyCollisions(t *testing.T) {
} }
func TestFileConfigParsesOrderedStepsAndReferenceSources(t *testing.T) { func TestFileConfigParsesOrderedStepsAndReferenceSources(t *testing.T) {
file := parseFileConfig(t, `version: 3 file := parseFileConfig(t, `version: 4
pipelines: pipelines:
session: session:
input: seriatim input: seriatim
@@ -397,7 +484,7 @@ func TestFileConfigRejectsAmbiguousReferenceSourceForms(t *testing.T) {
"artifact: {step: 1, lane: b}", "artifact: {step: 1, lane: b}",
"1", "1",
} { } {
_, err := ParseFileConfigYAML([]byte("version: 3\npipelines:\n p:\n input: text\n references:\n slot: " + source + "\n")) _, err := ParseFileConfigYAML([]byte("version: 4\npipelines:\n p:\n input: text\n references:\n slot: " + source + "\n"))
if err == nil { if err == nil {
t.Fatalf("ParseFileConfigYAML(%q) error = nil", source) t.Fatalf("ParseFileConfigYAML(%q) error = nil", source)
} }
@@ -412,12 +499,12 @@ func TestFileConfigRejectsEmptyAndAmbiguousPipelineShapes(t *testing.T) {
}{ }{
{ {
name: "empty steps", name: "empty steps",
yaml: "version: 3\npipelines:\n p:\n input: text\n steps: []\n", yaml: "version: 4\npipelines:\n p:\n input: text\n steps: []\n",
want: "at least one ordered step", want: "at least one ordered step",
}, },
{ {
name: "both forms", name: "both forms",
yaml: "version: 3\npipelines:\n p:\n input: text\n artifacts: {}\n steps: []\n", yaml: "version: 4\npipelines:\n p:\n input: text\n artifacts: {}\n steps: []\n",
want: "both artifacts and steps", want: "both artifacts and steps",
}, },
} }

View File

@@ -10,7 +10,7 @@ import (
func (c Config) Validate() error { func (c Config) Validate() error {
c.Concurrency.recomputeStageWorkerDefaults() c.Concurrency.recomputeStageWorkerDefaults()
if err := validateScriptorium(c.Scriptorium); err != nil { if err := validatePromptKit(c.PromptKit); err != nil {
return err return err
} }
if err := validateStateSurfaces(c); err != nil { if err := validateStateSurfaces(c); err != nil {
@@ -49,9 +49,9 @@ func validateStageWorkers(cfg ConcurrencyConfig) error {
return nil return nil
} }
func validateScriptorium(cfg ScriptoriumConfig) error { func validatePromptKit(cfg PromptKitConfig) error {
if strings.TrimSpace(cfg.ProfileDir) != "" && strings.TrimSpace(cfg.ProfileFile) != "" { if strings.TrimSpace(cfg.ProfileDir) != "" && strings.TrimSpace(cfg.ProfileFile) != "" {
return fmt.Errorf("scriptorium profile_dir and profile_file are mutually exclusive") return fmt.Errorf("promptkit profile_dir and profile_file are mutually exclusive")
} }
return nil return nil
} }

View File

@@ -87,10 +87,10 @@ func TestValidateConcurrencyRules(t *testing.T) {
} }
} }
func TestValidateScriptoriumSourcesAreMutuallyExclusive(t *testing.T) { func TestValidatePromptKitSourcesAreMutuallyExclusive(t *testing.T) {
cfg := Default() cfg := Default()
cfg.Scriptorium = ScriptoriumConfig{ProfileDir: "./profiles", ProfileFile: "./profile.yml"} cfg.PromptKit = PromptKitConfig{ProfileDir: "./profiles", ProfileFile: "./profile.yml"}
assertValidationContains(t, cfg, "scriptorium profile_dir and profile_file are mutually exclusive") assertValidationContains(t, cfg, "promptkit profile_dir and profile_file are mutually exclusive")
} }
func TestValidateStateSurfaceRules(t *testing.T) { func TestValidateStateSurfaceRules(t *testing.T) {

View File

@@ -12,7 +12,7 @@ import (
"strings" "strings"
"time" "time"
"gitea.maximumdirect.net/eric/scriptorium" "gitea.maximumdirect.net/eric/promptkit"
) )
type AssetSource struct { type AssetSource struct {
@@ -72,7 +72,7 @@ func (r *AssetRegistry) SchemaFS() (fs.FS, error) {
return flattenAssetSources(r.schemas) return flattenAssetSources(r.schemas)
} }
func (r *AssetRegistry) ScriptoriumOptions() ([]scriptorium.Option, error) { func (r *AssetRegistry) PromptKitOptions() ([]promptkit.Option, error) {
promptFS, err := r.PromptFS() promptFS, err := r.PromptFS()
if err != nil { if err != nil {
return nil, fmt.Errorf("prepare prompt assets: %w", err) return nil, fmt.Errorf("prepare prompt assets: %w", err)
@@ -81,9 +81,9 @@ func (r *AssetRegistry) ScriptoriumOptions() ([]scriptorium.Option, error) {
if err != nil { if err != nil {
return nil, fmt.Errorf("prepare schema assets: %w", err) return nil, fmt.Errorf("prepare schema assets: %w", err)
} }
return []scriptorium.Option{ return []promptkit.Option{
scriptorium.WithPromptFS(promptFS, "."), promptkit.WithPromptFS(promptFS, "."),
scriptorium.WithSchemaFS(schemaFS, "."), promptkit.WithSchemaFS(schemaFS, "."),
}, nil }, nil
} }

View File

@@ -7,7 +7,7 @@ import (
"testing/fstest" "testing/fstest"
"time" "time"
"gitea.maximumdirect.net/eric/scriptorium" "gitea.maximumdirect.net/eric/promptkit"
) )
func TestAssetRegistryCombinesPromptAndSchemaSources(t *testing.T) { func TestAssetRegistryCombinesPromptAndSchemaSources(t *testing.T) {
@@ -22,12 +22,12 @@ func TestAssetRegistryCombinesPromptAndSchemaSources(t *testing.T) {
"root/schemas/out.json": {Data: []byte(`{"type":"object"}`)}, "root/schemas/out.json": {Data: []byte(`{"type":"object"}`)},
}, "root") }, "root")
engine := newAssetTestEngine(t, registry) engine := newPromptKitAssetTestEngine(t, registry)
prepared, err := engine.Prepare(context.Background(), scriptorium.RunRequest{ prepared, err := engine.Prepare(context.Background(), promptkit.RunRequest{
PromptID: "asset.test", PromptID: "asset.test",
ProfileID: "asset-test-profile", ProfileID: "asset-test-profile",
Inputs: map[string]scriptorium.ArtifactRef{ Inputs: map[string]promptkit.ArtifactRef{
"transcript": scriptorium.Inline(`{"ok":true}`), "transcript": promptkit.Inline(`{"ok":true}`),
}, },
}) })
if err != nil { if err != nil {
@@ -50,12 +50,12 @@ func TestAssetRegistryPrepareFailsForMissingPromptAsset(t *testing.T) {
"out.json": {Data: []byte(`{"type":"object"}`)}, "out.json": {Data: []byte(`{"type":"object"}`)},
}, ".") }, ".")
engine := newAssetTestEngine(t, registry) engine := newPromptKitAssetTestEngine(t, registry)
_, err := engine.Prepare(context.Background(), scriptorium.RunRequest{ _, err := engine.Prepare(context.Background(), promptkit.RunRequest{
PromptID: "asset.test", PromptID: "asset.test",
ProfileID: "asset-test-profile", ProfileID: "asset-test-profile",
Inputs: map[string]scriptorium.ArtifactRef{ Inputs: map[string]promptkit.ArtifactRef{
"transcript": scriptorium.Inline(`{"ok":true}`), "transcript": promptkit.Inline(`{"ok":true}`),
}, },
}) })
if err == nil || !strings.Contains(err.Error(), "content_file") { if err == nil || !strings.Contains(err.Error(), "content_file") {
@@ -74,12 +74,12 @@ func TestAssetRegistryPrepareFailsForMissingSchemaAsset(t *testing.T) {
"present.json": {Data: []byte(`{"type":"object"}`)}, "present.json": {Data: []byte(`{"type":"object"}`)},
}, ".") }, ".")
engine := newAssetTestEngine(t, registry) engine := newPromptKitAssetTestEngine(t, registry)
_, err := engine.Prepare(context.Background(), scriptorium.RunRequest{ _, err := engine.Prepare(context.Background(), promptkit.RunRequest{
PromptID: "asset.test", PromptID: "asset.test",
ProfileID: "asset-test-profile", ProfileID: "asset-test-profile",
Inputs: map[string]scriptorium.ArtifactRef{ Inputs: map[string]promptkit.ArtifactRef{
"transcript": scriptorium.Inline(`{"ok":true}`), "transcript": promptkit.Inline(`{"ok":true}`),
}, },
}) })
if err == nil || !strings.Contains(err.Error(), "missing.json") { if err == nil || !strings.Contains(err.Error(), "missing.json") {
@@ -145,18 +145,18 @@ func TestHashAssetsOmitsRawAssetContent(t *testing.T) {
} }
} }
func newAssetTestEngine(t *testing.T, registry *AssetRegistry) *scriptorium.Engine { func newPromptKitAssetTestEngine(t *testing.T, registry *AssetRegistry) *promptkit.Engine {
t.Helper() t.Helper()
options, err := registry.ScriptoriumOptions() options, err := registry.PromptKitOptions()
if err != nil { if err != nil {
t.Fatalf("ScriptoriumOptions() error = %v, want nil", err) t.Fatalf("PromptKitOptions() error = %v, want nil", err)
} }
options = append(options, scriptorium.WithProfiles(scriptorium.OpenAICompatibleProfile(scriptorium.OpenAICompatibleProfileConfig{ options = append(options, promptkit.WithProfiles(promptkit.OpenAICompatibleProfile(promptkit.OpenAICompatibleProfileConfig{
ID: "asset-test-profile", ID: "asset-test-profile",
Endpoint: "http://127.0.0.1:1/v1", Endpoint: "http://127.0.0.1:1/v1",
Model: "asset-test-model", Model: "asset-test-model",
}))) })))
engine, err := scriptorium.NewEngine(scriptorium.Config{Timeout: time.Second}, options...) engine, err := promptkit.NewEngine(promptkit.Config{Timeout: time.Second}, options...)
if err != nil { if err != nil {
t.Fatalf("NewEngine() error = %v, want nil", err) t.Fatalf("NewEngine() error = %v, want nil", err)
} }

View File

@@ -13,23 +13,23 @@ import (
"gitea.maximumdirect.net/eric/notarius/internal/core/artifacts" "gitea.maximumdirect.net/eric/notarius/internal/core/artifacts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts" "gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/scriptorium" "gitea.maximumdirect.net/eric/promptkit"
) )
const scriptoriumProviderName = "scriptorium" const promptKitProviderName = "promptkit"
type ScriptoriumClientConfig struct { type PromptKitClientConfig struct {
ProfileDir string ProfileDir string
ProfileFile string ProfileFile string
Assets *AssetRegistry Assets *AssetRegistry
Timeout time.Duration Timeout time.Duration
HTTPClient *http.Client HTTPClient *http.Client
EngineOptions []scriptorium.Option EngineOptions []promptkit.Option
Recorder *LLMProfileRecorder Recorder *LLMProfileRecorder
} }
type ScriptoriumClient struct { type PromptKitClient struct {
engine *scriptorium.Engine engine *promptkit.Engine
recorder *LLMProfileRecorder recorder *LLMProfileRecorder
} }
@@ -38,49 +38,49 @@ type LLMProfileRecorder struct {
profiles map[string]artifacts.LLMProfileManifest profiles map[string]artifacts.LLMProfileManifest
} }
var _ contracts.StructuredLLMClient = (*ScriptoriumClient)(nil) var _ contracts.StructuredLLMClient = (*PromptKitClient)(nil)
var _ contracts.LLMProfileManifestProvider = (*ScriptoriumClient)(nil) var _ contracts.LLMProfileManifestProvider = (*PromptKitClient)(nil)
func NewScriptoriumClient(cfg ScriptoriumClientConfig) (*ScriptoriumClient, error) { func NewPromptKitClient(cfg PromptKitClientConfig) (*PromptKitClient, error) {
if cfg.Assets == nil { if cfg.Assets == nil {
return nil, fmt.Errorf("scriptorium client assets must not be nil") return nil, fmt.Errorf("PromptKit client assets must not be nil")
} }
if strings.TrimSpace(cfg.ProfileDir) != "" && strings.TrimSpace(cfg.ProfileFile) != "" { if strings.TrimSpace(cfg.ProfileDir) != "" && strings.TrimSpace(cfg.ProfileFile) != "" {
return nil, fmt.Errorf("scriptorium profile_dir and profile_file are mutually exclusive") return nil, fmt.Errorf("PromptKit profile_dir and profile_file are mutually exclusive")
} }
options, err := cfg.Assets.ScriptoriumOptions() options, err := cfg.Assets.PromptKitOptions()
if err != nil { if err != nil {
return nil, err return nil, err
} }
if profileFile := strings.TrimSpace(cfg.ProfileFile); profileFile != "" { if profileFile := strings.TrimSpace(cfg.ProfileFile); profileFile != "" {
options = append(options, scriptorium.WithProfileFile(profileFile)) options = append(options, promptkit.WithProfileFile(profileFile))
} }
options = append(options, cfg.EngineOptions...) options = append(options, cfg.EngineOptions...)
engine, err := scriptorium.NewEngine(scriptorium.Config{ engine, err := promptkit.NewEngine(promptkit.Config{
ProfileDir: strings.TrimSpace(cfg.ProfileDir), ProfileDir: strings.TrimSpace(cfg.ProfileDir),
Timeout: cfg.Timeout, Timeout: cfg.Timeout,
HTTPClient: cfg.HTTPClient, HTTPClient: cfg.HTTPClient,
}, options...) }, options...)
if err != nil { if err != nil {
return nil, fmt.Errorf("create Scriptorium engine: %w", err) return nil, fmt.Errorf("create PromptKit engine: %w", err)
} }
recorder := cfg.Recorder recorder := cfg.Recorder
if recorder == nil { if recorder == nil {
recorder = NewLLMProfileRecorder() recorder = NewLLMProfileRecorder()
} }
return &ScriptoriumClient{ return &PromptKitClient{
engine: engine, engine: engine,
recorder: recorder, recorder: recorder,
}, nil }, nil
} }
func (c *ScriptoriumClient) CompleteStructured(ctx context.Context, req contracts.StructuredCompletionRequest, out any) (contracts.StructuredCompletionResponse, error) { func (c *PromptKitClient) CompleteStructured(ctx context.Context, req contracts.StructuredCompletionRequest, out any) (contracts.StructuredCompletionResponse, error) {
if c == nil { if c == nil {
return contracts.StructuredCompletionResponse{}, fmt.Errorf("scriptorium client must not be nil") return contracts.StructuredCompletionResponse{}, fmt.Errorf("PromptKit client must not be nil")
} }
if c.engine == nil { if c.engine == nil {
return contracts.StructuredCompletionResponse{}, fmt.Errorf("scriptorium client engine must not be nil") return contracts.StructuredCompletionResponse{}, fmt.Errorf("PromptKit client engine must not be nil")
} }
if err := validateOutputTarget(out); err != nil { if err := validateOutputTarget(out); err != nil {
return contracts.StructuredCompletionResponse{}, err return contracts.StructuredCompletionResponse{}, err
@@ -90,52 +90,52 @@ func (c *ScriptoriumClient) CompleteStructured(ctx context.Context, req contract
return contracts.StructuredCompletionResponse{}, fmt.Errorf("structured completion prompt_id must not be empty") return contracts.StructuredCompletionResponse{}, fmt.Errorf("structured completion prompt_id must not be empty")
} }
runReq := scriptorium.RunRequest{ runReq := promptkit.RunRequest{
PromptID: promptID, PromptID: promptID,
PromptVersion: strings.TrimSpace(req.PromptVersion), PromptVersion: strings.TrimSpace(req.PromptVersion),
ProfileID: strings.TrimSpace(req.ProfileID), ProfileID: strings.TrimSpace(req.ProfileID),
Inputs: scriptoriumInputs(req.Inputs), Inputs: promptKitInputs(req.Inputs),
Vars: scriptoriumVars(req), Vars: promptKitVars(req),
Metadata: scriptoriumMetadata(req), Metadata: promptKitMetadata(req),
} }
prepared, err := c.engine.Prepare(ctx, runReq) prepared, err := c.engine.Prepare(ctx, runReq)
if err != nil { if err != nil {
if ctxErr := ctx.Err(); ctxErr != nil { if ctxErr := ctx.Err(); ctxErr != nil {
return contracts.StructuredCompletionResponse{}, ctxErr return contracts.StructuredCompletionResponse{}, ctxErr
} }
return contracts.StructuredCompletionResponse{}, fmt.Errorf("prepare Scriptorium prompt %q: %w", promptID, redactScriptoriumError(err)) return contracts.StructuredCompletionResponse{}, fmt.Errorf("prepare PromptKit prompt %q: %w", promptID, redactPromptKitError(err))
} }
result, err := c.engine.Run(ctx, runReq) result, err := c.engine.Run(ctx, runReq)
if err != nil { if err != nil {
if ctxErr := ctx.Err(); ctxErr != nil { if ctxErr := ctx.Err(); ctxErr != nil {
return contracts.StructuredCompletionResponse{}, ctxErr return contracts.StructuredCompletionResponse{}, ctxErr
} }
return contracts.StructuredCompletionResponse{}, fmt.Errorf("run Scriptorium prompt %q: %w", promptID, redactScriptoriumError(err)) return contracts.StructuredCompletionResponse{}, fmt.Errorf("run PromptKit prompt %q: %w", promptID, redactPromptKitError(err))
} }
if result == nil { if result == nil {
return contracts.StructuredCompletionResponse{}, fmt.Errorf("run Scriptorium prompt %q: %w: empty result", promptID, contracts.ErrInvalidStructuredOutput) return contracts.StructuredCompletionResponse{}, fmt.Errorf("run PromptKit prompt %q: %w: empty result", promptID, contracts.ErrInvalidStructuredOutput)
} }
response := c.responseFromResult(result, prepared) response := c.responseFromResult(result, prepared)
if result.Validation.Status == scriptorium.ValidationFailed || !result.Validation.IsValid { if result.Validation.Status == promptkit.ValidationFailed || !result.Validation.IsValid {
return response, fmt.Errorf("run Scriptorium prompt %q: %w: validation failed: %s", promptID, contracts.ErrInvalidStructuredOutput, strings.Join(result.Validation.Errors, "; ")) return response, fmt.Errorf("run PromptKit prompt %q: %w: validation failed: %s", promptID, contracts.ErrInvalidStructuredOutput, strings.Join(result.Validation.Errors, "; "))
} }
if len(strings.TrimSpace(string(response.Content))) == 0 { if len(strings.TrimSpace(string(response.Content))) == 0 {
return response, fmt.Errorf("run Scriptorium prompt %q: %w: empty structured output", promptID, contracts.ErrInvalidStructuredOutput) return response, fmt.Errorf("run PromptKit prompt %q: %w: empty structured output", promptID, contracts.ErrInvalidStructuredOutput)
} }
if err := json.Unmarshal(response.Content, out); err != nil { if err := json.Unmarshal(response.Content, out); err != nil {
return response, fmt.Errorf("decode Scriptorium structured output for prompt %q: %w: %w", promptID, contracts.ErrInvalidStructuredOutput, err) return response, fmt.Errorf("decode PromptKit structured output for prompt %q: %w: %w", promptID, contracts.ErrInvalidStructuredOutput, err)
} }
return response, nil return response, nil
} }
func (c *ScriptoriumClient) responseFromResult(result *scriptorium.RunResult, prepared *scriptorium.PreparedRun) contracts.StructuredCompletionResponse { func (c *PromptKitClient) responseFromResult(result *promptkit.RunResult, prepared *promptkit.PreparedRun) contracts.StructuredCompletionResponse {
content := result.Artifact.Body content := result.Artifact.Body
if len(content) == 0 { if len(content) == 0 {
content = []byte(result.RawOutput) content = []byte(result.RawOutput)
} }
profile := artifacts.LLMProfileManifest{ profile := artifacts.LLMProfileManifest{
ID: strings.TrimSpace(result.SelectedProfileID), ID: strings.TrimSpace(result.SelectedProfileID),
Provider: scriptoriumProviderName, Provider: promptKitProviderName,
Model: firstNonEmpty(result.ModelName, result.EffectiveModelParams.Model), Model: firstNonEmpty(result.ModelName, result.EffectiveModelParams.Model),
} }
if c.recorder != nil { if c.recorder != nil {
@@ -149,17 +149,17 @@ func (c *ScriptoriumClient) responseFromResult(result *scriptorium.RunResult, pr
PromptTokens: result.Usage.PromptTokens, PromptTokens: result.Usage.PromptTokens,
CompletionTokens: result.Usage.CompletionTokens, CompletionTokens: result.Usage.CompletionTokens,
TotalTokens: result.Usage.TotalTokens, TotalTokens: result.Usage.TotalTokens,
Debug: scriptoriumDebugMaterial(prepared, result), Debug: promptKitDebugMaterial(prepared, result),
} }
} }
func scriptoriumDebugMaterial(prepared *scriptorium.PreparedRun, result *scriptorium.RunResult) *contracts.LLMDebugMaterial { func promptKitDebugMaterial(prepared *promptkit.PreparedRun, result *promptkit.RunResult) *contracts.LLMDebugMaterial {
material := &contracts.LLMDebugMaterial{} material := &contracts.LLMDebugMaterial{}
if prepared != nil { if prepared != nil {
material.Prompt = scriptoriumDebugPrompt(prepared) material.Prompt = promptKitDebugPrompt(prepared)
} }
if result != nil { if result != nil {
material.Response = scriptoriumDebugResponse(result) material.Response = promptKitDebugResponse(result)
} }
if material.Prompt == nil && material.Response == nil { if material.Prompt == nil && material.Response == nil {
return nil return nil
@@ -167,7 +167,7 @@ func scriptoriumDebugMaterial(prepared *scriptorium.PreparedRun, result *scripto
return material return material
} }
func scriptoriumDebugPrompt(prepared *scriptorium.PreparedRun) *contracts.LLMDebugPrompt { func promptKitDebugPrompt(prepared *promptkit.PreparedRun) *contracts.LLMDebugPrompt {
if prepared == nil { if prepared == nil {
return nil return nil
} }
@@ -194,7 +194,7 @@ func scriptoriumDebugPrompt(prepared *scriptorium.PreparedRun) *contracts.LLMDeb
} }
} }
func scriptoriumDebugResponse(result *scriptorium.RunResult) *contracts.LLMDebugResponse { func promptKitDebugResponse(result *promptkit.RunResult) *contracts.LLMDebugResponse {
if result == nil { if result == nil {
return nil return nil
} }
@@ -254,7 +254,7 @@ func cloneStringMap(values map[string]string) map[string]string {
return out return out
} }
func (c *ScriptoriumClient) LLMProfileManifests() []artifacts.LLMProfileManifest { func (c *PromptKitClient) LLMProfileManifests() []artifacts.LLMProfileManifest {
if c == nil || c.recorder == nil { if c == nil || c.recorder == nil {
return nil return nil
} }
@@ -302,11 +302,11 @@ func (r *LLMProfileRecorder) Manifests() []artifacts.LLMProfileManifest {
return out return out
} }
func scriptoriumInputs(inputs contracts.LLMInputSet) map[string]scriptorium.ArtifactRef { func promptKitInputs(inputs contracts.LLMInputSet) map[string]promptkit.ArtifactRef {
if len(inputs) == 0 { if len(inputs) == 0 {
return nil return nil
} }
out := make(map[string]scriptorium.ArtifactRef, len(inputs)) out := make(map[string]promptkit.ArtifactRef, len(inputs))
for key, material := range inputs { for key, material := range inputs {
name := strings.TrimSpace(key) name := strings.TrimSpace(key)
if name == "" { if name == "" {
@@ -320,15 +320,15 @@ func scriptoriumInputs(inputs contracts.LLMInputSet) map[string]scriptorium.Arti
body = " " body = " "
} }
if origin := strings.TrimSpace(material.OriginURI); origin != "" { if origin := strings.TrimSpace(material.OriginURI); origin != "" {
out[name] = scriptorium.InlineWithURI(origin, body) out[name] = promptkit.InlineWithURI(origin, body)
} else { } else {
out[name] = scriptorium.Inline(body) out[name] = promptkit.Inline(body)
} }
} }
return out return out
} }
func scriptoriumVars(req contracts.StructuredCompletionRequest) map[string]string { func promptKitVars(req contracts.StructuredCompletionRequest) map[string]string {
vars := make(map[string]string, len(req.Vars)+1) vars := make(map[string]string, len(req.Vars)+1)
for key, value := range req.Vars { for key, value := range req.Vars {
name := strings.TrimSpace(key) name := strings.TrimSpace(key)
@@ -346,7 +346,7 @@ func scriptoriumVars(req contracts.StructuredCompletionRequest) map[string]strin
return vars return vars
} }
func scriptoriumMetadata(req contracts.StructuredCompletionRequest) map[string]string { func promptKitMetadata(req contracts.StructuredCompletionRequest) map[string]string {
metadata := map[string]string{} metadata := map[string]string{}
if stageName := strings.TrimSpace(req.StageName); stageName != "" { if stageName := strings.TrimSpace(req.StageName); stageName != "" {
metadata["stage_name"] = stageName metadata["stage_name"] = stageName
@@ -359,7 +359,7 @@ func scriptoriumMetadata(req contracts.StructuredCompletionRequest) map[string]s
var bearerTokenPattern = regexp.MustCompile(`(?i)Bearer\s+[A-Za-z0-9._~+/=-]+`) var bearerTokenPattern = regexp.MustCompile(`(?i)Bearer\s+[A-Za-z0-9._~+/=-]+`)
func redactScriptoriumError(err error) error { func redactPromptKitError(err error) error {
if err == nil { if err == nil {
return nil return nil
} }

View File

@@ -4,6 +4,8 @@ import (
"context" "context"
"encoding/json" "encoding/json"
"errors" "errors"
"io"
"net/http"
"strings" "strings"
"sync" "sync"
"sync/atomic" "sync/atomic"
@@ -12,12 +14,12 @@ import (
"time" "time"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts" "gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/scriptorium" "gitea.maximumdirect.net/eric/promptkit"
) )
func TestScriptoriumClientMapsPromptRequestAndUnmarshalsOutput(t *testing.T) { func TestPromptKitClientMapsPromptRequestAndUnmarshalsOutput(t *testing.T) {
fake := &fakeScriptoriumLLM{content: `{"ok":true}`} fake := &fakePromptKitLLM{content: `{"ok":true}`}
client := newTestScriptoriumClient(t, fake) client := newTestPromptKitClient(t, fake)
var out struct { var out struct {
OK bool `json:"ok"` OK bool `json:"ok"`
@@ -39,7 +41,7 @@ func TestScriptoriumClientMapsPromptRequestAndUnmarshalsOutput(t *testing.T) {
if !out.OK { if !out.OK {
t.Fatalf("decoded output OK = false, want true") t.Fatalf("decoded output OK = false, want true")
} }
if resp.Provider != scriptoriumProviderName || resp.Model != "explicit-model" || resp.ProfileID != "explicit-profile" { if resp.Provider != "promptkit" || resp.Model != "explicit-model" || resp.ProfileID != "explicit-profile" {
t.Fatalf("response metadata = %#v", resp) t.Fatalf("response metadata = %#v", resp)
} }
if resp.PromptTokens != 11 || resp.CompletionTokens != 7 || resp.TotalTokens != 18 { if resp.PromptTokens != 11 || resp.CompletionTokens != 7 || resp.TotalTokens != 18 {
@@ -57,6 +59,9 @@ func TestScriptoriumClientMapsPromptRequestAndUnmarshalsOutput(t *testing.T) {
if resp.Debug.Response == nil || resp.Debug.Response.Content != `{"ok":true}` { if resp.Debug.Response == nil || resp.Debug.Response.Content != `{"ok":true}` {
t.Fatalf("debug response = %#v, want raw response content", resp.Debug.Response) t.Fatalf("debug response = %#v, want raw response content", resp.Debug.Response)
} }
if resp.Debug.Response.Usage.CachedTokens != 5 || resp.Debug.Response.Usage.CacheWriteTokens != 3 {
t.Fatalf("debug usage = %#v, want cached token counts", resp.Debug.Response.Usage)
}
debugJSON, err := json.Marshal(resp.Debug) debugJSON, err := json.Marshal(resp.Debug)
if err != nil { if err != nil {
t.Fatalf("marshal debug material: %v", err) t.Fatalf("marshal debug material: %v", err)
@@ -71,21 +76,52 @@ func TestScriptoriumClientMapsPromptRequestAndUnmarshalsOutput(t *testing.T) {
if gotReq.Target.Model != "explicit-model" { if gotReq.Target.Model != "explicit-model" {
t.Fatalf("model = %q, want explicit-model", gotReq.Target.Model) t.Fatalf("model = %q, want explicit-model", gotReq.Target.Model)
} }
if len(gotReq.Prompt.Messages) != 1 || !strings.Contains(gotReq.Prompt.Messages[0].Content, `{"source":true}`) { if len(gotReq.Prompt.Messages) != 1 ||
t.Fatalf("rendered messages = %#v, want transcript input content", gotReq.Prompt.Messages) !strings.Contains(gotReq.Prompt.Messages[0].Content, `{"source":true}`) ||
!strings.Contains(gotReq.Prompt.Messages[0].Content, "value") {
t.Fatalf("rendered messages = %#v, want transcript input and variable content", gotReq.Prompt.Messages)
} }
if gotReq.StructuredOutput == nil { if gotReq.StructuredOutput == nil {
t.Fatalf("structured output = nil, want JSON schema") t.Fatalf("structured output = nil, want JSON schema")
} }
manifests := client.LLMProfileManifests() manifests := client.LLMProfileManifests()
if len(manifests) != 1 || manifests[0].ID != "explicit-profile" || manifests[0].Model != "explicit-model" { if len(manifests) != 1 ||
manifests[0].ID != "explicit-profile" ||
manifests[0].Provider != "promptkit" ||
manifests[0].Model != "explicit-model" {
t.Fatalf("profile manifests = %#v", manifests) t.Fatalf("profile manifests = %#v", manifests)
} }
} }
func TestScriptoriumClientUsesPromptDefaultProfileWhenRequestProfileEmpty(t *testing.T) { func TestNewPromptKitClientReportsAssetAndEngineConstructionFailures(t *testing.T) {
fake := &fakeScriptoriumLLM{content: `{"ok":true}`} t.Run("assets", func(t *testing.T) {
client := newTestScriptoriumClient(t, fake) registry := NewAssetRegistry()
for _, root := range []string{"one", "two"} {
if err := registry.RegisterPromptFS(fstest.MapFS{
root + "/prompt.yaml": {Data: []byte("id: duplicate")},
}, root); err != nil {
t.Fatalf("RegisterPromptFS() error = %v", err)
}
}
if _, err := NewPromptKitClient(PromptKitClientConfig{Assets: registry}); err == nil ||
!strings.Contains(err.Error(), "duplicate asset path") {
t.Fatalf("NewPromptKitClient() error = %v, want asset construction failure", err)
}
})
t.Run("engine", func(t *testing.T) {
if _, err := NewPromptKitClient(PromptKitClientConfig{
Assets: newTestPromptKitAssets(t),
EngineOptions: []promptkit.Option{promptkit.WithProfileFile("")},
}); err == nil || !strings.Contains(err.Error(), "create PromptKit engine") {
t.Fatalf("NewPromptKitClient() error = %v, want engine construction failure", err)
}
})
}
func TestPromptKitClientUsesPromptDefaultProfileWhenRequestProfileEmpty(t *testing.T) {
fake := &fakePromptKitLLM{content: `{"ok":true}`}
client := newTestPromptKitClient(t, fake)
var out map[string]any var out map[string]any
if _, err := client.CompleteStructured(context.Background(), contracts.StructuredCompletionRequest{ if _, err := client.CompleteStructured(context.Background(), contracts.StructuredCompletionRequest{
@@ -102,8 +138,8 @@ func TestScriptoriumClientUsesPromptDefaultProfileWhenRequestProfileEmpty(t *tes
} }
} }
func TestScriptoriumClientValidationFailureReturnsError(t *testing.T) { func TestPromptKitClientValidationFailureReturnsError(t *testing.T) {
client := newTestScriptoriumClient(t, &fakeScriptoriumLLM{content: `{"bad":true}`}) client := newTestPromptKitClient(t, &fakePromptKitLLM{content: `{"bad":true}`})
var out map[string]any var out map[string]any
resp, err := client.CompleteStructured(context.Background(), contracts.StructuredCompletionRequest{ resp, err := client.CompleteStructured(context.Background(), contracts.StructuredCompletionRequest{
@@ -127,8 +163,8 @@ func TestScriptoriumClientValidationFailureReturnsError(t *testing.T) {
} }
} }
func TestScriptoriumClientDecodeFailureReturnsRawResponse(t *testing.T) { func TestPromptKitClientDecodeFailureReturnsRawResponse(t *testing.T) {
client := newTestScriptoriumClient(t, &fakeScriptoriumLLM{content: `{"ok":true}`}) client := newTestPromptKitClient(t, &fakePromptKitLLM{content: `{"ok":true}`})
var out []any var out []any
resp, err := client.CompleteStructured(context.Background(), contracts.StructuredCompletionRequest{ resp, err := client.CompleteStructured(context.Background(), contracts.StructuredCompletionRequest{
@@ -138,7 +174,7 @@ func TestScriptoriumClientDecodeFailureReturnsRawResponse(t *testing.T) {
"transcript": contracts.NewLLMInputMaterial("transcript", "application/json", []byte(`{"source":true}`), "", ""), "transcript": contracts.NewLLMInputMaterial("transcript", "application/json", []byte(`{"source":true}`), "", ""),
}, },
}, &out) }, &out)
if err == nil || !errors.Is(err, contracts.ErrInvalidStructuredOutput) || !strings.Contains(err.Error(), "decode Scriptorium structured output") { if err == nil || !errors.Is(err, contracts.ErrInvalidStructuredOutput) || !strings.Contains(err.Error(), "decode PromptKit structured output") {
t.Fatalf("CompleteStructured() error = %v, want decode failure", err) t.Fatalf("CompleteStructured() error = %v, want decode failure", err)
} }
if got := string(resp.Content); got != `{"ok":true}` { if got := string(resp.Content); got != `{"ok":true}` {
@@ -149,8 +185,8 @@ func TestScriptoriumClientDecodeFailureReturnsRawResponse(t *testing.T) {
} }
} }
func TestScriptoriumClientProviderFailureIncludesContextAndRedactsBearerToken(t *testing.T) { func TestPromptKitClientProviderFailureIncludesContextAndRedactsBearerToken(t *testing.T) {
client := newTestScriptoriumClient(t, &fakeScriptoriumLLM{err: errors.New("provider failed with Bearer secret-token")}) client := newTestPromptKitClient(t, &fakePromptKitLLM{err: errors.New("provider failed with Bearer secret-token")})
var out map[string]any var out map[string]any
resp, err := client.CompleteStructured(context.Background(), contracts.StructuredCompletionRequest{ resp, err := client.CompleteStructured(context.Background(), contracts.StructuredCompletionRequest{
@@ -166,7 +202,7 @@ func TestScriptoriumClientProviderFailureIncludesContextAndRedactsBearerToken(t
if errors.Is(err, contracts.ErrInvalidStructuredOutput) { if errors.Is(err, contracts.ErrInvalidStructuredOutput) {
t.Fatalf("provider error = %v, must not be classified as invalid structured output", err) t.Fatalf("provider error = %v, must not be classified as invalid structured output", err)
} }
if !strings.Contains(err.Error(), `run Scriptorium prompt "adapter.test"`) { if !strings.Contains(err.Error(), `run PromptKit prompt "adapter.test"`) {
t.Fatalf("error = %q, want operation context", err.Error()) t.Fatalf("error = %q, want operation context", err.Error())
} }
if strings.Contains(err.Error(), "secret-token") || !strings.Contains(err.Error(), "Bearer [REDACTED]") { if strings.Contains(err.Error(), "secret-token") || !strings.Contains(err.Error(), "Bearer [REDACTED]") {
@@ -177,10 +213,10 @@ func TestScriptoriumClientProviderFailureIncludesContextAndRedactsBearerToken(t
} }
} }
func TestScriptoriumClientContextCancellationIsRespected(t *testing.T) { func TestPromptKitClientContextCancellationIsRespected(t *testing.T) {
ctx, cancel := context.WithCancel(context.Background()) ctx, cancel := context.WithCancel(context.Background())
cancel() cancel()
client := newTestScriptoriumClient(t, &fakeScriptoriumLLM{content: `{"ok":true}`}) client := newTestPromptKitClient(t, &fakePromptKitLLM{content: `{"ok":true}`})
var out map[string]any var out map[string]any
_, err := client.CompleteStructured(ctx, contracts.StructuredCompletionRequest{ _, err := client.CompleteStructured(ctx, contracts.StructuredCompletionRequest{
@@ -194,8 +230,57 @@ func TestScriptoriumClientContextCancellationIsRespected(t *testing.T) {
} }
} }
func TestScriptoriumClientClassifiesEmptyStructuredCompletion(t *testing.T) { func TestPromptKitClientForwardsConfiguredTransportTimeout(t *testing.T) {
client := newTestScriptoriumClient(t, &fakeScriptoriumLLM{allowEmpty: true}) var remaining time.Duration
transport := roundTripFunc(func(req *http.Request) (*http.Response, error) {
deadline, ok := req.Context().Deadline()
if !ok {
t.Fatal("outbound request context has no deadline")
}
remaining = time.Until(deadline)
return &http.Response{
StatusCode: http.StatusOK,
Status: "200 OK",
Header: make(http.Header),
Body: io.NopCloser(strings.NewReader(
`{"choices":[{"message":{"content":"{\"ok\":true}"}}]}`,
)),
Request: req,
}, nil
})
const configuredTimeout = 2 * time.Second
client, err := NewPromptKitClient(PromptKitClientConfig{
Assets: newTestPromptKitAssets(t),
Timeout: configuredTimeout,
HTTPClient: &http.Client{Transport: transport},
EngineOptions: []promptkit.Option{
promptkit.WithProfiles(promptkit.OpenAICompatibleProfile(promptkit.OpenAICompatibleProfileConfig{
ID: "default-profile",
Endpoint: "http://promptkit.test/v1",
Model: "default-model",
})),
},
})
if err != nil {
t.Fatalf("NewPromptKitClient() error = %v", err)
}
var out map[string]any
if _, err := client.CompleteStructured(context.Background(), contracts.StructuredCompletionRequest{
PromptID: "adapter.test",
SessionID: "session-123",
Inputs: contracts.LLMInputSet{
"transcript": contracts.NewLLMInputMaterial("transcript", "application/json", []byte(`{"source":true}`), "", ""),
},
}, &out); err != nil {
t.Fatalf("CompleteStructured() error = %v", err)
}
if remaining < configuredTimeout-time.Second || remaining > configuredTimeout {
t.Fatalf("transport deadline remaining = %v, want near %v", remaining, configuredTimeout)
}
}
func TestPromptKitClientClassifiesEmptyStructuredCompletion(t *testing.T) {
client := newTestPromptKitClient(t, &fakePromptKitLLM{allowEmpty: true})
var out map[string]any var out map[string]any
_, err := client.CompleteStructured(context.Background(), contracts.StructuredCompletionRequest{ _, err := client.CompleteStructured(context.Background(), contracts.StructuredCompletionRequest{
@@ -210,12 +295,12 @@ func TestScriptoriumClientClassifiesEmptyStructuredCompletion(t *testing.T) {
} }
} }
func TestScheduledScriptoriumClientBoundsConcurrentCalls(t *testing.T) { func TestScheduledPromptKitClientBoundsConcurrentCalls(t *testing.T) {
fake := &fakeScriptoriumLLM{ fake := &fakePromptKitLLM{
content: `{"ok":true}`, content: `{"ok":true}`,
block: make(chan struct{}), block: make(chan struct{}),
} }
client := newTestScriptoriumClient(t, fake) client := newTestPromptKitClient(t, fake)
scheduler, err := NewScheduler(1) scheduler, err := NewScheduler(1)
if err != nil { if err != nil {
t.Fatalf("NewScheduler() error = %v, want nil", err) t.Fatalf("NewScheduler() error = %v, want nil", err)
@@ -249,8 +334,8 @@ func TestScheduledScriptoriumClientBoundsConcurrentCalls(t *testing.T) {
wg.Wait() wg.Wait()
} }
func TestScriptoriumClientValidatesRequest(t *testing.T) { func TestPromptKitClientValidatesRequest(t *testing.T) {
client := newTestScriptoriumClient(t, &fakeScriptoriumLLM{content: `{"ok":true}`}) client := newTestPromptKitClient(t, &fakePromptKitLLM{content: `{"ok":true}`})
var out map[string]any var out map[string]any
if _, err := client.CompleteStructured(context.Background(), contracts.StructuredCompletionRequest{}, &out); err == nil || !strings.Contains(err.Error(), "prompt_id") { if _, err := client.CompleteStructured(context.Background(), contracts.StructuredCompletionRequest{}, &out); err == nil || !strings.Contains(err.Error(), "prompt_id") {
t.Fatalf("missing prompt id error = %v, want prompt_id validation", err) t.Fatalf("missing prompt id error = %v, want prompt_id validation", err)
@@ -260,7 +345,34 @@ func TestScriptoriumClientValidatesRequest(t *testing.T) {
} }
} }
func newTestScriptoriumClient(t *testing.T, fake *fakeScriptoriumLLM) *ScriptoriumClient { func newTestPromptKitClient(t *testing.T, fake *fakePromptKitLLM) *PromptKitClient {
t.Helper()
registry := newTestPromptKitAssets(t)
client, err := NewPromptKitClient(PromptKitClientConfig{
Assets: registry,
EngineOptions: []promptkit.Option{
promptkit.WithProfiles(
promptkit.OpenAICompatibleProfile(promptkit.OpenAICompatibleProfileConfig{
ID: "default-profile",
Endpoint: "http://127.0.0.1:1/v1",
Model: "default-model",
}),
promptkit.OpenAICompatibleProfile(promptkit.OpenAICompatibleProfileConfig{
ID: "explicit-profile",
Endpoint: "http://127.0.0.1:1/v1",
Model: "explicit-model",
}),
),
promptkit.WithLLMClient(fake),
},
})
if err != nil {
t.Fatalf("NewPromptKitClient() error = %v, want nil", err)
}
return client
}
func newTestPromptKitAssets(t *testing.T) *AssetRegistry {
t.Helper() t.Helper()
registry := NewAssetRegistry() registry := NewAssetRegistry()
if err := registry.RegisterPromptFS(fstest.MapFS{ if err := registry.RegisterPromptFS(fstest.MapFS{
@@ -274,7 +386,7 @@ inputs:
content_type: application/json content_type: application/json
messages: messages:
- role: user - role: user
content: "Transcript: {{ input \"transcript\" }}" content: "Transcript: {{ input \"transcript\" }} Custom: {{ index . \"custom\" }}"
output: output:
format: json format: json
validation_mode: json_schema validation_mode: json_schema
@@ -289,43 +401,22 @@ output:
}, "."); err != nil { }, "."); err != nil {
t.Fatalf("RegisterSchemaFS() error = %v", err) t.Fatalf("RegisterSchemaFS() error = %v", err)
} }
client, err := NewScriptoriumClient(ScriptoriumClientConfig{ return registry
Assets: registry,
EngineOptions: []scriptorium.Option{
scriptorium.WithProfiles(
scriptorium.OpenAICompatibleProfile(scriptorium.OpenAICompatibleProfileConfig{
ID: "default-profile",
Endpoint: "http://127.0.0.1:1/v1",
Model: "default-model",
}),
scriptorium.OpenAICompatibleProfile(scriptorium.OpenAICompatibleProfileConfig{
ID: "explicit-profile",
Endpoint: "http://127.0.0.1:1/v1",
Model: "explicit-model",
}),
),
scriptorium.WithLLMClient(fake),
},
})
if err != nil {
t.Fatalf("NewScriptoriumClient() error = %v, want nil", err)
}
return client
} }
type fakeScriptoriumLLM struct { type fakePromptKitLLM struct {
content string content string
allowEmpty bool allowEmpty bool
err error err error
block chan struct{} block chan struct{}
mu sync.Mutex mu sync.Mutex
last scriptorium.GenerateRequest last promptkit.GenerateRequest
calls int32 calls int32
inFlight int32 inFlight int32
maxInFlight int32 maxInFlight int32
} }
func (f *fakeScriptoriumLLM) Generate(ctx context.Context, req scriptorium.GenerateRequest) (*scriptorium.GenerateResponse, error) { func (f *fakePromptKitLLM) Generate(ctx context.Context, req promptkit.GenerateRequest) (*promptkit.GenerateResponse, error) {
f.mu.Lock() f.mu.Lock()
f.last = req f.last = req
f.mu.Unlock() f.mu.Unlock()
@@ -355,17 +446,25 @@ func (f *fakeScriptoriumLLM) Generate(ctx context.Context, req scriptorium.Gener
if !f.allowEmpty && !json.Valid([]byte(content)) { if !f.allowEmpty && !json.Valid([]byte(content)) {
return nil, errors.New("test fake must return JSON content") return nil, errors.New("test fake must return JSON content")
} }
return &scriptorium.GenerateResponse{ return &promptkit.GenerateResponse{
Content: content, Content: content,
Usage: scriptorium.TokenUsage{ Usage: promptkit.TokenUsage{
PromptTokens: 11, PromptTokens: 11,
CompletionTokens: 7, CompletionTokens: 7,
TotalTokens: 18, TotalTokens: 18,
CachedTokens: 5,
CacheWriteTokens: 3,
}, },
}, nil }, nil
} }
func (f *fakeScriptoriumLLM) lastRequest() scriptorium.GenerateRequest { type roundTripFunc func(*http.Request) (*http.Response, error)
func (f roundTripFunc) RoundTrip(req *http.Request) (*http.Response, error) {
return f(req)
}
func (f *fakePromptKitLLM) lastRequest() promptkit.GenerateRequest {
f.mu.Lock() f.mu.Lock()
defer f.mu.Unlock() defer f.mu.Unlock()
return f.last return f.last

View File

@@ -12,14 +12,14 @@ import (
) )
// ModulePromptFile maps a module-owned embedded prompt file into the // ModulePromptFile maps a module-owned embedded prompt file into the
// Scriptorium-visible module prompt directory. // PromptKit-visible module prompt directory.
type ModulePromptFile struct { type ModulePromptFile struct {
Name string Name string
Path string Path string
} }
// SharedPromptFile maps a caller-owned shared prompt file into a module's // SharedPromptFile maps a caller-owned shared prompt file into a module's
// Scriptorium-visible sharedassets prompt subdirectory. // PromptKit-visible sharedassets prompt subdirectory.
type SharedPromptFile struct { type SharedPromptFile struct {
Name string Name string
FS fs.FS FS fs.FS

View File

@@ -56,7 +56,7 @@ func (c *Chunker) ReferenceSlots() []contracts.ReferenceSlot {
} }
func (c *Chunker) ManifestMetadata() map[string]any { func (c *Chunker) ManifestMetadata() map[string]any {
promptSHA, err := scriptoriumPromptMetadata() promptSHA, err := promptAssetMetadata()
if err != nil { if err != nil {
promptSHA = "" promptSHA = ""
} }

View File

@@ -9,7 +9,7 @@ import (
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared" "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
) )
const scriptoriumPromptRoot = "assets/prompts" const promptAssetRoot = "assets/prompts"
var promptAssetManifest = shared.PromptAssetManifest{ var promptAssetManifest = shared.PromptAssetManifest{
ModuleDir: "dnd.scenes", ModuleDir: "dnd.scenes",
@@ -30,21 +30,21 @@ func RegisterPromptAssets(registry *llm.AssetRegistry) error {
if err != nil { if err != nil {
return fmt.Errorf("prepare scene prompt assets: %w", err) return fmt.Errorf("prepare scene prompt assets: %w", err)
} }
if err := registry.RegisterPromptFS(promptFS, scriptoriumPromptRoot); err != nil { if err := registry.RegisterPromptFS(promptFS, promptAssetRoot); err != nil {
return err return err
} }
return registry.RegisterSchemaFS(embeddedAssets, "assets/schemas") return registry.RegisterSchemaFS(embeddedAssets, "assets/schemas")
} }
func scriptoriumPromptMetadata() (string, error) { func promptAssetMetadata() (string, error) {
scriptoriumPromptHashOnce.Do(func() { promptAssetHashOnce.Do(func() {
scriptoriumPromptHash, scriptoriumPromptHashErr = promptAssetManifest.Hash(embeddedAssets) promptAssetHash, promptAssetHashErr = promptAssetManifest.Hash(embeddedAssets)
}) })
return scriptoriumPromptHash, scriptoriumPromptHashErr return promptAssetHash, promptAssetHashErr
} }
var ( var (
scriptoriumPromptHashOnce sync.Once promptAssetHashOnce sync.Once
scriptoriumPromptHash string promptAssetHash string
scriptoriumPromptHashErr error promptAssetHashErr error
) )

View File

@@ -8,10 +8,10 @@ import (
"time" "time"
"gitea.maximumdirect.net/eric/notarius/internal/framework/llm" "gitea.maximumdirect.net/eric/notarius/internal/framework/llm"
"gitea.maximumdirect.net/eric/scriptorium" "gitea.maximumdirect.net/eric/promptkit"
) )
func TestScriptoriumPromptPreparesTranscriptAndTaskMessages(t *testing.T) { func TestPromptAssetsPrepareTranscriptAndTaskMessages(t *testing.T) {
transcript := []byte(`{"sentinel":"scene-transcript"}`) transcript := []byte(`{"sentinel":"scene-transcript"}`)
prepared := prepareScenesPrompt(t, transcript, "scene-players", "scene-party", "scene-glossary") prepared := prepareScenesPrompt(t, transcript, "scene-players", "scene-party", "scene-glossary")
@@ -27,7 +27,7 @@ func TestScriptoriumPromptPreparesTranscriptAndTaskMessages(t *testing.T) {
} }
} }
for _, index := range []int{1, 4} { for _, index := range []int{1, 4} {
if cache := prepared.Messages[index].CacheControl; cache == nil || cache.Type != scriptorium.CacheControlEphemeral { if cache := prepared.Messages[index].CacheControl; cache == nil || cache.Type != promptkit.CacheControlEphemeral {
t.Errorf("message %d cache control = %#v, want ephemeral", index, cache) t.Errorf("message %d cache control = %#v, want ephemeral", index, cache)
} }
} }
@@ -52,7 +52,7 @@ func TestScriptoriumPromptPreparesTranscriptAndTaskMessages(t *testing.T) {
} }
} }
func TestScriptoriumPromptDiagnosticsOmitRawMaterials(t *testing.T) { func TestPromptAssetDiagnosticsOmitRawMaterials(t *testing.T) {
transcript := []byte(`{"secret":"source text"}`) transcript := []byte(`{"secret":"source text"}`)
prepared := prepareScenesPrompt(t, transcript, "private player note", "private party note", "private glossary note") prepared := prepareScenesPrompt(t, transcript, "private player note", "private party note", "private glossary note")
metadata := newChunker(t, &fakeScenesLLMClient{}).ManifestMetadata() metadata := newChunker(t, &fakeScenesLLMClient{}).ManifestMetadata()
@@ -94,22 +94,22 @@ func TestScriptoriumPromptDiagnosticsOmitRawMaterials(t *testing.T) {
} }
} }
func prepareScenesPrompt(t *testing.T, transcript []byte, players string, party string, glossary string) *scriptorium.PreparedRun { func prepareScenesPrompt(t *testing.T, transcript []byte, players string, party string, glossary string) *promptkit.PreparedRun {
t.Helper() t.Helper()
registry := llm.NewAssetRegistry() registry := llm.NewAssetRegistry()
if err := RegisterPromptAssets(registry); err != nil { if err := RegisterPromptAssets(registry); err != nil {
t.Fatalf("register scene prompt assets: %v", err) t.Fatalf("register scene prompt assets: %v", err)
} }
engine := newScenesScriptoriumEngine(t, registry) engine := newScenesPromptEngine(t, registry)
prepared, err := engine.Prepare(context.Background(), scriptorium.RunRequest{ prepared, err := engine.Prepare(context.Background(), promptkit.RunRequest{
PromptID: PromptID, PromptID: PromptID,
PromptVersion: ResponseSchemaVersion, PromptVersion: ResponseSchemaVersion,
ProfileID: "scene-test-profile", ProfileID: "scene-test-profile",
Inputs: map[string]scriptorium.ArtifactRef{ Inputs: map[string]promptkit.ArtifactRef{
"transcript": scriptorium.InlineWithURI("file:///session.json", string(transcript)), "transcript": promptkit.InlineWithURI("file:///session.json", string(transcript)),
"players": scriptorium.Inline(players), "players": promptkit.Inline(players),
"party": scriptorium.Inline(party), "party": promptkit.Inline(party),
"glossary": scriptorium.Inline(glossary), "glossary": promptkit.Inline(glossary),
}, },
}) })
if err != nil { if err != nil {
@@ -118,18 +118,18 @@ func prepareScenesPrompt(t *testing.T, transcript []byte, players string, party
return prepared return prepared
} }
func newScenesScriptoriumEngine(t *testing.T, registry *llm.AssetRegistry) *scriptorium.Engine { func newScenesPromptEngine(t *testing.T, registry *llm.AssetRegistry) *promptkit.Engine {
t.Helper() t.Helper()
options, err := registry.ScriptoriumOptions() options, err := registry.PromptKitOptions()
if err != nil { if err != nil {
t.Fatalf("ScriptoriumOptions() error = %v, want nil", err) t.Fatalf("PromptKitOptions() error = %v, want nil", err)
} }
options = append(options, scriptorium.WithProfiles(scriptorium.OpenAICompatibleProfile(scriptorium.OpenAICompatibleProfileConfig{ options = append(options, promptkit.WithProfiles(promptkit.OpenAICompatibleProfile(promptkit.OpenAICompatibleProfileConfig{
ID: "scene-test-profile", ID: "scene-test-profile",
Endpoint: "http://127.0.0.1:1/v1", Endpoint: "http://127.0.0.1:1/v1",
Model: "scene-test-model", Model: "scene-test-model",
}))) })))
engine, err := scriptorium.NewEngine(scriptorium.Config{Timeout: time.Second}, options...) engine, err := promptkit.NewEngine(promptkit.Config{Timeout: time.Second}, options...)
if err != nil { if err != nil {
t.Fatalf("NewEngine() error = %v, want nil", err) t.Fatalf("NewEngine() error = %v, want nil", err)
} }

View File

@@ -96,7 +96,7 @@ func New(llmClient contracts.StructuredLLMClient, _ Options, references ...contr
if err != nil { if err != nil {
return nil, extractorErrorf("prepare scene eligibility: %w", err) return nil, extractorErrorf("prepare scene eligibility: %w", err)
} }
promptSHA, err := scriptoriumPromptMetadata() promptSHA, err := promptAssetMetadata()
if err != nil { if err != nil {
return nil, extractorErrorf("load prompt metadata: %w", err) return nil, extractorErrorf("load prompt metadata: %w", err)
} }

View File

@@ -9,7 +9,7 @@ import (
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared" "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
) )
const scriptoriumPromptRoot = "assets/prompts" const promptAssetRoot = "assets/prompts"
var promptAssetManifest = shared.PromptAssetManifest{ var promptAssetManifest = shared.PromptAssetManifest{
ModuleDir: "dnd.combat_turns", ModuleDir: "dnd.combat_turns",
@@ -33,21 +33,21 @@ func RegisterPromptAssets(registry *llm.AssetRegistry) error {
if err != nil { if err != nil {
return fmt.Errorf("prepare combat-turn prompt assets: %w", err) return fmt.Errorf("prepare combat-turn prompt assets: %w", err)
} }
if err := registry.RegisterPromptFS(promptFS, scriptoriumPromptRoot); err != nil { if err := registry.RegisterPromptFS(promptFS, promptAssetRoot); err != nil {
return err return err
} }
return registry.RegisterSchemaFS(embeddedAssets, "assets/schemas") return registry.RegisterSchemaFS(embeddedAssets, "assets/schemas")
} }
func scriptoriumPromptMetadata() (string, error) { func promptAssetMetadata() (string, error) {
scriptoriumPromptHashOnce.Do(func() { promptAssetHashOnce.Do(func() {
scriptoriumPromptHash, scriptoriumPromptHashErr = promptAssetManifest.Hash(embeddedAssets) promptAssetHash, promptAssetHashErr = promptAssetManifest.Hash(embeddedAssets)
}) })
return scriptoriumPromptHash, scriptoriumPromptHashErr return promptAssetHash, promptAssetHashErr
} }
var ( var (
scriptoriumPromptHashOnce sync.Once promptAssetHashOnce sync.Once
scriptoriumPromptHash string promptAssetHash string
scriptoriumPromptHashErr error promptAssetHashErr error
) )

View File

@@ -8,7 +8,7 @@ import (
"time" "time"
"gitea.maximumdirect.net/eric/notarius/internal/framework/llm" "gitea.maximumdirect.net/eric/notarius/internal/framework/llm"
"gitea.maximumdirect.net/eric/scriptorium" "gitea.maximumdirect.net/eric/promptkit"
) )
func TestRegisterPromptAssetsAndPrepareCombatPrompt(t *testing.T) { func TestRegisterPromptAssetsAndPrepareCombatPrompt(t *testing.T) {
@@ -23,37 +23,37 @@ func TestRegisterPromptAssetsAndPrepareCombatPrompt(t *testing.T) {
if _, err := fs.ReadFile(schemaFS, "dnd_combat_turns_llm.v1.json"); err != nil { if _, err := fs.ReadFile(schemaFS, "dnd_combat_turns_llm.v1.json"); err != nil {
t.Fatalf("response schema asset: %v", err) t.Fatalf("response schema asset: %v", err)
} }
hash, err := scriptoriumPromptMetadata() hash, err := promptAssetMetadata()
if err != nil || !strings.HasPrefix(hash, "sha256:") { if err != nil || !strings.HasPrefix(hash, "sha256:") {
t.Fatalf("scriptoriumPromptMetadata() = %q, %v; want digest", hash, err) t.Fatalf("promptAssetMetadata() = %q, %v; want digest", hash, err)
} }
} }
func TestScriptoriumPromptPreparesRequiredInputs(t *testing.T) { func TestPromptAssetsPrepareRequiredInputs(t *testing.T) {
registry := llm.NewAssetRegistry() registry := llm.NewAssetRegistry()
if err := RegisterPromptAssets(registry); err != nil { if err := RegisterPromptAssets(registry); err != nil {
t.Fatalf("RegisterPromptAssets() error = %v, want nil", err) t.Fatalf("RegisterPromptAssets() error = %v, want nil", err)
} }
options, err := registry.ScriptoriumOptions() options, err := registry.PromptKitOptions()
if err != nil { if err != nil {
t.Fatalf("ScriptoriumOptions() error = %v, want nil", err) t.Fatalf("PromptKitOptions() error = %v, want nil", err)
} }
options = append(options, scriptorium.WithProfiles(scriptorium.OpenAICompatibleProfile(scriptorium.OpenAICompatibleProfileConfig{ options = append(options, promptkit.WithProfiles(promptkit.OpenAICompatibleProfile(promptkit.OpenAICompatibleProfileConfig{
ID: "combat-test-profile", Endpoint: "http://127.0.0.1:1/v1", Model: "combat-test-model", ID: "combat-test-profile", Endpoint: "http://127.0.0.1:1/v1", Model: "combat-test-model",
}))) })))
engine, err := scriptorium.NewEngine(scriptorium.Config{Timeout: time.Second}, options...) engine, err := promptkit.NewEngine(promptkit.Config{Timeout: time.Second}, options...)
if err != nil { if err != nil {
t.Fatalf("NewEngine() error = %v, want nil", err) t.Fatalf("NewEngine() error = %v, want nil", err)
} }
transcript := `{"units":[1]}` transcript := `{"units":[1]}`
prepared, err := engine.Prepare(context.Background(), scriptorium.RunRequest{ prepared, err := engine.Prepare(context.Background(), promptkit.RunRequest{
PromptID: PromptID, PromptVersion: SchemaVersion, ProfileID: "combat-test-profile", PromptID: PromptID, PromptVersion: SchemaVersion, ProfileID: "combat-test-profile",
Inputs: map[string]scriptorium.ArtifactRef{ Inputs: map[string]promptkit.ArtifactRef{
"transcript": scriptorium.InlineWithURI("file:///session.json", transcript), "transcript": promptkit.InlineWithURI("file:///session.json", transcript),
"players": scriptorium.Inline(" "), "players": promptkit.Inline(" "),
"party": scriptorium.Inline(" "), "party": promptkit.Inline(" "),
"glossary": scriptorium.Inline(" "), "glossary": promptkit.Inline(" "),
"npcs": scriptorium.Inline(" "), "npcs": promptkit.Inline(" "),
}, },
}) })
if err != nil { if err != nil {

View File

@@ -54,7 +54,7 @@ func New(llmClient contracts.StructuredLLMClient, _ Options, references ...contr
if len(references) > 1 { if len(references) > 1 {
return nil, extractorErrorf("at most one reference set may be supplied") return nil, extractorErrorf("at most one reference set may be supplied")
} }
promptSHA, err := scriptoriumPromptMetadata() promptSHA, err := promptAssetMetadata()
if err != nil { if err != nil {
return nil, extractorErrorf("load prompt metadata: %w", err) return nil, extractorErrorf("load prompt metadata: %w", err)
} }

View File

@@ -9,7 +9,7 @@ import (
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared" "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
) )
const scriptoriumPromptRoot = "assets/prompts" const promptAssetRoot = "assets/prompts"
var promptAssetManifest = shared.PromptAssetManifest{ var promptAssetManifest = shared.PromptAssetManifest{
ModuleDir: "dnd.item_events", ModuleDir: "dnd.item_events",
@@ -32,21 +32,21 @@ func RegisterPromptAssets(registry *llm.AssetRegistry) error {
if err != nil { if err != nil {
return fmt.Errorf("prepare item event prompt assets: %w", err) return fmt.Errorf("prepare item event prompt assets: %w", err)
} }
if err := registry.RegisterPromptFS(promptFS, scriptoriumPromptRoot); err != nil { if err := registry.RegisterPromptFS(promptFS, promptAssetRoot); err != nil {
return err return err
} }
return registry.RegisterSchemaFS(embeddedAssets, "assets/schemas") return registry.RegisterSchemaFS(embeddedAssets, "assets/schemas")
} }
func scriptoriumPromptMetadata() (string, error) { func promptAssetMetadata() (string, error) {
scriptoriumPromptHashOnce.Do(func() { promptAssetHashOnce.Do(func() {
scriptoriumPromptHash, scriptoriumPromptHashErr = promptAssetManifest.Hash(embeddedAssets) promptAssetHash, promptAssetHashErr = promptAssetManifest.Hash(embeddedAssets)
}) })
return scriptoriumPromptHash, scriptoriumPromptHashErr return promptAssetHash, promptAssetHashErr
} }
var ( var (
scriptoriumPromptHashOnce sync.Once promptAssetHashOnce sync.Once
scriptoriumPromptHash string promptAssetHash string
scriptoriumPromptHashErr error promptAssetHashErr error
) )

View File

@@ -9,7 +9,7 @@ import (
"time" "time"
"gitea.maximumdirect.net/eric/notarius/internal/framework/llm" "gitea.maximumdirect.net/eric/notarius/internal/framework/llm"
"gitea.maximumdirect.net/eric/scriptorium" "gitea.maximumdirect.net/eric/promptkit"
) )
func TestPromptAssetsPrepareItemEventPrompt(t *testing.T) { func TestPromptAssetsPrepareItemEventPrompt(t *testing.T) {
@@ -17,24 +17,24 @@ func TestPromptAssetsPrepareItemEventPrompt(t *testing.T) {
if err := RegisterPromptAssets(registry); err != nil { if err := RegisterPromptAssets(registry); err != nil {
t.Fatal(err) t.Fatal(err)
} }
options, err := registry.ScriptoriumOptions() options, err := registry.PromptKitOptions()
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
options = append(options, scriptorium.WithProfiles(scriptorium.OpenAICompatibleProfile(scriptorium.OpenAICompatibleProfileConfig{ options = append(options, promptkit.WithProfiles(promptkit.OpenAICompatibleProfile(promptkit.OpenAICompatibleProfileConfig{
ID: "item-events-test-profile", Endpoint: "http://127.0.0.1:1/v1", Model: "item-events-test-model", ID: "item-events-test-profile", Endpoint: "http://127.0.0.1:1/v1", Model: "item-events-test-model",
}))) })))
engine, err := scriptorium.NewEngine(scriptorium.Config{Timeout: time.Second}, options...) engine, err := promptkit.NewEngine(promptkit.Config{Timeout: time.Second}, options...)
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
prepared, err := engine.Prepare(context.Background(), scriptorium.RunRequest{ prepared, err := engine.Prepare(context.Background(), promptkit.RunRequest{
PromptID: PromptID, PromptVersion: SchemaVersion, ProfileID: "item-events-test-profile", PromptID: PromptID, PromptVersion: SchemaVersion, ProfileID: "item-events-test-profile",
Inputs: map[string]scriptorium.ArtifactRef{ Inputs: map[string]promptkit.ArtifactRef{
"transcript": scriptorium.InlineWithURI("file:///session.json", `{"segments":[1]}`), "transcript": promptkit.InlineWithURI("file:///session.json", `{"segments":[1]}`),
"players": scriptorium.Inline(" "), "players": promptkit.Inline(" "),
"party": scriptorium.Inline(" "), "party": promptkit.Inline(" "),
"glossary": scriptorium.Inline(" "), "glossary": promptkit.Inline(" "),
}, },
}) })
if err != nil { if err != nil {
@@ -46,9 +46,9 @@ func TestPromptAssetsPrepareItemEventPrompt(t *testing.T) {
} }
func TestPromptAssetsDoNotLeakIntoMetadata(t *testing.T) { func TestPromptAssetsDoNotLeakIntoMetadata(t *testing.T) {
hash, err := scriptoriumPromptMetadata() hash, err := promptAssetMetadata()
if err != nil || !strings.HasPrefix(hash, "sha256:") { if err != nil || !strings.HasPrefix(hash, "sha256:") {
t.Fatalf("scriptoriumPromptMetadata() = %q, %v", hash, err) t.Fatalf("promptAssetMetadata() = %q, %v", hash, err)
} }
metadata := newExtractor(t, &fakeItemEventsLLMClient{}).ManifestMetadata() metadata := newExtractor(t, &fakeItemEventsLLMClient{}).ManifestMetadata()
payload, err := json.Marshal(metadata) payload, err := json.Marshal(metadata)

View File

@@ -80,7 +80,7 @@ func New(llmClient contracts.StructuredLLMClient, _ Options, references ...contr
if err != nil { if err != nil {
return nil, extractorErrorf("prepare NPC registry prompt input: %w", err) return nil, extractorErrorf("prepare NPC registry prompt input: %w", err)
} }
promptSHA, err := scriptoriumPromptMetadata() promptSHA, err := promptAssetMetadata()
if err != nil { if err != nil {
return nil, extractorErrorf("load prompt metadata: %w", err) return nil, extractorErrorf("load prompt metadata: %w", err)
} }

View File

@@ -9,7 +9,7 @@ import (
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared" "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
) )
const scriptoriumPromptRoot = "assets/prompts" const promptAssetRoot = "assets/prompts"
var promptAssetManifest = shared.PromptAssetManifest{ var promptAssetManifest = shared.PromptAssetManifest{
ModuleDir: "dnd.npc_interactions", ModuleDir: "dnd.npc_interactions",
@@ -33,21 +33,21 @@ func RegisterPromptAssets(registry *llm.AssetRegistry) error {
if err != nil { if err != nil {
return fmt.Errorf("prepare NPC-interaction prompt assets: %w", err) return fmt.Errorf("prepare NPC-interaction prompt assets: %w", err)
} }
if err := registry.RegisterPromptFS(promptFS, scriptoriumPromptRoot); err != nil { if err := registry.RegisterPromptFS(promptFS, promptAssetRoot); err != nil {
return err return err
} }
return registry.RegisterSchemaFS(embeddedAssets, "assets/schemas") return registry.RegisterSchemaFS(embeddedAssets, "assets/schemas")
} }
func scriptoriumPromptMetadata() (string, error) { func promptAssetMetadata() (string, error) {
scriptoriumPromptHashOnce.Do(func() { promptAssetHashOnce.Do(func() {
scriptoriumPromptHash, scriptoriumPromptHashErr = promptAssetManifest.Hash(embeddedAssets) promptAssetHash, promptAssetHashErr = promptAssetManifest.Hash(embeddedAssets)
}) })
return scriptoriumPromptHash, scriptoriumPromptHashErr return promptAssetHash, promptAssetHashErr
} }
var ( var (
scriptoriumPromptHashOnce sync.Once promptAssetHashOnce sync.Once
scriptoriumPromptHash string promptAssetHash string
scriptoriumPromptHashErr error promptAssetHashErr error
) )

View File

@@ -8,7 +8,7 @@ import (
"time" "time"
"gitea.maximumdirect.net/eric/notarius/internal/framework/llm" "gitea.maximumdirect.net/eric/notarius/internal/framework/llm"
"gitea.maximumdirect.net/eric/scriptorium" "gitea.maximumdirect.net/eric/promptkit"
) )
func TestRegisterPromptAssetsAndPrepareInteractionPrompt(t *testing.T) { func TestRegisterPromptAssetsAndPrepareInteractionPrompt(t *testing.T) {
@@ -23,26 +23,26 @@ func TestRegisterPromptAssetsAndPrepareInteractionPrompt(t *testing.T) {
if _, err := fs.ReadFile(schemaFS, "dnd_npc_interactions_llm.v1.json"); err != nil { if _, err := fs.ReadFile(schemaFS, "dnd_npc_interactions_llm.v1.json"); err != nil {
t.Fatalf("response schema asset: %v", err) t.Fatalf("response schema asset: %v", err)
} }
options, err := registry.ScriptoriumOptions() options, err := registry.PromptKitOptions()
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
options = append(options, scriptorium.WithProfiles(scriptorium.OpenAICompatibleProfile(scriptorium.OpenAICompatibleProfileConfig{ options = append(options, promptkit.WithProfiles(promptkit.OpenAICompatibleProfile(promptkit.OpenAICompatibleProfileConfig{
ID: "npc-interactions-test-profile", Endpoint: "http://127.0.0.1:1/v1", Model: "npc-interactions-test-model", ID: "npc-interactions-test-profile", Endpoint: "http://127.0.0.1:1/v1", Model: "npc-interactions-test-model",
}))) })))
engine, err := scriptorium.NewEngine(scriptorium.Config{Timeout: time.Second}, options...) engine, err := promptkit.NewEngine(promptkit.Config{Timeout: time.Second}, options...)
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
transcript := `{"units":[1]}` transcript := `{"units":[1]}`
prepared, err := engine.Prepare(context.Background(), scriptorium.RunRequest{ prepared, err := engine.Prepare(context.Background(), promptkit.RunRequest{
PromptID: PromptID, PromptVersion: SchemaVersion, ProfileID: "npc-interactions-test-profile", PromptID: PromptID, PromptVersion: SchemaVersion, ProfileID: "npc-interactions-test-profile",
Inputs: map[string]scriptorium.ArtifactRef{ Inputs: map[string]promptkit.ArtifactRef{
"transcript": scriptorium.InlineWithURI("file:///session.json", transcript), "transcript": promptkit.InlineWithURI("file:///session.json", transcript),
"players": scriptorium.Inline("Dana: Mira"), "players": promptkit.Inline("Dana: Mira"),
"party": scriptorium.Inline("Mira: ranger"), "party": promptkit.Inline("Mira: ranger"),
"glossary": scriptorium.Inline("Greencloak: title"), "glossary": promptkit.Inline("Greencloak: title"),
"npcs": scriptorium.Inline(`{"npcs":[{"name":"Mira Thorn"}]}`), "npcs": promptkit.Inline(`{"npcs":[{"name":"Mira Thorn"}]}`),
}, },
}) })
if err != nil { if err != nil {
@@ -54,9 +54,9 @@ func TestRegisterPromptAssetsAndPrepareInteractionPrompt(t *testing.T) {
} }
func TestPromptMetadataDoesNotExposeAssetContent(t *testing.T) { func TestPromptMetadataDoesNotExposeAssetContent(t *testing.T) {
hash, err := scriptoriumPromptMetadata() hash, err := promptAssetMetadata()
if err != nil || !strings.HasPrefix(hash, "sha256:") { if err != nil || !strings.HasPrefix(hash, "sha256:") {
t.Fatalf("scriptoriumPromptMetadata() = %q, %v", hash, err) t.Fatalf("promptAssetMetadata() = %q, %v", hash, err)
} }
metadata := newExtractor(t, &fakeInteractionsLLMClient{}).ManifestMetadata() metadata := newExtractor(t, &fakeInteractionsLLMClient{}).ManifestMetadata()
for _, forbidden := range []string{"common-dnd-system", "dnd_npc_interactions_llm.v1.json"} { for _, forbidden := range []string{"common-dnd-system", "dnd_npc_interactions_llm.v1.json"} {

View File

@@ -54,7 +54,7 @@ func New(llmClient contracts.StructuredLLMClient, _ Options, references ...contr
if len(references) > 1 { if len(references) > 1 {
return nil, extractorErrorf("at most one reference set may be supplied") return nil, extractorErrorf("at most one reference set may be supplied")
} }
promptSHA, err := scriptoriumPromptMetadata() promptSHA, err := promptAssetMetadata()
if err != nil { if err != nil {
return nil, extractorErrorf("load prompt metadata: %w", err) return nil, extractorErrorf("load prompt metadata: %w", err)
} }

View File

@@ -9,7 +9,7 @@ import (
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared" "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
) )
const scriptoriumPromptRoot = "assets/prompts" const promptAssetRoot = "assets/prompts"
var promptAssetManifest = shared.PromptAssetManifest{ var promptAssetManifest = shared.PromptAssetManifest{
ModuleDir: "dnd.npcs", ModuleDir: "dnd.npcs",
@@ -32,21 +32,21 @@ func RegisterPromptAssets(registry *llm.AssetRegistry) error {
if err != nil { if err != nil {
return fmt.Errorf("prepare NPC prompt assets: %w", err) return fmt.Errorf("prepare NPC prompt assets: %w", err)
} }
if err := registry.RegisterPromptFS(promptFS, scriptoriumPromptRoot); err != nil { if err := registry.RegisterPromptFS(promptFS, promptAssetRoot); err != nil {
return err return err
} }
return registry.RegisterSchemaFS(embeddedAssets, "assets/schemas") return registry.RegisterSchemaFS(embeddedAssets, "assets/schemas")
} }
func scriptoriumPromptMetadata() (string, error) { func promptAssetMetadata() (string, error) {
scriptoriumPromptHashOnce.Do(func() { promptAssetHashOnce.Do(func() {
scriptoriumPromptHash, scriptoriumPromptHashErr = promptAssetManifest.Hash(embeddedAssets) promptAssetHash, promptAssetHashErr = promptAssetManifest.Hash(embeddedAssets)
}) })
return scriptoriumPromptHash, scriptoriumPromptHashErr return promptAssetHash, promptAssetHashErr
} }
var ( var (
scriptoriumPromptHashOnce sync.Once promptAssetHashOnce sync.Once
scriptoriumPromptHash string promptAssetHash string
scriptoriumPromptHashErr error promptAssetHashErr error
) )

View File

@@ -8,7 +8,7 @@ import (
"time" "time"
"gitea.maximumdirect.net/eric/notarius/internal/framework/llm" "gitea.maximumdirect.net/eric/notarius/internal/framework/llm"
"gitea.maximumdirect.net/eric/scriptorium" "gitea.maximumdirect.net/eric/promptkit"
) )
func TestRegisterPromptAssetsAndPrepareNPCPrompt(t *testing.T) { func TestRegisterPromptAssetsAndPrepareNPCPrompt(t *testing.T) {
@@ -16,24 +16,24 @@ func TestRegisterPromptAssetsAndPrepareNPCPrompt(t *testing.T) {
if err := RegisterPromptAssets(registry); err != nil { if err := RegisterPromptAssets(registry); err != nil {
t.Fatalf("RegisterPromptAssets() error = %v, want nil", err) t.Fatalf("RegisterPromptAssets() error = %v, want nil", err)
} }
options, err := registry.ScriptoriumOptions() options, err := registry.PromptKitOptions()
if err != nil { if err != nil {
t.Fatalf("ScriptoriumOptions() error = %v, want nil", err) t.Fatalf("PromptKitOptions() error = %v, want nil", err)
} }
options = append(options, scriptorium.WithProfiles(scriptorium.OpenAICompatibleProfile(scriptorium.OpenAICompatibleProfileConfig{ options = append(options, promptkit.WithProfiles(promptkit.OpenAICompatibleProfile(promptkit.OpenAICompatibleProfileConfig{
ID: "npc-test-profile", Endpoint: "http://127.0.0.1:1/v1", Model: "npc-test-model", ID: "npc-test-profile", Endpoint: "http://127.0.0.1:1/v1", Model: "npc-test-model",
}))) })))
engine, err := scriptorium.NewEngine(scriptorium.Config{Timeout: time.Second}, options...) engine, err := promptkit.NewEngine(promptkit.Config{Timeout: time.Second}, options...)
if err != nil { if err != nil {
t.Fatalf("NewEngine() error = %v, want nil", err) t.Fatalf("NewEngine() error = %v, want nil", err)
} }
prepared, err := engine.Prepare(context.Background(), scriptorium.RunRequest{ prepared, err := engine.Prepare(context.Background(), promptkit.RunRequest{
PromptID: PromptID, PromptVersion: SchemaVersion, ProfileID: "npc-test-profile", PromptID: PromptID, PromptVersion: SchemaVersion, ProfileID: "npc-test-profile",
Inputs: map[string]scriptorium.ArtifactRef{ Inputs: map[string]promptkit.ArtifactRef{
"transcript": scriptorium.InlineWithURI("file:///session.json", `{"units":[1]}`), "transcript": promptkit.InlineWithURI("file:///session.json", `{"units":[1]}`),
"players": scriptorium.Inline(" "), "players": promptkit.Inline(" "),
"party": scriptorium.Inline(" "), "party": promptkit.Inline(" "),
"glossary": scriptorium.Inline(" "), "glossary": promptkit.Inline(" "),
}, },
}) })
if err != nil { if err != nil {
@@ -45,9 +45,9 @@ func TestRegisterPromptAssetsAndPrepareNPCPrompt(t *testing.T) {
} }
func TestPromptMetadataAndDiagnosticsDoNotContainRawAssets(t *testing.T) { func TestPromptMetadataAndDiagnosticsDoNotContainRawAssets(t *testing.T) {
hash, err := scriptoriumPromptMetadata() hash, err := promptAssetMetadata()
if err != nil || !strings.HasPrefix(hash, "sha256:") { if err != nil || !strings.HasPrefix(hash, "sha256:") {
t.Fatalf("scriptoriumPromptMetadata() = %q, %v; want hash", hash, err) t.Fatalf("promptAssetMetadata() = %q, %v; want hash", hash, err)
} }
metadata := newExtractor(t, &fakeNPCsLLMClient{}).ManifestMetadata() metadata := newExtractor(t, &fakeNPCsLLMClient{}).ManifestMetadata()
payload, err := json.Marshal(metadata) payload, err := json.Marshal(metadata)

View File

@@ -59,7 +59,7 @@ func New(llmClient contracts.StructuredLLMClient, _ Options, references ...contr
if len(references) > 1 { if len(references) > 1 {
return nil, extractorErrorf("at most one reference set may be supplied") return nil, extractorErrorf("at most one reference set may be supplied")
} }
promptSHA, err := scriptoriumPromptMetadata() promptSHA, err := promptAssetMetadata()
if err != nil { if err != nil {
return nil, extractorErrorf("load prompt metadata: %w", err) return nil, extractorErrorf("load prompt metadata: %w", err)
} }

View File

@@ -9,7 +9,7 @@ import (
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared" "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
) )
const scriptoriumPromptRoot = "assets/prompts" const promptAssetRoot = "assets/prompts"
var promptAssetManifest = shared.PromptAssetManifest{ var promptAssetManifest = shared.PromptAssetManifest{
ModuleDir: "dnd.scene_descriptions", ModuleDir: "dnd.scene_descriptions",
@@ -31,21 +31,21 @@ func RegisterPromptAssets(registry *llm.AssetRegistry) error {
if err != nil { if err != nil {
return fmt.Errorf("prepare scene-description prompt assets: %w", err) return fmt.Errorf("prepare scene-description prompt assets: %w", err)
} }
if err := registry.RegisterPromptFS(promptFS, scriptoriumPromptRoot); err != nil { if err := registry.RegisterPromptFS(promptFS, promptAssetRoot); err != nil {
return err return err
} }
return registry.RegisterSchemaFS(embeddedAssets, "assets/schemas") return registry.RegisterSchemaFS(embeddedAssets, "assets/schemas")
} }
func scriptoriumPromptMetadata() (string, error) { func promptAssetMetadata() (string, error) {
scriptoriumPromptHashOnce.Do(func() { promptAssetHashOnce.Do(func() {
scriptoriumPromptHash, scriptoriumPromptHashErr = promptAssetManifest.Hash(embeddedAssets) promptAssetHash, promptAssetHashErr = promptAssetManifest.Hash(embeddedAssets)
}) })
return scriptoriumPromptHash, scriptoriumPromptHashErr return promptAssetHash, promptAssetHashErr
} }
var ( var (
scriptoriumPromptHashOnce sync.Once promptAssetHashOnce sync.Once
scriptoriumPromptHash string promptAssetHash string
scriptoriumPromptHashErr error promptAssetHashErr error
) )

View File

@@ -8,7 +8,7 @@ import (
"time" "time"
"gitea.maximumdirect.net/eric/notarius/internal/framework/llm" "gitea.maximumdirect.net/eric/notarius/internal/framework/llm"
"gitea.maximumdirect.net/eric/scriptorium" "gitea.maximumdirect.net/eric/promptkit"
) )
func TestRegisterPromptAssetsPreparesSceneDescriptionPrompt(t *testing.T) { func TestRegisterPromptAssetsPreparesSceneDescriptionPrompt(t *testing.T) {
@@ -16,24 +16,24 @@ func TestRegisterPromptAssetsPreparesSceneDescriptionPrompt(t *testing.T) {
if err := RegisterPromptAssets(registry); err != nil { if err := RegisterPromptAssets(registry); err != nil {
t.Fatalf("RegisterPromptAssets() error = %v, want nil", err) t.Fatalf("RegisterPromptAssets() error = %v, want nil", err)
} }
options, err := registry.ScriptoriumOptions() options, err := registry.PromptKitOptions()
if err != nil { if err != nil {
t.Fatalf("ScriptoriumOptions() error = %v, want nil", err) t.Fatalf("PromptKitOptions() error = %v, want nil", err)
} }
options = append(options, scriptorium.WithProfiles(scriptorium.OpenAICompatibleProfile(scriptorium.OpenAICompatibleProfileConfig{ options = append(options, promptkit.WithProfiles(promptkit.OpenAICompatibleProfile(promptkit.OpenAICompatibleProfileConfig{
ID: "scene-description-test-profile", Endpoint: "http://127.0.0.1:1/v1", Model: "scene-description-test-model", ID: "scene-description-test-profile", Endpoint: "http://127.0.0.1:1/v1", Model: "scene-description-test-model",
}))) })))
engine, err := scriptorium.NewEngine(scriptorium.Config{Timeout: time.Second}, options...) engine, err := promptkit.NewEngine(promptkit.Config{Timeout: time.Second}, options...)
if err != nil { if err != nil {
t.Fatalf("NewEngine() error = %v, want nil", err) t.Fatalf("NewEngine() error = %v, want nil", err)
} }
prepared, err := engine.Prepare(context.Background(), scriptorium.RunRequest{ prepared, err := engine.Prepare(context.Background(), promptkit.RunRequest{
PromptID: PromptID, PromptVersion: SchemaVersion, ProfileID: "scene-description-test-profile", PromptID: PromptID, PromptVersion: SchemaVersion, ProfileID: "scene-description-test-profile",
Inputs: map[string]scriptorium.ArtifactRef{ Inputs: map[string]promptkit.ArtifactRef{
"transcript": scriptorium.InlineWithURI("file:///session.json", `{"units":[1]}`), "transcript": promptkit.InlineWithURI("file:///session.json", `{"units":[1]}`),
"players": scriptorium.Inline(" "), "players": promptkit.Inline(" "),
"party": scriptorium.Inline(" "), "party": promptkit.Inline(" "),
"glossary": scriptorium.Inline(" "), "glossary": promptkit.Inline(" "),
}, },
}) })
if err != nil { if err != nil {
@@ -45,9 +45,9 @@ func TestRegisterPromptAssetsPreparesSceneDescriptionPrompt(t *testing.T) {
} }
func TestPromptMetadataAndDiagnosticsDoNotContainRawAssets(t *testing.T) { func TestPromptMetadataAndDiagnosticsDoNotContainRawAssets(t *testing.T) {
hash, err := scriptoriumPromptMetadata() hash, err := promptAssetMetadata()
if err != nil || !strings.HasPrefix(hash, "sha256:") { if err != nil || !strings.HasPrefix(hash, "sha256:") {
t.Fatalf("scriptoriumPromptMetadata() = %q, %v; want hash", hash, err) t.Fatalf("promptAssetMetadata() = %q, %v; want hash", hash, err)
} }
payload, err := json.Marshal(newExtractor(t, &fakeSceneDescriptionsLLMClient{}).ManifestMetadata()) payload, err := json.Marshal(newExtractor(t, &fakeSceneDescriptionsLLMClient{}).ManifestMetadata())
if err != nil { if err != nil {

View File

@@ -94,7 +94,7 @@ func New(llmClient contracts.StructuredLLMClient, _ Options, references ...contr
if err != nil { if err != nil {
return nil, extractorErrorf("prepare NPC registry prompt input: %w", err) return nil, extractorErrorf("prepare NPC registry prompt input: %w", err)
} }
promptSHA, err := scriptoriumPromptMetadata() promptSHA, err := promptAssetMetadata()
if err != nil { if err != nil {
return nil, extractorErrorf("load prompt metadata: %w", err) return nil, extractorErrorf("load prompt metadata: %w", err)
} }

View File

@@ -9,7 +9,7 @@ import (
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared" "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
) )
const scriptoriumPromptRoot = "assets/prompts" const promptAssetRoot = "assets/prompts"
var promptAssetManifest = shared.PromptAssetManifest{ var promptAssetManifest = shared.PromptAssetManifest{
ModuleDir: "dnd.spells", ModuleDir: "dnd.spells",
@@ -34,21 +34,21 @@ func RegisterPromptAssets(registry *llm.AssetRegistry) error {
if err != nil { if err != nil {
return fmt.Errorf("prepare spell prompt assets: %w", err) return fmt.Errorf("prepare spell prompt assets: %w", err)
} }
if err := registry.RegisterPromptFS(promptFS, scriptoriumPromptRoot); err != nil { if err := registry.RegisterPromptFS(promptFS, promptAssetRoot); err != nil {
return err return err
} }
return registry.RegisterSchemaFS(embeddedAssets, "assets/schemas") return registry.RegisterSchemaFS(embeddedAssets, "assets/schemas")
} }
func scriptoriumPromptMetadata() (string, error) { func promptAssetMetadata() (string, error) {
scriptoriumPromptHashOnce.Do(func() { promptAssetHashOnce.Do(func() {
scriptoriumPromptHash, scriptoriumPromptHashErr = promptAssetManifest.Hash(embeddedAssets) promptAssetHash, promptAssetHashErr = promptAssetManifest.Hash(embeddedAssets)
}) })
return scriptoriumPromptHash, scriptoriumPromptHashErr return promptAssetHash, promptAssetHashErr
} }
var ( var (
scriptoriumPromptHashOnce sync.Once promptAssetHashOnce sync.Once
scriptoriumPromptHash string promptAssetHash string
scriptoriumPromptHashErr error promptAssetHashErr error
) )

View File

@@ -8,10 +8,10 @@ import (
"time" "time"
"gitea.maximumdirect.net/eric/notarius/internal/framework/llm" "gitea.maximumdirect.net/eric/notarius/internal/framework/llm"
"gitea.maximumdirect.net/eric/scriptorium" "gitea.maximumdirect.net/eric/promptkit"
) )
func TestScriptoriumPromptPreparesSpellPrompt(t *testing.T) { func TestPromptAssetsPrepareSpellPrompt(t *testing.T) {
transcript := []byte(`{"id":"session-1","segments":[{"id":"u1","text":"Mira casts shield."}]}`) transcript := []byte(`{"id":"session-1","segments":[{"id":"u1","text":"Mira casts shield."}]}`)
prepared := prepareSpellsPrompt(t, transcript, "Dana: Mira", "Mira: wizard", "Shield: abjuration") prepared := prepareSpellsPrompt(t, transcript, "Dana: Mira", "Mira: wizard", "Shield: abjuration")
@@ -23,7 +23,7 @@ func TestScriptoriumPromptPreparesSpellPrompt(t *testing.T) {
} }
} }
func TestScriptoriumPromptPreparesWithMissingOptionalReferences(t *testing.T) { func TestPromptAssetsPrepareWithMissingOptionalReferences(t *testing.T) {
transcript := []byte(`{"id":"session-1","segments":[]}`) transcript := []byte(`{"id":"session-1","segments":[]}`)
prepared := prepareSpellsPrompt(t, transcript, " ", " ", " ") prepared := prepareSpellsPrompt(t, transcript, " ", " ", " ")
@@ -35,7 +35,7 @@ func TestScriptoriumPromptPreparesWithMissingOptionalReferences(t *testing.T) {
t.Fatalf("reference message did not render empty optional reference placeholders") t.Fatalf("reference message did not render empty optional reference placeholders")
} }
func TestScriptoriumPromptDiagnosticsOmitRawMaterials(t *testing.T) { func TestPromptAssetDiagnosticsOmitRawMaterials(t *testing.T) {
transcript := []byte(`{"secret":"source text"}`) transcript := []byte(`{"secret":"source text"}`)
reference := "private party note" reference := "private party note"
prepared := prepareSpellsPrompt(t, transcript, "private player note", reference, " ") prepared := prepareSpellsPrompt(t, transcript, "private player note", reference, " ")
@@ -82,36 +82,36 @@ func TestScriptoriumPromptDiagnosticsOmitRawMaterials(t *testing.T) {
} }
} }
func prepareSpellsPrompt(t *testing.T, transcript []byte, players string, party string, glossary string) *scriptorium.PreparedRun { func prepareSpellsPrompt(t *testing.T, transcript []byte, players string, party string, glossary string) *promptkit.PreparedRun {
t.Helper() t.Helper()
registry := llm.NewAssetRegistry() registry := llm.NewAssetRegistry()
if err := RegisterPromptAssets(registry); err != nil { if err := RegisterPromptAssets(registry); err != nil {
t.Fatalf("register spell prompt assets: %v", err) t.Fatalf("register spell prompt assets: %v", err)
} }
options, err := registry.ScriptoriumOptions() options, err := registry.PromptKitOptions()
if err != nil { if err != nil {
t.Fatalf("ScriptoriumOptions() error = %v, want nil", err) t.Fatalf("PromptKitOptions() error = %v, want nil", err)
} }
options = append(options, scriptorium.WithProfiles(scriptorium.OpenAICompatibleProfile(scriptorium.OpenAICompatibleProfileConfig{ options = append(options, promptkit.WithProfiles(promptkit.OpenAICompatibleProfile(promptkit.OpenAICompatibleProfileConfig{
ID: "spell-test-profile", ID: "spell-test-profile",
Endpoint: "http://127.0.0.1:1/v1", Endpoint: "http://127.0.0.1:1/v1",
Model: "spell-test-model", Model: "spell-test-model",
}))) })))
engine, err := scriptorium.NewEngine(scriptorium.Config{Timeout: time.Second}, options...) engine, err := promptkit.NewEngine(promptkit.Config{Timeout: time.Second}, options...)
if err != nil { if err != nil {
t.Fatalf("NewEngine() error = %v, want nil", err) t.Fatalf("NewEngine() error = %v, want nil", err)
} }
prepared, err := engine.Prepare(context.Background(), scriptorium.RunRequest{ prepared, err := engine.Prepare(context.Background(), promptkit.RunRequest{
PromptID: PromptID, PromptID: PromptID,
PromptVersion: SchemaVersion, PromptVersion: SchemaVersion,
ProfileID: "spell-test-profile", ProfileID: "spell-test-profile",
Inputs: map[string]scriptorium.ArtifactRef{ Inputs: map[string]promptkit.ArtifactRef{
"transcript": scriptorium.InlineWithURI("file:///session.json", string(transcript)), "transcript": promptkit.InlineWithURI("file:///session.json", string(transcript)),
"spell_catalog": scriptorium.Inline(`{"spell_names":["Cure Wounds"]}`), "spell_catalog": promptkit.Inline(`{"spell_names":["Cure Wounds"]}`),
"npcs": scriptorium.Inline(`{"npcs":[]}`), "npcs": promptkit.Inline(`{"npcs":[]}`),
"players": scriptorium.Inline(players), "players": promptkit.Inline(players),
"party": scriptorium.Inline(party), "party": promptkit.Inline(party),
"glossary": scriptorium.Inline(glossary), "glossary": promptkit.Inline(glossary),
}, },
}) })
if err != nil { if err != nil {

View File

@@ -53,7 +53,7 @@ func New(llmClient contracts.StructuredLLMClient, _ Options) (*Normalizer, error
if llmClient == nil { if llmClient == nil {
return nil, normalizerErrorf("LLM client must not be nil") return nil, normalizerErrorf("LLM client must not be nil")
} }
promptSHA, err := scriptoriumPromptMetadata() promptSHA, err := promptAssetMetadata()
if err != nil { if err != nil {
return nil, normalizerErrorf("load prompt metadata: %w", err) return nil, normalizerErrorf("load prompt metadata: %w", err)
} }

View File

@@ -9,7 +9,7 @@ import (
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared" "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
) )
const scriptoriumPromptRoot = "assets/prompts" const promptAssetRoot = "assets/prompts"
var promptAssetManifest = shared.PromptAssetManifest{ var promptAssetManifest = shared.PromptAssetManifest{
ModuleDir: PromptID, ModuleDir: PromptID,
@@ -30,21 +30,21 @@ func RegisterPromptAssets(registry *llm.AssetRegistry) error {
if err != nil { if err != nil {
return fmt.Errorf("prepare NPC normalization prompt assets: %w", err) return fmt.Errorf("prepare NPC normalization prompt assets: %w", err)
} }
if err := registry.RegisterPromptFS(promptFS, scriptoriumPromptRoot); err != nil { if err := registry.RegisterPromptFS(promptFS, promptAssetRoot); err != nil {
return err return err
} }
return registry.RegisterSchemaFS(embeddedAssets, "assets/schemas") return registry.RegisterSchemaFS(embeddedAssets, "assets/schemas")
} }
func scriptoriumPromptMetadata() (string, error) { func promptAssetMetadata() (string, error) {
scriptoriumPromptHashOnce.Do(func() { promptAssetHashOnce.Do(func() {
scriptoriumPromptHash, scriptoriumPromptHashErr = promptAssetManifest.Hash(embeddedAssets) promptAssetHash, promptAssetHashErr = promptAssetManifest.Hash(embeddedAssets)
}) })
return scriptoriumPromptHash, scriptoriumPromptHashErr return promptAssetHash, promptAssetHashErr
} }
var ( var (
scriptoriumPromptHashOnce sync.Once promptAssetHashOnce sync.Once
scriptoriumPromptHash string promptAssetHash string
scriptoriumPromptHashErr error promptAssetHashErr error
) )

View File

@@ -8,36 +8,36 @@ import (
"time" "time"
"gitea.maximumdirect.net/eric/notarius/internal/framework/llm" "gitea.maximumdirect.net/eric/notarius/internal/framework/llm"
"gitea.maximumdirect.net/eric/scriptorium" "gitea.maximumdirect.net/eric/promptkit"
) )
func TestRegisterPromptAssetsPreparesNormalizationPrompt(t *testing.T) { func TestRegisterPromptAssetsPreparesNormalizationPrompt(t *testing.T) {
if want := []string{"common-dnd-system.md", "common-dnd-transcript.md"}; !reflect.DeepEqual(promptAssetManifest.SharedFiles, want) { if want := []string{"common-dnd-system.md", "common-dnd-transcript.md"}; !reflect.DeepEqual(promptAssetManifest.SharedFiles, want) {
t.Fatalf("shared prompt assets = %#v, want %#v", promptAssetManifest.SharedFiles, want) t.Fatalf("shared prompt assets = %#v, want %#v", promptAssetManifest.SharedFiles, want)
} }
if promptHash, err := scriptoriumPromptMetadata(); err != nil || promptHash == "" { if promptHash, err := promptAssetMetadata(); err != nil || promptHash == "" {
t.Fatalf("scriptoriumPromptMetadata() = %q, %v; want prompt fingerprint", promptHash, err) t.Fatalf("promptAssetMetadata() = %q, %v; want prompt fingerprint", promptHash, err)
} }
registry := llm.NewAssetRegistry() registry := llm.NewAssetRegistry()
if err := RegisterPromptAssets(registry); err != nil { if err := RegisterPromptAssets(registry); err != nil {
t.Fatalf("RegisterPromptAssets() error = %v", err) t.Fatalf("RegisterPromptAssets() error = %v", err)
} }
options, err := registry.ScriptoriumOptions() options, err := registry.PromptKitOptions()
if err != nil { if err != nil {
t.Fatalf("ScriptoriumOptions() error = %v", err) t.Fatalf("PromptKitOptions() error = %v", err)
} }
options = append(options, scriptorium.WithProfiles(scriptorium.OpenAICompatibleProfile(scriptorium.OpenAICompatibleProfileConfig{ options = append(options, promptkit.WithProfiles(promptkit.OpenAICompatibleProfile(promptkit.OpenAICompatibleProfileConfig{
ID: "normalize-test-profile", Endpoint: "http://127.0.0.1:1/v1", Model: "normalize-test-model", ID: "normalize-test-profile", Endpoint: "http://127.0.0.1:1/v1", Model: "normalize-test-model",
}))) })))
engine, err := scriptorium.NewEngine(scriptorium.Config{Timeout: time.Second}, options...) engine, err := promptkit.NewEngine(promptkit.Config{Timeout: time.Second}, options...)
if err != nil { if err != nil {
t.Fatalf("NewEngine() error = %v", err) t.Fatalf("NewEngine() error = %v", err)
} }
prepared, err := engine.Prepare(context.Background(), scriptorium.RunRequest{ prepared, err := engine.Prepare(context.Background(), promptkit.RunRequest{
PromptID: PromptID, PromptVersion: SchemaVersion, ProfileID: "normalize-test-profile", PromptID: PromptID, PromptVersion: SchemaVersion, ProfileID: "normalize-test-profile",
Inputs: map[string]scriptorium.ArtifactRef{ Inputs: map[string]promptkit.ArtifactRef{
"candidates": scriptorium.Inline(`{"npcs":[{"name":"Mira","source_refs":[]}]}`), "candidates": promptkit.Inline(`{"npcs":[{"name":"Mira","source_refs":[]}]}`),
"transcript": scriptorium.Inline(`{"windows":[{"units":[]}]}`), "transcript": promptkit.Inline(`{"windows":[{"units":[]}]}`),
}, },
}) })
if err != nil { if err != nil {
@@ -55,7 +55,7 @@ func TestRegisterPromptAssetsPreparesNormalizationPrompt(t *testing.T) {
} }
} }
for _, index := range []int{2, 4} { for _, index := range []int{2, 4} {
if cache := prepared.Messages[index].CacheControl; cache == nil || cache.Type != scriptorium.CacheControlEphemeral { if cache := prepared.Messages[index].CacheControl; cache == nil || cache.Type != promptkit.CacheControlEphemeral {
t.Errorf("message %d cache control = %#v, want ephemeral", index, cache) t.Errorf("message %d cache control = %#v, want ephemeral", index, cache)
} }
} }

View File

@@ -14,7 +14,7 @@ import (
npcextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/npcs" npcextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/npcs"
scenedescriptionextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/scenedescriptions" scenedescriptionextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/scenedescriptions"
spellextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/spells" spellextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/spells"
"gitea.maximumdirect.net/eric/scriptorium" "gitea.maximumdirect.net/eric/promptkit"
) )
func TestExtractionPromptsShareRenderedPrefix(t *testing.T) { func TestExtractionPromptsShareRenderedPrefix(t *testing.T) {
@@ -31,17 +31,17 @@ func TestExtractionPromptsShareRenderedPrefix(t *testing.T) {
t.Fatalf("registerPromptAssets() error = %v", err) t.Fatalf("registerPromptAssets() error = %v", err)
} }
engine := newPromptCacheEngine(t, registry) engine := newPromptCacheEngine(t, registry)
commonInputs := map[string]scriptorium.ArtifactRef{ commonInputs := map[string]promptkit.ArtifactRef{
"transcript": scriptorium.InlineWithURI("file:///session.json", `{"sentinel":"`+transcriptSentinel+`"}`), "transcript": promptkit.InlineWithURI("file:///session.json", `{"sentinel":"`+transcriptSentinel+`"}`),
"players": scriptorium.Inline(playersSentinel), "players": promptkit.Inline(playersSentinel),
"party": scriptorium.Inline(partySentinel), "party": promptkit.Inline(partySentinel),
"glossary": scriptorium.Inline(glossarySentinel), "glossary": promptkit.Inline(glossarySentinel),
} }
cases := []struct { cases := []struct {
name string name string
promptID string promptID string
promptVersion string promptVersion string
inputs map[string]scriptorium.ArtifactRef inputs map[string]promptkit.ArtifactRef
npcInput bool npcInput bool
spellCatalogInput bool spellCatalogInput bool
}{ }{
@@ -52,8 +52,8 @@ func TestExtractionPromptsShareRenderedPrefix(t *testing.T) {
name: "combat turns", name: "combat turns",
promptID: combatextract.PromptID, promptID: combatextract.PromptID,
promptVersion: combatextract.SchemaVersion, promptVersion: combatextract.SchemaVersion,
inputs: withPromptInputs(commonInputs, map[string]scriptorium.ArtifactRef{ inputs: withPromptInputs(commonInputs, map[string]promptkit.ArtifactRef{
"npcs": scriptorium.Inline(`{"sentinel":"` + npcSentinel + `"}`), "npcs": promptkit.Inline(`{"sentinel":"` + npcSentinel + `"}`),
}), }),
npcInput: true, npcInput: true,
}, },
@@ -61,8 +61,8 @@ func TestExtractionPromptsShareRenderedPrefix(t *testing.T) {
name: "npc interactions", name: "npc interactions",
promptID: interactionextract.PromptID, promptID: interactionextract.PromptID,
promptVersion: interactionextract.SchemaVersion, promptVersion: interactionextract.SchemaVersion,
inputs: withPromptInputs(commonInputs, map[string]scriptorium.ArtifactRef{ inputs: withPromptInputs(commonInputs, map[string]promptkit.ArtifactRef{
"npcs": scriptorium.Inline(`{"sentinel":"` + npcSentinel + `"}`), "npcs": promptkit.Inline(`{"sentinel":"` + npcSentinel + `"}`),
}), }),
npcInput: true, npcInput: true,
}, },
@@ -70,19 +70,19 @@ func TestExtractionPromptsShareRenderedPrefix(t *testing.T) {
name: "spells", name: "spells",
promptID: spellextract.PromptID, promptID: spellextract.PromptID,
promptVersion: spellextract.SchemaVersion, promptVersion: spellextract.SchemaVersion,
inputs: withPromptInputs(commonInputs, map[string]scriptorium.ArtifactRef{ inputs: withPromptInputs(commonInputs, map[string]promptkit.ArtifactRef{
"npcs": scriptorium.Inline(`{"sentinel":"` + npcSentinel + `"}`), "npcs": promptkit.Inline(`{"sentinel":"` + npcSentinel + `"}`),
"spell_catalog": scriptorium.Inline(`{"sentinel":"` + catalogSentinel + `"}`), "spell_catalog": promptkit.Inline(`{"sentinel":"` + catalogSentinel + `"}`),
}), }),
npcInput: true, npcInput: true,
spellCatalogInput: true, spellCatalogInput: true,
}, },
} }
var sharedPrefix []scriptorium.RenderedMessage var sharedPrefix []promptkit.RenderedMessage
for _, testCase := range cases { for _, testCase := range cases {
t.Run(testCase.name, func(t *testing.T) { t.Run(testCase.name, func(t *testing.T) {
prepared, err := engine.Prepare(context.Background(), scriptorium.RunRequest{ prepared, err := engine.Prepare(context.Background(), promptkit.RunRequest{
PromptID: testCase.promptID, PromptID: testCase.promptID,
PromptVersion: testCase.promptVersion, PromptVersion: testCase.promptVersion,
ProfileID: "prompt-cache-test-profile", ProfileID: "prompt-cache-test-profile",
@@ -100,7 +100,7 @@ func TestExtractionPromptsShareRenderedPrefix(t *testing.T) {
t.Fatalf("prepared prompt has %d messages, want lane-specific suffix after transcript", len(prepared.Messages)) t.Fatalf("prepared prompt has %d messages, want lane-specific suffix after transcript", len(prepared.Messages))
} }
if sharedPrefix == nil { if sharedPrefix == nil {
sharedPrefix = append([]scriptorium.RenderedMessage(nil), prefix...) sharedPrefix = append([]promptkit.RenderedMessage(nil), prefix...)
} else if !reflect.DeepEqual(prefix, sharedPrefix) { } else if !reflect.DeepEqual(prefix, sharedPrefix) {
t.Fatalf("rendered prefix = %#v, want %#v", prefix, sharedPrefix) t.Fatalf("rendered prefix = %#v, want %#v", prefix, sharedPrefix)
} }
@@ -114,24 +114,24 @@ func TestExtractionPromptsShareRenderedPrefix(t *testing.T) {
} }
} }
func newPromptCacheEngine(t *testing.T, registry *llm.AssetRegistry) *scriptorium.Engine { func newPromptCacheEngine(t *testing.T, registry *llm.AssetRegistry) *promptkit.Engine {
t.Helper() t.Helper()
options, err := registry.ScriptoriumOptions() options, err := registry.PromptKitOptions()
if err != nil { if err != nil {
t.Fatalf("ScriptoriumOptions() error = %v", err) t.Fatalf("PromptKitOptions() error = %v", err)
} }
options = append(options, scriptorium.WithProfiles(scriptorium.OpenAICompatibleProfile(scriptorium.OpenAICompatibleProfileConfig{ options = append(options, promptkit.WithProfiles(promptkit.OpenAICompatibleProfile(promptkit.OpenAICompatibleProfileConfig{
ID: "prompt-cache-test-profile", Endpoint: "http://127.0.0.1:1/v1", Model: "prompt-cache-test-model", ID: "prompt-cache-test-profile", Endpoint: "http://127.0.0.1:1/v1", Model: "prompt-cache-test-model",
}))) })))
engine, err := scriptorium.NewEngine(scriptorium.Config{Timeout: time.Second}, options...) engine, err := promptkit.NewEngine(promptkit.Config{Timeout: time.Second}, options...)
if err != nil { if err != nil {
t.Fatalf("NewEngine() error = %v", err) t.Fatalf("NewEngine() error = %v", err)
} }
return engine return engine
} }
func withPromptInputs(inputs, extras map[string]scriptorium.ArtifactRef) map[string]scriptorium.ArtifactRef { func withPromptInputs(inputs, extras map[string]promptkit.ArtifactRef) map[string]promptkit.ArtifactRef {
merged := make(map[string]scriptorium.ArtifactRef, len(inputs)+len(extras)) merged := make(map[string]promptkit.ArtifactRef, len(inputs)+len(extras))
for name, input := range inputs { for name, input := range inputs {
merged[name] = input merged[name] = input
} }
@@ -141,14 +141,14 @@ func withPromptInputs(inputs, extras map[string]scriptorium.ArtifactRef) map[str
return merged return merged
} }
func assertRenderedInputAfter(t *testing.T, messages []scriptorium.RenderedMessage, sentinel string, index int) { func assertRenderedInputAfter(t *testing.T, messages []promptkit.RenderedMessage, sentinel string, index int) {
t.Helper() t.Helper()
if inputIndex := renderedInputMessageIndex(t, messages, sentinel); inputIndex <= index { if inputIndex := renderedInputMessageIndex(t, messages, sentinel); inputIndex <= index {
t.Fatalf("input sentinel %q rendered at message %d, want after transcript message %d", sentinel, inputIndex, index) t.Fatalf("input sentinel %q rendered at message %d, want after transcript message %d", sentinel, inputIndex, index)
} }
} }
func renderedInputMessageIndex(t *testing.T, messages []scriptorium.RenderedMessage, sentinel string) int { func renderedInputMessageIndex(t *testing.T, messages []promptkit.RenderedMessage, sentinel string) int {
t.Helper() t.Helper()
index := -1 index := -1
occurrences := 0 occurrences := 0

View File

@@ -1,4 +1,4 @@
version: 3 version: 4
output: output:
directory: ./notarius-output directory: ./notarius-output
cache: cache:

View File

@@ -1,4 +1,4 @@
version: 3 version: 4
output: output:
directory: ./notarius-output directory: ./notarius-output
cache: cache:

View File

@@ -1,4 +1,4 @@
version: 3 version: 4
output: output:
directory: ./notarius-output directory: ./notarius-output
cache: cache:

View File

@@ -1,4 +1,4 @@
version: 3 version: 4
output: output:
directory: ./notarius-output directory: ./notarius-output
cache: cache: