Compare commits

...

5 Commits

66 changed files with 878 additions and 688 deletions

View File

@@ -1,7 +1,7 @@
# Configuration
This is the canonical reference for Notarius configuration. Configuration files
are YAML and must declare version 3. They select pipelines and their modules;
are YAML and must declare version 4. They select pipelines and their modules;
the [CLI reference](cli.md) owns invocation syntax, and
[Operations](operations.md) owns run-state procedures.
@@ -45,8 +45,8 @@ other than **version** is optional.
| Field | Type | Default | Rules |
| --- | --- | --- | --- |
| **version** | integer | none | Required; must be 3. |
| **scriptorium** | object | none | Profile source configuration. |
| **version** | integer | none | Required; must be 4. |
| **promptkit** | object | none | Profile source configuration. |
| **pipelines** | map | empty | Maps pipeline IDs to pipeline definitions. |
| **concurrency** | object | see below | Global LLM and extraction limits. |
| **output** | object | see below | Published output settings. |
@@ -69,9 +69,16 @@ Built-in defaults are:
An empty cache directory in YAML deliberately selects the corresponding
per-user root. An explicit empty output or debug directory is invalid.
## Scriptorium Profiles
## PromptKit Profiles
The optional **scriptorium** object selects one source of profile definitions:
The optional **promptkit** object selects one source of profile definitions:
~~~yaml
version: 4
promptkit:
profile_dir: /path/to/profiles
# profile_file: /path/to/profiles.yml
~~~
| Field | Type | Rules |
| --- | --- | --- |
@@ -79,10 +86,19 @@ The optional **scriptorium** object selects one source of profile definitions:
| **profile_file** | string | Non-empty profile file. |
Set at most one of these fields. Profile IDs used by a binding must be available
from the selected Scriptorium profile source when the pipeline is resolved.
from the selected PromptKit profile source when the pipeline is resolved.
Keep credentials out of this file: configure a profile to read its credential
from an environment variable, then set that environment variable only in the
run environment.
run environment. PromptKit owns the profile-file format; see the
[PromptKit upstream boundary](integrations/pkg-promptkit.md) for the pinned
package and canonical format reference.
## Migrating Version 3 Configuration
Version 3 files are not decoded or rewritten. Change **version: 3** to
**version: 4** and rename the top-level **scriptorium:** section to
**promptkit:**. Version 4 decoding is strict, so a remaining **scriptorium**
field is rejected as unknown.
## Operational Environment Variables
@@ -199,7 +215,7 @@ extract:
| Binding field | Type | Default | Rules |
| --- | --- | --- | --- |
| **module** | string | none | Required for an object binding. Must be a registered compatible key. |
| **llm_profile** | string | none | Optional non-empty Scriptorium profile ID. |
| **llm_profile** | string | none | Optional non-empty PromptKit profile ID. |
| **retries** | integer | 0 | Non-negative additional attempts for chunk, extract, merge, and normalize bindings. |
| **options** | object | none | Must satisfy the selected module. |
| **references** | map | none | Valid only on chunk, extract, merge, and normalize bindings. |

View File

@@ -22,7 +22,7 @@ implemented component map.
| Configuration loading, resolution, or user-visible configuration behavior | [Configuration Internals](internal/configuration.md) and [Configuration](config.md) | The internal guide owns loading and resolution mechanics; the reference owns the configuration contract. |
| Pipeline resolution or execution | [Pipeline Internals](internal/pipeline.md) | It documents profiles, references, validation, retries, checkpoints, and runner behavior. |
| Production modules or validators | [Module Internals](internal/modules.md), [D&D Module Internals](internal/dnd.md), and [D&D integration contracts](integrations/) | The generic guide owns extension mechanics, the D&D guide owns shared family conventions, and the contracts own durable output shapes. |
| LLM clients, prompts, schemas, profiles, or scheduling | [LLM Runtime](internal/llm.md) | It documents the transport boundary and Scriptorium integration. |
| LLM clients, prompts, schemas, profiles, or scheduling | [LLM Runtime](internal/llm.md) | It documents the transport boundary and PromptKit integration. |
| Output, cache, resume, or debug artifacts | [Run State Internals](internal/state.md), [Operations](operations.md), and [Configuration](config.md) | These separate implementation details, operator behavior, and configuration contracts. |
| External input formats, artifact schemas, or durable output files | [Integration Contracts](integrations/) | Integration documents define external and durable data contracts. |
| Proposed or unimplemented behavior | [Roadmap](roadmap/) | Future work belongs only in roadmap documentation until implemented. |

View File

@@ -1,151 +1,41 @@
# Package `promptkit`
# PromptKit Integration
Import path:
Notarius pins
[`gitea.maximumdirect.net/eric/promptkit` v0.1.0](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0)
as its in-process prompt engine. The upstream
[Go package consumer guide](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/docs/consumers/pkg-promptkit.md)
owns the public engine API, and the upstream
[format reference](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/docs/formats.md)
owns prompt, profile, and schema file contracts.
```go
import "gitea.maximumdirect.net/eric/promptkit"
```
## Supported Boundary
Package `promptkit` is the supported Go contract for in-process prompt
preparation and execution. The declarations and their GoDoc in the
[root package](../../doc.go) own the exact API; this guide explains how the
pieces are used together. The [framework format reference](../formats.md) owns
prompt, profile, and schema file contracts.
Notarius relies on the root `promptkit` package to:
## Engine Construction And Sources
- construct an `Engine` with filesystem-backed prompt, schema, and optional
profile sources;
- prepare and run a `RunRequest` with named inline artifacts, variables,
metadata, prompt identity, and profile selection;
- return rendered debug material, validated structured output, selected
profile and model metadata, and token usage;
- distinguish structured-output validation failure from execution failure; and
- identify a missing explicit profile through `ErrProfileNotFound`.
Construct an engine with [`NewEngine`, `Config`, and
`Option`](../../engine.go). `PromptDir` is required unless a prompt source
option is supplied. `ProfileDir` optionally overlays built-in profiles, and an
empty `SchemaDir` uses the current directory. `Timeout` is the transport-wide
safety cap for the built-in OpenAI-compatible client. An optional `HTTPClient`
is cloned; its positive timeout takes precedence.
Notarius does not use PromptKit's optional `ArtifactReader`. It materializes
source and reference content itself and supplies owned inline artifacts at the
adapter boundary. It also retains responsibility for pipeline retries,
scheduling, debug persistence, redaction, profile provenance, and conversion
from private model responses into durable domain artifacts.
Nil options are ignored. Invalid construction, including a nil injected client
or artifact reader, returns an error matching `ErrInvalidConfig`.
## Notarius Ownership
The [source options](../../engine.go) replace their matching directory source:
[LLM Runtime Internals](../internal/llm.md) describes how Notarius mounts
module assets, maps its transport-neutral completion contract, prepares and
executes requests, validates output, records provenance, captures debug
material, redacts errors, and preserves timeout ownership.
[Configuration](../config.md#promptkit-profiles) defines how a Notarius
configuration selects one PromptKit profile source.
- `WithPromptFS` and `WithPromptFile` select prompt definitions;
- `WithProfileFS` and `WithProfileFile` overlay built-in profiles;
- `WithProfiles` adds in-memory profiles ahead of file and built-in profiles;
- `WithSchemaFS` and `WithSchemaFile` select JSON Schema documents;
- `WithLLMClient` replaces the built-in model client; and
- `WithArtifactReader` replaces the default reader for every input.
Source selection, path resolution, strict decoding, profile overlays, and
file-to-request precedence are defined in the
[framework format reference](../formats.md).
Per-generation timeout values from profiles or requests are independent of
the transport cap and caller context. An explicit request value of zero
disables only the per-generation deadline. The
[outbound integration contract](../integrations/openai-compatible-chat.md#timeout-and-cancellation)
defines the complete timeout layering.
## Preparation And Execution
[`Engine.Prepare` and `Engine.Run`](../../engine.go) accept the public
[`RunRequest`](../../types.go). `Prepare` resolves the prompt, profile, input
artifacts, validation contract, and rendered messages without calling an LLM.
`Run` performs the same preparation, calls the configured client, and validates
the generated content. The maintained
[offline preparation example](../../examples/go-library/prepare/main.go)
provides a complete runnable workflow using a prompt file, in-memory profile,
and inline input.
[`PreparedRun` and `RunResult`](../../types.go) expose copied public values.
Preparation returns effective settings, hashes, rendered messages, selected
profile, structured-output information, and timing without resolved secrets or
model output. Execution adds the generated artifact and raw output, validation
state, model metadata, usage, run ID, and duration.
A generated-content validation failure returns a result with
`Validation.Status == ValidationFailed`. An inability to perform validation
returns an error matching `ErrValidation`.
## Requests, Inputs, And Overrides
The [request and value declarations](../../types.go) own the available fields,
serialized constants, and result shapes. Use `File`, `Inline`, or
`InlineWithURI` to construct artifact references. The
[framework format reference](../formats.md) defines declared inputs, template
references, output contracts, and the relationship between file values and
request overrides.
`ExecutionTargetOverride` uses pointers for numeric settings so an explicit
zero remains distinct from no override. `ExtraParams` accepts JSON-compatible
strings, booleans, finite numbers, string-keyed objects, arrays or slices, and
nil. Unsupported values, non-string map keys, non-finite numbers, and cycles
match `ErrInvalidConfig` in profiles or `ErrInvalidRequest` in request
overrides.
Returned requests, profiles, prepared values, results, artifacts, maps, and
slices are isolated from internal engine state. Consumers and injected
extensions should not retain or mutate values owned by another caller.
## Profiles And Credentials
[`OpenAICompatibleProfile`](../../profiles.go) constructs an ordinary
in-memory profile for an OpenAI-compatible chat-completions endpoint.
`WithProfiles` rejects duplicate IDs in one call and gives in-memory profiles
precedence over explicit file sources and built-ins.
Raw API keys do not belong in profiles. File-backed profiles may name an
environment variable, while an in-memory profile can require a request key.
A direct `RunRequest.APIKey` is request-scoped and takes precedence over an
environment lookup for the built-in client. Profile fields, ranges, built-ins,
precedence, and credential rules are owned by the
[framework format reference](../formats.md).
API keys are excluded from JSON, prepared values, and results. The public
`String` and `GoString` methods report only whether a direct key is present.
Avoid reflection-based dumps of request structs, which can bypass that
redaction.
## Extension Interfaces
The [`LLMClient`, `GenerateRequest`, and
`GenerateResponse`](../../types.go) boundary lets a consumer replace model
generation. Injected clients receive copied rendered messages, effective
settings, explicit numeric-setting presence, structured-output constraints,
and the request-scoped key. They return generated content and token usage.
The [`ArtifactReader`](../../types.go) boundary replaces the default inline and
file reader for every input. Readers provide artifact content and metadata; the
engine fills an empty artifact name from the input-map key. A reader error
matches `ErrArtifactLoad` while preserving the original identity for
`errors.Is`. A nil artifact with a nil error is also an artifact-load failure.
Extensions should honor context cancellation and avoid logging raw prompts,
artifacts, or credentials.
## Errors
The [public error declarations](../../engine.go) and
[mapping](../../errors.go) preserve these sentinel checks through `errors.Is`:
- `ErrInvalidConfig`
- `ErrInvalidRequest`
- `ErrPromptNotFound`
- `ErrProfileNotFound`
- `ErrProfileRequired`
- `ErrPromptLoad`
- `ErrProfileLoad`
- `ErrAPIKeyEnvMissing`
- `ErrArtifactLoad`
- `ErrPromptRender`
- `ErrLLMGenerate`
- `ErrValidation`
`ErrProfileRequired` and `ErrAPIKeyEnvMissing` also match
`ErrInvalidRequest`, allowing either broad request handling or a specific
condition. Wrapped collaborator errors retain their identity where the public
contract promises it.
## Consumer Boundary
Promptkit is an importable library. It does not own a command, inbound HTTP
API, process configuration, or deployment policy. Scriptorium is one
downstream application that maps this root package contract into those
application concerns.
PromptKit API or format changes outside this boundary are not implicitly
supported. Updating the pinned version requires reviewing the adapter and
profile/configuration contracts against the upstream documentation.

View File

@@ -39,8 +39,10 @@ in [Configuration Internals](configuration.md).
Configuration validation without a selected pipeline checks structural
configuration only. Validation with a selected pipeline also builds the
effective catalog, resolves the pipeline, and verifies explicitly selected
Scriptorium profiles. Pipeline listing validates configuration before returning
normalized, sorted identifiers.
PromptKit profiles. Each explicit binding or validator profile is prepared
against the configured PromptKit source without performing generation, so an
unknown profile fails before pipeline preparation. Pipeline listing validates
configuration before returning normalized, sorted identifiers.
## Production Composition
@@ -51,12 +53,13 @@ catalog used for resolution and the concrete constructors used for preparation.
Tests may provide a catalog or registries instead; production code must not
silently merge an injected partial catalog with production registrations.
The production LLM factory builds the Scriptorium-backed client from resolved
configuration, creates one scheduler from the effective global LLM limit, and
wraps the client before it reaches modules. Registration and LLM construction
errors are returned before a pipeline is prepared. Concrete module keys and
validator chains are public configuration choices and remain documented in
[Configuration](../config.md).
The production LLM factory builds one PromptKit-backed client from the resolved
**promptkit.profile_dir** or **promptkit.profile_file** source, attaches the
profile-provenance recorder, creates one scheduler from the effective global
LLM limit, and wraps the client before it reaches modules. Registration and LLM
construction errors are returned before a pipeline is prepared. Configuration
field definitions remain in [Configuration](../config.md#promptkit-profiles);
the adapter mechanics remain in [LLM Runtime](llm.md).
## Run Orchestration
@@ -68,7 +71,7 @@ handoff:
2. create and validate a safe run identity, then allocate a debug bundle only
when requested;
3. build the effective catalog, resolve requested reference changes, resolve
the effective pipeline, and verify explicit Scriptorium profiles;
the effective pipeline, and verify explicit PromptKit profiles;
4. materialize external or generated references and record redacted invocation
and resolution provenance when debug capture is enabled;
5. construct registries, the scheduled LLM client, prepared modules, and the

View File

@@ -1,12 +1,12 @@
# LLM Runtime Internals
`internal/framework/llm` is Notariuss provider-independent structured
completion boundary. It adapts framework requests to Scriptorium, bounds
completion boundary. It adapts framework requests to PromptKit, bounds
provider calls, assembles registered prompt and schema assets, records selected
profiles, and redacts provider errors. The architectural boundary is defined in
[Architecture](../policy/architecture.md#llm-boundary); profile sources,
credentials, and concurrency settings belong in
[Configuration](../config.md#scriptorium-profiles) and
[Configuration](../config.md#promptkit-profiles) and
[Configuration](../config.md#concurrency-output-cache-and-debug).
## Structured Completion Boundary
@@ -24,22 +24,26 @@ adapter does not own source evidence, artifact conversion, normalization, or
durable schemas. Those responsibilities remain with the module and its
[integration contract](../integrations/).
`ScriptoriumClient` validates the request target and prompt identity, maps each
named material to a Scriptorium inline artifact while preserving its origin URI,
forwards session and profile selection, then prepares and runs the prompt. It
returns Scriptoriums validated raw bytes rather than re-encoding the decoded
target. An empty optional material is represented as one space so its named
input is retained by Scriptorium.
`PromptKitClient` validates the request target and prompt identity, maps each
named material to a PromptKit inline artifact while preserving its origin URI,
maps the request session to the existing `session_id` prompt variable, forwards
profile selection, then prepares and runs the prompt. PromptKit v0.1.0 has no
direct request-level session field. The adapter returns PromptKits validated
raw bytes rather than re-encoding the decoded target. An empty optional
material is represented as one space so its named input is retained by
PromptKit.
An empty request profile lets the prompt select its configured default. The CLI
prepares every explicitly selected binding profile before a run begins, so a
missing explicit profile fails before stage execution. Calls record the profile
actually selected by Scriptorium; the recorder deduplicates non-secret profile
identity, provider, and model values for manifest use.
actually selected by PromptKit; the recorder deduplicates non-secret profile
identity, provider, and model values for manifest use. Successful completion
responses and recorded profile manifests identify the adapter provider as
`promptkit`.
## Shared Provider-Call Limit
Production construction creates one Scriptorium client and wraps it in one
Production construction creates one PromptKit client and wraps it in one
scheduled client. The scheduler has a fixed, positive permit limit, serves
queued calls in FIFO order, and removes a queued call when its context is
cancelled. A granted permit is released exactly once on every completion path.
@@ -54,7 +58,7 @@ and its effective default are owned by
## Prompt And Schema Assets
An `AssetRegistry` collects prompt and schema filesystems from production module
families. It flattens registered roots into the Scriptorium filesystems and
families. It flattens registered roots into the PromptKit filesystems and
rejects invalid roots, unreadable assets, duplicate paths, and missing prompt
or schema files during preparation. The frameworks `promptfs` helper combines
module-owned prompt files with reusable domain fragments without making the
@@ -95,14 +99,14 @@ meaningful input placement and cache controls of the prompt being changed.
## Validation, Repair, And Retries
Scriptorium performs prompt rendering, provider execution, and the prompts
PromptKit performs prompt rendering, provider execution, and the prompts
structured-output validation. The adapter reports an empty result, validation
failure, empty structured body, or decode failure as
`ErrInvalidStructuredOutput`, while retaining the returned raw bytes and debug
material when they exist. Provider failures remain operational errors rather
than output-validation failures.
Prompt-declared repair is executed within Scriptoriums structured-output flow.
Prompt-declared repair is executed within PromptKits structured-output flow.
The current production D&D prompt manifests set repair attempts to zero. That
setting does not replace pipeline retry behavior: a bindings configured retry
count reruns its stage attempt after an error or rejection, and an exhausted
@@ -111,6 +115,22 @@ attempt lifecycle, validation chains, and retry diagnostics; see
[Pipeline Internals](pipeline.md#validation-retries-and-output) and the
[binding reference](../config.md#module-bindings-and-validators).
## Timeout Ownership
The caller context remains the outer cancellation authority. PromptKit applies
a positive effective generation timeout as an inner request deadline; an
explicit zero disables only that generation deadline. The HTTP client timeout
is a separate transport-wide cap. Notarius forwards the caller context and
does not install another timeout wrapper around PromptKit.
The selected PromptKit profile owns generation settings. Notarius binding
retries remain outside the adapter and repeat the complete module operation
and validation chain. PromptKit v0.1.0 does not add a provider retry loop.
Operator-facing behavior is summarized in
[Operations](../operations.md#operational-limits), and the pinned upstream
contract is identified in
[PromptKit Integration](../integrations/pkg-promptkit.md).
## Observability And Redaction
When debug recording is enabled, the pipeline decorates the shared client. The

View File

@@ -207,9 +207,19 @@ or automatic cleanup command.
## Operational Limits
Provider retries and timeouts are supplied by the selected Scriptorium profile.
Module retry settings and concurrency limits are configuration contracts; see
[module bindings](config.md#module-bindings-and-validators) and
Provider execution settings and the generation timeout come from the selected
PromptKit profile. PromptKit v0.1.0 does not add a provider retry loop;
Notarius binding retries rerun the complete module operation and validation
chain as defined by [module bindings](config.md#module-bindings-and-validators).
Timeouts are layered. Caller cancellation is the outer authority. A positive
effective generation timeout adds an inner request deadline, while zero
disables only that generation deadline. The HTTP client timeout remains a
transport-wide cap. Notarius does not add another timeout around PromptKit.
The pinned upstream boundary and profile-format links are in
[PromptKit Integration](integrations/pkg-promptkit.md).
Concurrency limits are configuration contracts; see
[concurrency](config.md#concurrency-output-cache-and-debug). Extract-worker
limits and actual provider-call limits are independent. Notarius writes local
filesystem state only; remote storage, archival, and retention automation are

View File

@@ -95,9 +95,9 @@ safety checks, and deterministic application of accepted changes.
### Native Session Propagation
- Once upstream Scriptorium exposes a session identifier on its run request,
- Once upstream PromptKit exposes a direct request-level session identifier,
propagate the existing `StructuredCompletionRequest.SessionID` through the
Scriptorium adapter's native session field.
PromptKit adapter's native session field.
- Preserve the current `--session-id` invocation contract and its run-wide
propagation to every prompt-facing module and validator. Do not introduce a
second session configuration surface.
@@ -110,10 +110,11 @@ safety checks, and deterministic application of accepted changes.
concurrent-run isolation, and unsupported-provider behavior once the
upstream contract is available.
This work is blocked on native session support in the upstream Scriptorium
package. Notarius already carries a run-scoped session ID through its CLI,
pipeline requests, checkpoint identity, and prompt variables; the missing
capability is native propagation across the LLM adapter boundary.
This work is blocked because PromptKit v0.1.0 does not expose the required
direct request-level session field. Notarius already carries a run-scoped
session ID through its CLI, pipeline requests, checkpoint identity, and a
`session_id` prompt variable; that prompt-variable propagation is not native
provider session support.
## Further Reference Evolution

View File

@@ -2,7 +2,10 @@
## Status
Planned.
In progress. The dependency, framework adapter, version 4 PromptKit
configuration migration, provider-neutral module prompt-asset support,
provenance alignment, and canonical documentation are implemented; final
repository verification is still planned.
## Objective

View File

@@ -1,4 +1,4 @@
version: 3
version: 4
concurrency:
total_llm: 2
stage_workers:

View File

@@ -1,4 +1,4 @@
version: 3
version: 4
pipelines:
dnd-session:
input: seriatim

2
go.mod
View File

@@ -3,7 +3,7 @@ module gitea.maximumdirect.net/eric/notarius
go 1.25.5
require (
gitea.maximumdirect.net/eric/scriptorium v0.11.1
gitea.maximumdirect.net/eric/promptkit v0.1.0
github.com/santhosh-tekuri/jsonschema/v6 v6.0.2
gopkg.in/yaml.v3 v3.0.1
)

8
go.sum
View File

@@ -1,13 +1,9 @@
gitea.maximumdirect.net/eric/scriptorium v0.11.0 h1:rjvbt9FTaWHxYlHq7QlUzmMVUt3QdbTmeCkmH81N//o=
gitea.maximumdirect.net/eric/scriptorium v0.11.0/go.mod h1:FQ5lEuNxmrQyNgIomkpZdxvfTC0jWjbXYuq3tbJWF64=
gitea.maximumdirect.net/eric/scriptorium v0.11.1 h1:zBKtB3+fP8FcHGI8DJD99CiTL6crAGitBhWtE+xYJHc=
gitea.maximumdirect.net/eric/scriptorium v0.11.1/go.mod h1:FQ5lEuNxmrQyNgIomkpZdxvfTC0jWjbXYuq3tbJWF64=
gitea.maximumdirect.net/eric/promptkit v0.1.0 h1:vuKeBxkiY8E54LRFbLQFjlJJCiOfMvB1++DYBCrD/ug=
gitea.maximumdirect.net/eric/promptkit v0.1.0/go.mod h1:R95NM6fbMDGDC0/UomgnSBP6ui2ns+8SZb8bESNvrDQ=
github.com/dlclark/regexp2 v1.11.0 h1:G/nrcoOa7ZXlpoa/91N3X7mM3r8eIlMBBJZvsz/mxKI=
github.com/dlclark/regexp2 v1.11.0/go.mod h1:DHkYz0B9wPfa6wondMfaivmHpzrQ3v9q8cnmRbL6yW8=
github.com/santhosh-tekuri/jsonschema/v6 v6.0.2 h1:KRzFb2m7YtdldCEkzs6KqmJw4nqEVZGK7IN2kJkjTuQ=
github.com/santhosh-tekuri/jsonschema/v6 v6.0.2/go.mod h1:JXeL+ps8p7/KNMjDQk3TCwPpBy0wYklyWTfbkIzdIFU=
golang.org/x/text v0.14.0 h1:ScX5w1eTa3QqT8oi6+ziP7dTV1S2+ALU0bI+0zXKWiQ=
golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
golang.org/x/text v0.40.0 h1:Ub2Z6/xjgF1WrYQz2nuITOEegKFtiIy+rieRJ5lHZKs=
golang.org/x/text v0.40.0/go.mod h1:hpnzDAfGV753zIKo+wk3u1bVKCGPbrnF7+7LBF/UHVY=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=

View File

@@ -168,14 +168,14 @@ func buildProductionLLMClient(ctx context.Context, cfg config.Config, profileID
return nil, nil, fmt.Errorf("production asset registry must not be nil")
}
recorder := llm.NewLLMProfileRecorder()
client, err := llm.NewScriptoriumClient(llm.ScriptoriumClientConfig{
ProfileDir: cfg.Scriptorium.ProfileDir,
ProfileFile: cfg.Scriptorium.ProfileFile,
client, err := llm.NewPromptKitClient(llm.PromptKitClientConfig{
ProfileDir: cfg.PromptKit.ProfileDir,
ProfileFile: cfg.PromptKit.ProfileFile,
Assets: assets,
Recorder: recorder,
})
if err != nil {
return nil, nil, fmt.Errorf("create Scriptorium-backed LLM client: %w", err)
return nil, nil, fmt.Errorf("create PromptKit-backed LLM client: %w", err)
}
scheduler, err := llm.NewScheduler(cfg.Concurrency.TotalLLM)
if err != nil {

View File

@@ -214,13 +214,13 @@ func commandContractOptionsWithLookup(t *testing.T, lookup func(string) (string,
func writeCommandConfig(t *testing.T, firstID, secondID string) string {
t.Helper()
content := fmt.Sprintf("version: 3\npipelines:\n %q:\n input: seriatim\n %q:\n input: seriatim\n", firstID, secondID)
content := fmt.Sprintf("version: 4\npipelines:\n %q:\n input: seriatim\n %q:\n input: seriatim\n", firstID, secondID)
return writeCommandConfigContent(t, content)
}
func writeResolvableCommandConfig(t *testing.T) string {
t.Helper()
return writeCommandConfigContent(t, `version: 3
return writeCommandConfigContent(t, `version: 4
pipelines:
demo:
input: seriatim

View File

@@ -6,8 +6,10 @@ import (
"fmt"
"reflect"
"strings"
"sync"
"testing"
"gitea.maximumdirect.net/eric/notarius/internal/core/artifacts"
"gitea.maximumdirect.net/eric/notarius/internal/core/config"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
@@ -28,7 +30,7 @@ func TestProductionSceneDescriptionWorkflow(t *testing.T) {
Output: pipeline.Binding("json"),
Artifacts: map[string]pipeline.ArtifactLaneProfile{
"scene-descriptions": {
Extract: pipeline.Binding(sceneextract.Key),
Extract: pipeline.ModuleBinding{Module: sceneextract.Key, LLMProfile: "scene-description-profile"},
Normalize: pipeline.Binding(scenenormalize.Key),
},
},
@@ -45,7 +47,8 @@ func TestProductionSceneDescriptionWorkflow(t *testing.T) {
t.Fatalf("resolved references = %#v / %#v, want no generated or required references", lane.ExtractReferences, lane.NormalizeReferences)
}
prepared, err := pipeline.Prepare(effective.ResolvedPipeline, components.registries, pipeline.ModuleDependencies{LLM: sceneDescriptionLLM{}})
llmClient := &sceneDescriptionLLM{}
prepared, err := pipeline.Prepare(effective.ResolvedPipeline, components.registries, pipeline.ModuleDependencies{LLM: llmClient})
if err != nil {
t.Fatalf("Prepare() error = %v", err)
}
@@ -60,6 +63,14 @@ func TestProductionSceneDescriptionWorkflow(t *testing.T) {
if output.Manifest.ValidationStatus != "approved" || len(output.Rejected) != 0 || len(output.NormalizeOutputs) != 1 {
t.Fatalf("run output = %#v, want one approved normalized artifact", output)
}
wantProfiles := []artifacts.LLMProfileManifest{{
ID: "scene-description-profile",
Provider: "promptkit",
Model: "deterministic",
}}
if !reflect.DeepEqual(output.Manifest.LLMProfiles, wantProfiles) {
t.Fatalf("manifest LLM profiles = %#v, want %#v", output.Manifest.LLMProfiles, wantProfiles)
}
normalizedOutput := output.NormalizeOutputs[0]
if normalizedOutput.NormalizerKey != scenenormalize.Key || normalizedOutput.Artifact.Kind != dnd.SceneDescriptionListKind || normalizedOutput.Artifact.Schema.ID != scenecodec.SchemaID || normalizedOutput.Artifact.Schema.Name != scenecodec.SchemaName || normalizedOutput.Artifact.Schema.Version != scenecodec.SchemaVersion {
t.Fatalf("normalized output = %#v, want registered durable scene-description schema", normalizedOutput)
@@ -85,9 +96,12 @@ func TestProductionSceneDescriptionWorkflow(t *testing.T) {
}
}
type sceneDescriptionLLM struct{}
type sceneDescriptionLLM struct {
mu sync.Mutex
profile *artifacts.LLMProfileManifest
}
func (sceneDescriptionLLM) CompleteStructured(ctx context.Context, req contracts.StructuredCompletionRequest, out any) (contracts.StructuredCompletionResponse, error) {
func (client *sceneDescriptionLLM) CompleteStructured(ctx context.Context, req contracts.StructuredCompletionRequest, out any) (contracts.StructuredCompletionResponse, error) {
if err := ctx.Err(); err != nil {
return contracts.StructuredCompletionResponse{}, err
}
@@ -107,5 +121,27 @@ func (sceneDescriptionLLM) CompleteStructured(ctx context.Context, req contracts
if err := json.Unmarshal([]byte(content), out); err != nil {
return contracts.StructuredCompletionResponse{}, fmt.Errorf("populate structured response: %w", err)
}
return contracts.StructuredCompletionResponse{Content: []byte(content), Provider: "test", Model: "deterministic", ProfileID: req.ProfileID}, nil
profile := artifacts.LLMProfileManifest{
ID: req.ProfileID,
Provider: "promptkit",
Model: "deterministic",
}
client.mu.Lock()
client.profile = &profile
client.mu.Unlock()
return contracts.StructuredCompletionResponse{
Content: []byte(content),
Provider: profile.Provider,
Model: profile.Model,
ProfileID: profile.ID,
}, nil
}
func (client *sceneDescriptionLLM) LLMProfileManifests() []artifacts.LLMProfileManifest {
client.mu.Lock()
defer client.mu.Unlock()
if client.profile == nil {
return nil
}
return []artifacts.LLMProfileManifest{*client.profile}
}

View File

@@ -23,7 +23,7 @@ func TestOversizedNPCRegistryFailsBeforeRuntimeAndCheckpointConstruction(t *test
t.Fatal(err)
}
checkpointRoot := filepath.Join(t.TempDir(), "checkpoints")
content := fmt.Sprintf(`version: 3
content := fmt.Sprintf(`version: 4
cache:
chunk_plans:
mode: bypass

View File

@@ -583,7 +583,7 @@ func maintainedExampleFiles(t *testing.T) []maintainedExample {
func productionSpellCatalogContractConfig(t *testing.T) string {
t.Helper()
return fmt.Sprintf(`version: 3
return fmt.Sprintf(`version: 4
cache:
chunk_plans:
mode: bypass
@@ -680,7 +680,7 @@ func productionRunOptions(t *testing.T, fake *productionFakeLLMClient) Options {
}
func productionRunConfig(outputRoot, chunkModule string) string {
return fmt.Sprintf(`version: 3
return fmt.Sprintf(`version: 4
output:
directory: %q
cache:

View File

@@ -0,0 +1,68 @@
package cli
import (
"context"
"errors"
"fmt"
"testing/fstest"
"gitea.maximumdirect.net/eric/notarius/internal/core/config"
"gitea.maximumdirect.net/eric/promptkit"
)
const profileCheckPromptID = "notarius.profile.check"
var profileCheckPromptFS = fstest.MapFS{
"prompts/profile-check.yaml": &fstest.MapFile{Data: []byte(`id: notarius.profile.check
version: "1.0.0"
default_profile: mistral-small-3
inputs:
- name: transcript
required: true
messages:
- role: user
content: "{{input \"transcript\"}}"
output:
format: text
validation_mode: none
repair_attempts: 0
`)},
}
func validateExplicitPromptKitProfiles(ctx context.Context, cfg config.Config, profileIDs []string) error {
if len(profileIDs) == 0 {
return nil
}
engine, err := newProfileValidationEngine(cfg)
if err != nil {
return fmt.Errorf("load PromptKit profiles: %w", err)
}
for _, profileID := range profileIDs {
if _, err := engine.Prepare(ctx, promptkit.RunRequest{
PromptID: profileCheckPromptID,
ProfileID: profileID,
Inputs: map[string]promptkit.ArtifactRef{
"transcript": promptkit.Inline("profile check"),
},
}); err != nil {
if errors.Is(err, promptkit.ErrProfileNotFound) {
return fmt.Errorf("PromptKit profile %q is not configured", profileID)
}
return fmt.Errorf("validate PromptKit profile %q: %w", profileID, err)
}
}
return nil
}
func newProfileValidationEngine(cfg config.Config) (*promptkit.Engine, error) {
opts := []promptkit.Option{
promptkit.WithPromptFS(profileCheckPromptFS, "prompts"),
}
if cfg.PromptKit.ProfileFile != "" {
opts = append(opts, promptkit.WithProfileFile(cfg.PromptKit.ProfileFile))
}
return promptkit.NewEngine(promptkit.Config{
PromptDir: "unused",
ProfileDir: cfg.PromptKit.ProfileDir,
}, opts...)
}

View File

@@ -194,7 +194,7 @@ func (recomputeTestOutput) Encode(_ context.Context, req contracts.OutputRequest
func newRecomputeTestRoots(t *testing.T) stateTestRoots {
t.Helper()
roots := newStateTestRoots(t)
config := fmt.Sprintf(`version: 3
config := fmt.Sprintf(`version: 4
output:
directory: %q
cache:

View File

@@ -217,7 +217,7 @@ func TestReferenceMaterializationSeparatesCLIAndConfigPathOrigins(t *testing.T)
workingDir := t.TempDir()
cfg := referenceContractConfig()
configPath := filepath.Join(configDir, "config.yml")
if err := os.WriteFile(configPath, []byte("version: 3\n"), 0o600); err != nil {
if err := os.WriteFile(configPath, []byte("version: 4\n"), 0o600); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(configDir, "required.txt"), []byte("config reference"), 0o600); err != nil {

View File

@@ -315,7 +315,7 @@ func runPipelineCommand(args []string, stdout, stderr io.Writer, opts Options) i
return failPipelineCommand(stderr, commandState, terminalWriter, err)
}
profileIDs := effectiveLLMProfileIDs(effective.ResolvedPipeline)
if err := validateExplicitScriptoriumProfiles(context.Background(), effective.Config, profileIDs); err != nil {
if err := validateExplicitPromptKitProfiles(context.Background(), effective.Config, profileIDs); err != nil {
return failPipelineCommand(stderr, commandState, terminalWriter, err)
}
workingDir, err := os.Getwd()
@@ -979,7 +979,7 @@ func runConfigValidate(args []string, stdout, stderr io.Writer, opts Options) in
fmt.Fprintf(stderr, "notarius: %v\n", err)
return 1
}
if err := validateExplicitScriptoriumProfiles(context.Background(), effective.Config, effectiveLLMProfileIDs(effective.ResolvedPipeline)); err != nil {
if err := validateExplicitPromptKitProfiles(context.Background(), effective.Config, effectiveLLMProfileIDs(effective.ResolvedPipeline)); err != nil {
fmt.Fprintf(stderr, "notarius: %v\n", err)
return 1
}

View File

@@ -241,7 +241,7 @@ func TestRunLLMProfileOverrideAndValidationUseInjectedBoundaries(t *testing.T) {
t.Run("one effective profile reaches the factory and modules", func(t *testing.T) {
roots := newStateTestRoots(t)
profileDir := writeRunContractProfiles(t, "override-profile")
prependRunContractConfig(t, roots, fmt.Sprintf("scriptorium:\n profile_dir: %q\n", profileDir))
prependRunContractConfig(t, roots, fmt.Sprintf("promptkit:\n profile_dir: %q\n", profileDir))
harness := newStateTestHarness()
var factoryProfiles []string
opts := harness.options()
@@ -273,7 +273,7 @@ func TestRunLLMProfileOverrideAndValidationUseInjectedBoundaries(t *testing.T) {
t.Run("validator profile remains distinct", func(t *testing.T) {
roots := newStateTestRoots(t)
profileDir := writeRunContractProfiles(t, "override-profile", "validator-profile")
prependRunContractConfig(t, roots, fmt.Sprintf("scriptorium:\n profile_dir: %q\n", profileDir))
prependRunContractConfig(t, roots, fmt.Sprintf("promptkit:\n profile_dir: %q\n", profileDir))
harness := newStateTestHarness()
var validatorProfiles []string
opts := harness.options()
@@ -299,7 +299,7 @@ func TestRunLLMProfileOverrideAndValidationUseInjectedBoundaries(t *testing.T) {
t.Run("unknown profile is rejected without factory access", func(t *testing.T) {
roots := newStateTestRoots(t)
profileDir := writeRunContractProfiles(t, "override-profile")
prependRunContractConfig(t, roots, fmt.Sprintf("scriptorium:\n profile_dir: %q\n", profileDir))
prependRunContractConfig(t, roots, fmt.Sprintf("promptkit:\n profile_dir: %q\n", profileDir))
factoryCalls := 0
opts := newStateTestHarness().options()
opts.LLMClientFactory = func(context.Context, config.Config, string) (contracts.StructuredLLMClient, []artifacts.LLMProfileManifest, error) {

View File

@@ -1,68 +0,0 @@
package cli
import (
"context"
"errors"
"fmt"
"testing/fstest"
"gitea.maximumdirect.net/eric/notarius/internal/core/config"
"gitea.maximumdirect.net/eric/scriptorium"
)
const profileCheckPromptID = "notarius.profile.check"
var profileCheckPromptFS = fstest.MapFS{
"prompts/profile-check.yaml": &fstest.MapFile{Data: []byte(`id: notarius.profile.check
version: "1.0.0"
default_profile: mistral-small-3
inputs:
- name: transcript
required: true
messages:
- role: user
content: "{{input \"transcript\"}}"
output:
format: text
validation_mode: none
repair_attempts: 0
`)},
}
func validateExplicitScriptoriumProfiles(ctx context.Context, cfg config.Config, profileIDs []string) error {
if len(profileIDs) == 0 {
return nil
}
engine, err := newProfileValidationEngine(cfg)
if err != nil {
return fmt.Errorf("load Scriptorium profiles: %w", err)
}
for _, profileID := range profileIDs {
if _, err := engine.Prepare(ctx, scriptorium.RunRequest{
PromptID: profileCheckPromptID,
ProfileID: profileID,
Inputs: map[string]scriptorium.ArtifactRef{
"transcript": scriptorium.Inline("profile check"),
},
}); err != nil {
if errors.Is(err, scriptorium.ErrProfileNotFound) {
return fmt.Errorf("Scriptorium profile %q is not configured", profileID)
}
return fmt.Errorf("validate Scriptorium profile %q: %w", profileID, err)
}
}
return nil
}
func newProfileValidationEngine(cfg config.Config) (*scriptorium.Engine, error) {
opts := []scriptorium.Option{
scriptorium.WithPromptFS(profileCheckPromptFS, "prompts"),
}
if cfg.Scriptorium.ProfileFile != "" {
opts = append(opts, scriptorium.WithProfileFile(cfg.Scriptorium.ProfileFile))
}
return scriptorium.NewEngine(scriptorium.Config{
PromptDir: "unused",
ProfileDir: cfg.Scriptorium.ProfileDir,
}, opts...)
}

View File

@@ -635,7 +635,7 @@ func newStateTestRoots(t *testing.T) stateTestRoots {
t.Fatal(err)
}
roots.config = filepath.Join(base, "config.yml")
config := fmt.Sprintf("version: 3\noutput:\n directory: %q\ncache:\n chunk_plans:\n directory: %q\n mode: auto\n checkpoints:\n enabled: true\n directory: %q\ndebug:\n directory: %q\npipelines:\n sample:\n input: test/input\n chunk: test/chunk\n artifacts:\n items:\n extract: test/extract\n merge: test/merge\n normalize: test/normalize\n output: test/output\n", roots.output, roots.plans, roots.checkpoints, roots.debug)
config := fmt.Sprintf("version: 4\noutput:\n directory: %q\ncache:\n chunk_plans:\n directory: %q\n mode: auto\n checkpoints:\n enabled: true\n directory: %q\ndebug:\n directory: %q\npipelines:\n sample:\n input: test/input\n chunk: test/chunk\n artifacts:\n items:\n extract: test/extract\n merge: test/merge\n normalize: test/normalize\n output: test/output\n", roots.output, roots.plans, roots.checkpoints, roots.debug)
if err := os.WriteFile(roots.config, []byte(config), 0o600); err != nil {
t.Fatal(err)
}

View File

@@ -53,7 +53,7 @@ func TestRunManifestIncludesPipelineAndArtifactLaneFields(t *testing.T) {
PipelineID: "pipeline-1",
PipelineDigest: "sha256:abc123",
LLMProfiles: []LLMProfileManifest{
{ID: "default", Provider: "scriptorium", Model: "model-a"},
{ID: "default", Provider: "promptkit", Model: "model-a"},
},
ArtifactLanes: []ArtifactLaneManifest{
{
@@ -102,6 +102,9 @@ func TestRunManifestIncludesPipelineAndArtifactLaneFields(t *testing.T) {
t.Fatalf("llm_profiles[0] = %#v, want object", profiles[0])
}
assertHasKeys(t, profile, "id", "provider", "model")
if profile["provider"] != "promptkit" {
t.Fatalf("llm_profiles[0].provider = %#v, want promptkit", profile["provider"])
}
lanes, ok := got["artifact_lanes"].([]any)
if !ok {

View File

@@ -4,10 +4,10 @@ import (
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
)
const SupportedFileConfigVersion = 3
const SupportedFileConfigVersion = 4
type Config struct {
Scriptorium ScriptoriumConfig `json:"scriptorium,omitempty"`
PromptKit PromptKitConfig `json:"promptkit,omitempty"`
Pipelines map[string]pipeline.PipelineProfile `json:"pipelines"`
Concurrency ConcurrencyConfig `json:"concurrency"`
Output OutputConfig `json:"output"`
@@ -15,7 +15,7 @@ type Config struct {
Debug DebugConfig `json:"debug"`
}
type ScriptoriumConfig struct {
type PromptKitConfig struct {
ProfileDir string `json:"profile_dir,omitempty"`
ProfileFile string `json:"profile_file,omitempty"`
}

View File

@@ -84,6 +84,29 @@ func TestEffectiveConfigMaterializesDefaultBindingsThroughCatalog(t *testing.T)
}
}
func TestEffectiveConfigPreservesPromptKitProfileSource(t *testing.T) {
tests := []struct {
name string
profileSource PromptKitConfig
}{
{name: "profile directory", profileSource: PromptKitConfig{ProfileDir: "./profiles"}},
{name: "profile file", profileSource: PromptKitConfig{ProfileFile: "./profiles.yml"}},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
cfg := configForEffectiveTests(t, effectiveProfile())
cfg.PromptKit = tt.profileSource
effective, err := cfg.Resolve(ResolveInput{PipelineID: "main", Catalog: effectiveCatalog(t)})
if err != nil {
t.Fatalf("Resolve() error = %v", err)
}
if effective.Config.PromptKit != cfg.PromptKit {
t.Fatalf("effective PromptKit config = %#v, want %#v", effective.Config.PromptKit, cfg.PromptKit)
}
})
}
}
func TestEffectiveConfigResolutionFailuresRetainContext(t *testing.T) {
tests := []struct {
name string

View File

@@ -10,7 +10,7 @@ import (
)
func TestPrecedenceFileValuesOverrideBuiltInDefaults(t *testing.T) {
cfg := applyFileConfig(t, `version: 3
cfg := applyFileConfig(t, `version: 4
concurrency:
total_llm: 4
stage_workers:
@@ -35,7 +35,7 @@ debug:
}
func TestPrecedenceOperationalEnvironmentOverridesFileValues(t *testing.T) {
cfg := applyFileConfig(t, `version: 3
cfg := applyFileConfig(t, `version: 4
concurrency:
total_llm: 2
stage_workers:
@@ -81,21 +81,21 @@ func TestPrecedenceExtractWorkersFollowEffectiveConcurrencyUnlessExplicit(t *tes
}{
{
name: "default follows environment total",
file: "version: 3\n",
file: "version: 4\n",
env: map[string]string{"NOTARIUS_TOTAL_LLM_CONCURRENCY": "5"},
wantTotal: 5,
wantWorker: 5,
},
{
name: "file worker is retained",
file: "version: 3\nconcurrency:\n total_llm: 3\n stage_workers:\n extract: 2\n",
file: "version: 4\nconcurrency:\n total_llm: 3\n stage_workers:\n extract: 2\n",
env: map[string]string{"NOTARIUS_TOTAL_LLM_CONCURRENCY": "6"},
wantTotal: 6,
wantWorker: 2,
},
{
name: "environment worker is retained",
file: "version: 3\nconcurrency:\n total_llm: 2\n",
file: "version: 4\nconcurrency:\n total_llm: 2\n",
env: map[string]string{
"NOTARIUS_TOTAL_LLM_CONCURRENCY": "6",
"NOTARIUS_STAGE_WORKERS_EXTRACT": "4",
@@ -118,7 +118,7 @@ func TestPrecedenceExtractWorkersFollowEffectiveConcurrencyUnlessExplicit(t *tes
}
func TestPrecedenceEmptyFileCacheDirectoriesDeferPerUserResolution(t *testing.T) {
cfg := applyFileConfig(t, `version: 3
cfg := applyFileConfig(t, `version: 4
cache:
chunk_plans:
directory: ""

View File

@@ -14,7 +14,7 @@ import (
type FileConfig struct {
Version int `yaml:"version"`
Scriptorium *FileScriptoriumConfig `yaml:"scriptorium,omitempty"`
PromptKit *FilePromptKitConfig `yaml:"promptkit,omitempty"`
Pipelines map[string]FilePipelineProfile `yaml:"pipelines,omitempty"`
Concurrency *FileConcurrencyConfig `yaml:"concurrency,omitempty"`
Output *FileOutputConfig `yaml:"output,omitempty"`
@@ -22,7 +22,7 @@ type FileConfig struct {
Debug *FileDebugConfig `yaml:"debug,omitempty"`
}
type FileScriptoriumConfig struct {
type FilePromptKitConfig struct {
ProfileDir *string `yaml:"profile_dir,omitempty"`
ProfileFile *string `yaml:"profile_file,omitempty"`
}
@@ -325,6 +325,9 @@ func ParseFileConfigYAML(data []byte) (FileConfig, error) {
if header.Version == 2 {
return FileConfig{}, fmt.Errorf("config version 2 is no longer supported; migrate the file using the version 2-to-3 migration in docs/config.md")
}
if header.Version == 3 {
return FileConfig{}, fmt.Errorf("config version 3 is no longer supported; change \"version: 3\" to \"version: 4\" and rename \"scriptorium:\" to \"promptkit:\"")
}
if header.Version != SupportedFileConfigVersion {
return FileConfig{}, fmt.Errorf("unsupported config version %d (supported version is %d)", header.Version, SupportedFileConfigVersion)
}
@@ -450,20 +453,20 @@ func (c *Config) applyFileConfigWithLookup(fileCfg FileConfig, lookup func(strin
}
}
if fileCfg.Scriptorium != nil {
if fileCfg.Scriptorium.ProfileDir != nil {
value := strings.TrimSpace(*fileCfg.Scriptorium.ProfileDir)
if fileCfg.PromptKit != nil {
if fileCfg.PromptKit.ProfileDir != nil {
value := strings.TrimSpace(*fileCfg.PromptKit.ProfileDir)
if value == "" {
return fmt.Errorf("scriptorium.profile_dir must not be empty when set")
return fmt.Errorf("promptkit.profile_dir must not be empty when set")
}
c.Scriptorium.ProfileDir = value
c.PromptKit.ProfileDir = value
}
if fileCfg.Scriptorium.ProfileFile != nil {
value := strings.TrimSpace(*fileCfg.Scriptorium.ProfileFile)
if fileCfg.PromptKit.ProfileFile != nil {
value := strings.TrimSpace(*fileCfg.PromptKit.ProfileFile)
if value == "" {
return fmt.Errorf("scriptorium.profile_file must not be empty when set")
return fmt.Errorf("promptkit.profile_file must not be empty when set")
}
c.Scriptorium.ProfileFile = value
c.PromptKit.ProfileFile = value
}
}

View File

@@ -1,6 +1,7 @@
package config
import (
"encoding/json"
"os"
"path/filepath"
"reflect"
@@ -34,8 +35,8 @@ func TestDefaultReturnsDocumentedValuesAndIndependentMaps(t *testing.T) {
}
}
func TestFileConfigMinimalVersion3AppliesOverDefaults(t *testing.T) {
file := parseFileConfig(t, "version: 3\n")
func TestFileConfigMinimalVersion4AppliesOverDefaults(t *testing.T) {
file := parseFileConfig(t, "version: 4\n")
cfg := Default()
if err := cfg.ApplyFileConfig(file); err != nil {
t.Fatal(err)
@@ -48,6 +49,78 @@ func TestFileConfigMinimalVersion3AppliesOverDefaults(t *testing.T) {
}
}
func TestFilePromptKitProfileSourcesSurviveConfigBoundaries(t *testing.T) {
tests := []struct {
name string
yaml string
want PromptKitConfig
}{
{
name: "profile directory",
yaml: "version: 4\npromptkit:\n profile_dir: ' ./profiles '\n",
want: PromptKitConfig{ProfileDir: "./profiles"},
},
{
name: "profile file",
yaml: "version: 4\npromptkit:\n profile_file: ' ./profiles.yml '\n",
want: PromptKitConfig{ProfileFile: "./profiles.yml"},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
cfg := applyFileConfig(t, tt.yaml)
if cfg.PromptKit != tt.want {
t.Fatalf("PromptKit config = %#v, want %#v", cfg.PromptKit, tt.want)
}
if got := cloneConfig(cfg).PromptKit; got != tt.want {
t.Fatalf("cloned PromptKit config = %#v, want %#v", got, tt.want)
}
if got := cfg.Redacted().PromptKit; got != tt.want {
t.Fatalf("redacted PromptKit config = %#v, want %#v", got, tt.want)
}
data, err := json.Marshal(cfg)
if err != nil {
t.Fatalf("json.Marshal() error = %v", err)
}
var payload map[string]json.RawMessage
if err := json.Unmarshal(data, &payload); err != nil {
t.Fatalf("json.Unmarshal() error = %v", err)
}
if _, ok := payload["promptkit"]; !ok {
t.Fatalf("runtime JSON keys = %v, want promptkit", payload)
}
if _, ok := payload["scriptorium"]; ok {
t.Fatalf("runtime JSON keys = %v, must not contain removed section", payload)
}
})
}
}
func TestFilePromptKitExplicitEmptyProfileSourcesAreRejected(t *testing.T) {
for _, field := range []string{"profile_dir", "profile_file"} {
t.Run(field, func(t *testing.T) {
file := parseFileConfig(t, "version: 4\npromptkit:\n "+field+": ''\n")
cfg := Default()
err := cfg.ApplyFileConfig(file)
if err == nil || !strings.Contains(err.Error(), "promptkit."+field+" must not be empty") {
t.Fatalf("ApplyFileConfig() error = %v, want explicit-empty rejection", err)
}
})
}
}
func TestFilePromptKitProfileSourcesRemainMutuallyExclusive(t *testing.T) {
cfg := applyFileConfig(t, `version: 4
promptkit:
profile_dir: ./profiles
profile_file: ./profiles.yml
`)
if err := cfg.Validate(); err == nil || !strings.Contains(err.Error(), "promptkit profile_dir and profile_file are mutually exclusive") {
t.Fatalf("Validate() error = %v, want mutually exclusive profile sources", err)
}
}
func TestFileConfigMissingVersionIsReportedBeforeFieldDecoding(t *testing.T) {
_, err := ParseFileConfigYAML([]byte("workspace:\n directory: /tmp/old\n"))
if err == nil || !strings.Contains(err.Error(), "config version is required") {
@@ -55,6 +128,15 @@ func TestFileConfigMissingVersionIsReportedBeforeFieldDecoding(t *testing.T) {
}
}
func TestFileConfigVersion3ReportsPromptKitMigration(t *testing.T) {
_, err := ParseFileConfigYAML([]byte("version: 3\nscriptorium:\n profile_dir: ./profiles\n"))
if err == nil ||
!strings.Contains(err.Error(), `change "version: 3" to "version: 4"`) ||
!strings.Contains(err.Error(), `rename "scriptorium:" to "promptkit:"`) {
t.Fatalf("version 3 error = %v, want actionable version and section migration", err)
}
}
func TestFileConfigRejectsUnknownCurrentAndRemovedFields(t *testing.T) {
tests := []struct {
name string
@@ -63,14 +145,19 @@ func TestFileConfigRejectsUnknownCurrentAndRemovedFields(t *testing.T) {
}{
{
name: "removed diagnostics",
yaml: "version: 3\ndiagnostics: {}\n",
yaml: "version: 4\ndiagnostics: {}\n",
want: "field diagnostics not found",
},
{
name: "removed llm profiles",
yaml: "version: 3\nllm_profiles: {}\n",
yaml: "version: 4\nllm_profiles: {}\n",
want: "field llm_profiles not found",
},
{
name: "removed scriptorium section",
yaml: "version: 4\nscriptorium: {}\n",
want: "field scriptorium not found",
},
{
name: "version 2 migration",
yaml: "version: 2\nworkspace:\n directory: /tmp/old\n",
@@ -78,27 +165,27 @@ func TestFileConfigRejectsUnknownCurrentAndRemovedFields(t *testing.T) {
},
{
name: "pipeline field",
yaml: "version: 3\npipelines:\n main:\n unknown: true\n",
yaml: "version: 4\npipelines:\n main:\n unknown: true\n",
want: "field unknown not found",
},
{
name: "lane field",
yaml: "version: 3\npipelines:\n main:\n artifacts:\n spells:\n unknown: true\n",
yaml: "version: 4\npipelines:\n main:\n artifacts:\n spells:\n unknown: true\n",
want: "field unknown not found",
},
{
name: "module binding field",
yaml: "version: 3\npipelines:\n main:\n input:\n module: seriatim\n unknown: true\n",
yaml: "version: 4\npipelines:\n main:\n input:\n module: seriatim\n unknown: true\n",
want: "field unknown not found in module binding",
},
{
name: "checkpoint field",
yaml: "version: 3\ncache:\n checkpoints:\n unknown: true\n",
yaml: "version: 4\ncache:\n checkpoints:\n unknown: true\n",
want: "field unknown not found",
},
{
name: "checkpoint enabled type",
yaml: "version: 3\ncache:\n checkpoints:\n enabled: definitely\n",
yaml: "version: 4\ncache:\n checkpoints:\n enabled: definitely\n",
want: "cannot unmarshal",
},
}
@@ -113,7 +200,7 @@ func TestFileConfigRejectsUnknownCurrentAndRemovedFields(t *testing.T) {
}
func TestFileConfigModuleBindingsPreserveFormsAndValidatorPresence(t *testing.T) {
cfg := applyFileConfig(t, `version: 3
cfg := applyFileConfig(t, `version: 4
pipelines:
main:
input: seriatim
@@ -160,7 +247,7 @@ pipelines:
}
func TestFileConfigReferencePrecedenceIsRetained(t *testing.T) {
cfg := applyFileConfig(t, `version: 3
cfg := applyFileConfig(t, `version: 4
pipelines:
main:
input: seriatim
@@ -224,7 +311,7 @@ pipelines:
}
func TestFileConfigStageLocalValidatorsPreserveOrderAndFields(t *testing.T) {
cfg := applyFileConfig(t, `version: 3
cfg := applyFileConfig(t, `version: 4
pipelines:
main:
input: seriatim
@@ -277,8 +364,8 @@ pipelines:
}
func TestFileConfigStateSectionsApplyIndependently(t *testing.T) {
cfg := applyFileConfig(t, `version: 3
scriptorium:
cfg := applyFileConfig(t, `version: 4
promptkit:
profile_dir: ./profiles
concurrency:
total_llm: 7
@@ -294,15 +381,15 @@ cache:
debug:
directory: ./debug
`)
if cfg.Scriptorium.ProfileDir != "./profiles" || cfg.Scriptorium.ProfileFile != "" {
t.Fatalf("scriptorium = %#v", cfg.Scriptorium)
if cfg.PromptKit.ProfileDir != "./profiles" || cfg.PromptKit.ProfileFile != "" {
t.Fatalf("promptkit = %#v", cfg.PromptKit)
}
if cfg.Concurrency.TotalLLM != 7 || cfg.Concurrency.StageWorkers["extract"] != 7 {
t.Fatalf("concurrency = %#v", cfg.Concurrency)
}
if cfg.Output.Directory != "./output" || cfg.Cache.ChunkPlans.Directory != "plans" || cfg.Cache.ChunkPlans.Mode != pipeline.ChunkCacheBypass ||
!cfg.Cache.Checkpoints.Enabled || cfg.Cache.Checkpoints.Directory != "checkpoints" || cfg.Debug.Directory != "./debug" {
t.Fatalf("state sections = %#v, %#v, %#v, %#v", cfg.Output, cfg.Cache, cfg.Debug, cfg.Scriptorium)
t.Fatalf("state sections = %#v, %#v, %#v, %#v", cfg.Output, cfg.Cache, cfg.Debug, cfg.PromptKit)
}
if cfg.Output.Directory == cfg.Cache.ChunkPlans.Directory || cfg.Cache.ChunkPlans.Directory == cfg.Cache.Checkpoints.Directory || cfg.Cache.Checkpoints.Directory == cfg.Debug.Directory {
t.Fatal("state roots were coupled")
@@ -310,11 +397,11 @@ debug:
}
func TestFileConfigCheckpointEnabledCanBeExplicitlyDisabled(t *testing.T) {
cfg := applyFileConfig(t, "version: 3\ncache:\n checkpoints:\n enabled: true\n")
cfg := applyFileConfig(t, "version: 4\ncache:\n checkpoints:\n enabled: true\n")
if !cfg.Cache.Checkpoints.Enabled || !cloneConfig(cfg).Cache.Checkpoints.Enabled {
t.Fatalf("enabled checkpoint config was not retained: %#v", cfg.Cache.Checkpoints)
}
file := parseFileConfig(t, "version: 3\ncache:\n checkpoints:\n enabled: false\n")
file := parseFileConfig(t, "version: 4\ncache:\n checkpoints:\n enabled: false\n")
if err := cfg.ApplyFileConfig(file); err != nil {
t.Fatal(err)
}
@@ -331,17 +418,17 @@ func TestFileConfigRejectsTrimmedKeyCollisions(t *testing.T) {
}{
{
name: "pipeline ids",
yaml: "version: 3\npipelines:\n main: {}\n ' main ': {}\n",
yaml: "version: 4\npipelines:\n main: {}\n ' main ': {}\n",
want: "pipeline id \"main\" is duplicated after trimming",
},
{
name: "lane ids",
yaml: "version: 3\npipelines:\n main:\n artifacts:\n spells: {}\n ' spells ': {}\n",
yaml: "version: 4\npipelines:\n main:\n artifacts:\n spells: {}\n ' spells ': {}\n",
want: "artifact lane id \"spells\" is duplicated after trimming",
},
{
name: "reference slots",
yaml: "version: 3\npipelines:\n main:\n references:\n slot: ./one.txt\n ' slot ': ./two.txt\n",
yaml: "version: 4\npipelines:\n main:\n references:\n slot: ./one.txt\n ' slot ': ./two.txt\n",
want: "reference slot \"slot\" is duplicated after trimming",
},
}
@@ -358,7 +445,7 @@ func TestFileConfigRejectsTrimmedKeyCollisions(t *testing.T) {
}
func TestFileConfigParsesOrderedStepsAndReferenceSources(t *testing.T) {
file := parseFileConfig(t, `version: 3
file := parseFileConfig(t, `version: 4
pipelines:
session:
input: seriatim
@@ -397,7 +484,7 @@ func TestFileConfigRejectsAmbiguousReferenceSourceForms(t *testing.T) {
"artifact: {step: 1, lane: b}",
"1",
} {
_, err := ParseFileConfigYAML([]byte("version: 3\npipelines:\n p:\n input: text\n references:\n slot: " + source + "\n"))
_, err := ParseFileConfigYAML([]byte("version: 4\npipelines:\n p:\n input: text\n references:\n slot: " + source + "\n"))
if err == nil {
t.Fatalf("ParseFileConfigYAML(%q) error = nil", source)
}
@@ -412,12 +499,12 @@ func TestFileConfigRejectsEmptyAndAmbiguousPipelineShapes(t *testing.T) {
}{
{
name: "empty steps",
yaml: "version: 3\npipelines:\n p:\n input: text\n steps: []\n",
yaml: "version: 4\npipelines:\n p:\n input: text\n steps: []\n",
want: "at least one ordered step",
},
{
name: "both forms",
yaml: "version: 3\npipelines:\n p:\n input: text\n artifacts: {}\n steps: []\n",
yaml: "version: 4\npipelines:\n p:\n input: text\n artifacts: {}\n steps: []\n",
want: "both artifacts and steps",
},
}

View File

@@ -10,7 +10,7 @@ import (
func (c Config) Validate() error {
c.Concurrency.recomputeStageWorkerDefaults()
if err := validateScriptorium(c.Scriptorium); err != nil {
if err := validatePromptKit(c.PromptKit); err != nil {
return err
}
if err := validateStateSurfaces(c); err != nil {
@@ -49,9 +49,9 @@ func validateStageWorkers(cfg ConcurrencyConfig) error {
return nil
}
func validateScriptorium(cfg ScriptoriumConfig) error {
func validatePromptKit(cfg PromptKitConfig) error {
if strings.TrimSpace(cfg.ProfileDir) != "" && strings.TrimSpace(cfg.ProfileFile) != "" {
return fmt.Errorf("scriptorium profile_dir and profile_file are mutually exclusive")
return fmt.Errorf("promptkit profile_dir and profile_file are mutually exclusive")
}
return nil
}

View File

@@ -87,10 +87,10 @@ func TestValidateConcurrencyRules(t *testing.T) {
}
}
func TestValidateScriptoriumSourcesAreMutuallyExclusive(t *testing.T) {
func TestValidatePromptKitSourcesAreMutuallyExclusive(t *testing.T) {
cfg := Default()
cfg.Scriptorium = ScriptoriumConfig{ProfileDir: "./profiles", ProfileFile: "./profile.yml"}
assertValidationContains(t, cfg, "scriptorium profile_dir and profile_file are mutually exclusive")
cfg.PromptKit = PromptKitConfig{ProfileDir: "./profiles", ProfileFile: "./profile.yml"}
assertValidationContains(t, cfg, "promptkit profile_dir and profile_file are mutually exclusive")
}
func TestValidateStateSurfaceRules(t *testing.T) {

View File

@@ -12,7 +12,7 @@ import (
"strings"
"time"
"gitea.maximumdirect.net/eric/scriptorium"
"gitea.maximumdirect.net/eric/promptkit"
)
type AssetSource struct {
@@ -72,7 +72,7 @@ func (r *AssetRegistry) SchemaFS() (fs.FS, error) {
return flattenAssetSources(r.schemas)
}
func (r *AssetRegistry) ScriptoriumOptions() ([]scriptorium.Option, error) {
func (r *AssetRegistry) PromptKitOptions() ([]promptkit.Option, error) {
promptFS, err := r.PromptFS()
if err != nil {
return nil, fmt.Errorf("prepare prompt assets: %w", err)
@@ -81,9 +81,9 @@ func (r *AssetRegistry) ScriptoriumOptions() ([]scriptorium.Option, error) {
if err != nil {
return nil, fmt.Errorf("prepare schema assets: %w", err)
}
return []scriptorium.Option{
scriptorium.WithPromptFS(promptFS, "."),
scriptorium.WithSchemaFS(schemaFS, "."),
return []promptkit.Option{
promptkit.WithPromptFS(promptFS, "."),
promptkit.WithSchemaFS(schemaFS, "."),
}, nil
}

View File

@@ -7,7 +7,7 @@ import (
"testing/fstest"
"time"
"gitea.maximumdirect.net/eric/scriptorium"
"gitea.maximumdirect.net/eric/promptkit"
)
func TestAssetRegistryCombinesPromptAndSchemaSources(t *testing.T) {
@@ -22,12 +22,12 @@ func TestAssetRegistryCombinesPromptAndSchemaSources(t *testing.T) {
"root/schemas/out.json": {Data: []byte(`{"type":"object"}`)},
}, "root")
engine := newAssetTestEngine(t, registry)
prepared, err := engine.Prepare(context.Background(), scriptorium.RunRequest{
engine := newPromptKitAssetTestEngine(t, registry)
prepared, err := engine.Prepare(context.Background(), promptkit.RunRequest{
PromptID: "asset.test",
ProfileID: "asset-test-profile",
Inputs: map[string]scriptorium.ArtifactRef{
"transcript": scriptorium.Inline(`{"ok":true}`),
Inputs: map[string]promptkit.ArtifactRef{
"transcript": promptkit.Inline(`{"ok":true}`),
},
})
if err != nil {
@@ -50,12 +50,12 @@ func TestAssetRegistryPrepareFailsForMissingPromptAsset(t *testing.T) {
"out.json": {Data: []byte(`{"type":"object"}`)},
}, ".")
engine := newAssetTestEngine(t, registry)
_, err := engine.Prepare(context.Background(), scriptorium.RunRequest{
engine := newPromptKitAssetTestEngine(t, registry)
_, err := engine.Prepare(context.Background(), promptkit.RunRequest{
PromptID: "asset.test",
ProfileID: "asset-test-profile",
Inputs: map[string]scriptorium.ArtifactRef{
"transcript": scriptorium.Inline(`{"ok":true}`),
Inputs: map[string]promptkit.ArtifactRef{
"transcript": promptkit.Inline(`{"ok":true}`),
},
})
if err == nil || !strings.Contains(err.Error(), "content_file") {
@@ -74,12 +74,12 @@ func TestAssetRegistryPrepareFailsForMissingSchemaAsset(t *testing.T) {
"present.json": {Data: []byte(`{"type":"object"}`)},
}, ".")
engine := newAssetTestEngine(t, registry)
_, err := engine.Prepare(context.Background(), scriptorium.RunRequest{
engine := newPromptKitAssetTestEngine(t, registry)
_, err := engine.Prepare(context.Background(), promptkit.RunRequest{
PromptID: "asset.test",
ProfileID: "asset-test-profile",
Inputs: map[string]scriptorium.ArtifactRef{
"transcript": scriptorium.Inline(`{"ok":true}`),
Inputs: map[string]promptkit.ArtifactRef{
"transcript": promptkit.Inline(`{"ok":true}`),
},
})
if err == nil || !strings.Contains(err.Error(), "missing.json") {
@@ -145,18 +145,18 @@ func TestHashAssetsOmitsRawAssetContent(t *testing.T) {
}
}
func newAssetTestEngine(t *testing.T, registry *AssetRegistry) *scriptorium.Engine {
func newPromptKitAssetTestEngine(t *testing.T, registry *AssetRegistry) *promptkit.Engine {
t.Helper()
options, err := registry.ScriptoriumOptions()
options, err := registry.PromptKitOptions()
if err != nil {
t.Fatalf("ScriptoriumOptions() error = %v, want nil", err)
t.Fatalf("PromptKitOptions() error = %v, want nil", err)
}
options = append(options, scriptorium.WithProfiles(scriptorium.OpenAICompatibleProfile(scriptorium.OpenAICompatibleProfileConfig{
options = append(options, promptkit.WithProfiles(promptkit.OpenAICompatibleProfile(promptkit.OpenAICompatibleProfileConfig{
ID: "asset-test-profile",
Endpoint: "http://127.0.0.1:1/v1",
Model: "asset-test-model",
})))
engine, err := scriptorium.NewEngine(scriptorium.Config{Timeout: time.Second}, options...)
engine, err := promptkit.NewEngine(promptkit.Config{Timeout: time.Second}, options...)
if err != nil {
t.Fatalf("NewEngine() error = %v, want nil", err)
}

View File

@@ -13,23 +13,23 @@ import (
"gitea.maximumdirect.net/eric/notarius/internal/core/artifacts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/scriptorium"
"gitea.maximumdirect.net/eric/promptkit"
)
const scriptoriumProviderName = "scriptorium"
const promptKitProviderName = "promptkit"
type ScriptoriumClientConfig struct {
type PromptKitClientConfig struct {
ProfileDir string
ProfileFile string
Assets *AssetRegistry
Timeout time.Duration
HTTPClient *http.Client
EngineOptions []scriptorium.Option
EngineOptions []promptkit.Option
Recorder *LLMProfileRecorder
}
type ScriptoriumClient struct {
engine *scriptorium.Engine
type PromptKitClient struct {
engine *promptkit.Engine
recorder *LLMProfileRecorder
}
@@ -38,49 +38,49 @@ type LLMProfileRecorder struct {
profiles map[string]artifacts.LLMProfileManifest
}
var _ contracts.StructuredLLMClient = (*ScriptoriumClient)(nil)
var _ contracts.LLMProfileManifestProvider = (*ScriptoriumClient)(nil)
var _ contracts.StructuredLLMClient = (*PromptKitClient)(nil)
var _ contracts.LLMProfileManifestProvider = (*PromptKitClient)(nil)
func NewScriptoriumClient(cfg ScriptoriumClientConfig) (*ScriptoriumClient, error) {
func NewPromptKitClient(cfg PromptKitClientConfig) (*PromptKitClient, error) {
if cfg.Assets == nil {
return nil, fmt.Errorf("scriptorium client assets must not be nil")
return nil, fmt.Errorf("PromptKit client assets must not be nil")
}
if strings.TrimSpace(cfg.ProfileDir) != "" && strings.TrimSpace(cfg.ProfileFile) != "" {
return nil, fmt.Errorf("scriptorium profile_dir and profile_file are mutually exclusive")
return nil, fmt.Errorf("PromptKit profile_dir and profile_file are mutually exclusive")
}
options, err := cfg.Assets.ScriptoriumOptions()
options, err := cfg.Assets.PromptKitOptions()
if err != nil {
return nil, err
}
if profileFile := strings.TrimSpace(cfg.ProfileFile); profileFile != "" {
options = append(options, scriptorium.WithProfileFile(profileFile))
options = append(options, promptkit.WithProfileFile(profileFile))
}
options = append(options, cfg.EngineOptions...)
engine, err := scriptorium.NewEngine(scriptorium.Config{
engine, err := promptkit.NewEngine(promptkit.Config{
ProfileDir: strings.TrimSpace(cfg.ProfileDir),
Timeout: cfg.Timeout,
HTTPClient: cfg.HTTPClient,
}, options...)
if err != nil {
return nil, fmt.Errorf("create Scriptorium engine: %w", err)
return nil, fmt.Errorf("create PromptKit engine: %w", err)
}
recorder := cfg.Recorder
if recorder == nil {
recorder = NewLLMProfileRecorder()
}
return &ScriptoriumClient{
return &PromptKitClient{
engine: engine,
recorder: recorder,
}, nil
}
func (c *ScriptoriumClient) CompleteStructured(ctx context.Context, req contracts.StructuredCompletionRequest, out any) (contracts.StructuredCompletionResponse, error) {
func (c *PromptKitClient) CompleteStructured(ctx context.Context, req contracts.StructuredCompletionRequest, out any) (contracts.StructuredCompletionResponse, error) {
if c == nil {
return contracts.StructuredCompletionResponse{}, fmt.Errorf("scriptorium client must not be nil")
return contracts.StructuredCompletionResponse{}, fmt.Errorf("PromptKit client must not be nil")
}
if c.engine == nil {
return contracts.StructuredCompletionResponse{}, fmt.Errorf("scriptorium client engine must not be nil")
return contracts.StructuredCompletionResponse{}, fmt.Errorf("PromptKit client engine must not be nil")
}
if err := validateOutputTarget(out); err != nil {
return contracts.StructuredCompletionResponse{}, err
@@ -90,52 +90,52 @@ func (c *ScriptoriumClient) CompleteStructured(ctx context.Context, req contract
return contracts.StructuredCompletionResponse{}, fmt.Errorf("structured completion prompt_id must not be empty")
}
runReq := scriptorium.RunRequest{
runReq := promptkit.RunRequest{
PromptID: promptID,
PromptVersion: strings.TrimSpace(req.PromptVersion),
ProfileID: strings.TrimSpace(req.ProfileID),
Inputs: scriptoriumInputs(req.Inputs),
Vars: scriptoriumVars(req),
Metadata: scriptoriumMetadata(req),
Inputs: promptKitInputs(req.Inputs),
Vars: promptKitVars(req),
Metadata: promptKitMetadata(req),
}
prepared, err := c.engine.Prepare(ctx, runReq)
if err != nil {
if ctxErr := ctx.Err(); ctxErr != nil {
return contracts.StructuredCompletionResponse{}, ctxErr
}
return contracts.StructuredCompletionResponse{}, fmt.Errorf("prepare Scriptorium prompt %q: %w", promptID, redactScriptoriumError(err))
return contracts.StructuredCompletionResponse{}, fmt.Errorf("prepare PromptKit prompt %q: %w", promptID, redactPromptKitError(err))
}
result, err := c.engine.Run(ctx, runReq)
if err != nil {
if ctxErr := ctx.Err(); ctxErr != nil {
return contracts.StructuredCompletionResponse{}, ctxErr
}
return contracts.StructuredCompletionResponse{}, fmt.Errorf("run Scriptorium prompt %q: %w", promptID, redactScriptoriumError(err))
return contracts.StructuredCompletionResponse{}, fmt.Errorf("run PromptKit prompt %q: %w", promptID, redactPromptKitError(err))
}
if result == nil {
return contracts.StructuredCompletionResponse{}, fmt.Errorf("run Scriptorium prompt %q: %w: empty result", promptID, contracts.ErrInvalidStructuredOutput)
return contracts.StructuredCompletionResponse{}, fmt.Errorf("run PromptKit prompt %q: %w: empty result", promptID, contracts.ErrInvalidStructuredOutput)
}
response := c.responseFromResult(result, prepared)
if result.Validation.Status == scriptorium.ValidationFailed || !result.Validation.IsValid {
return response, fmt.Errorf("run Scriptorium prompt %q: %w: validation failed: %s", promptID, contracts.ErrInvalidStructuredOutput, strings.Join(result.Validation.Errors, "; "))
if result.Validation.Status == promptkit.ValidationFailed || !result.Validation.IsValid {
return response, fmt.Errorf("run PromptKit prompt %q: %w: validation failed: %s", promptID, contracts.ErrInvalidStructuredOutput, strings.Join(result.Validation.Errors, "; "))
}
if len(strings.TrimSpace(string(response.Content))) == 0 {
return response, fmt.Errorf("run Scriptorium prompt %q: %w: empty structured output", promptID, contracts.ErrInvalidStructuredOutput)
return response, fmt.Errorf("run PromptKit prompt %q: %w: empty structured output", promptID, contracts.ErrInvalidStructuredOutput)
}
if err := json.Unmarshal(response.Content, out); err != nil {
return response, fmt.Errorf("decode Scriptorium structured output for prompt %q: %w: %w", promptID, contracts.ErrInvalidStructuredOutput, err)
return response, fmt.Errorf("decode PromptKit structured output for prompt %q: %w: %w", promptID, contracts.ErrInvalidStructuredOutput, err)
}
return response, nil
}
func (c *ScriptoriumClient) responseFromResult(result *scriptorium.RunResult, prepared *scriptorium.PreparedRun) contracts.StructuredCompletionResponse {
func (c *PromptKitClient) responseFromResult(result *promptkit.RunResult, prepared *promptkit.PreparedRun) contracts.StructuredCompletionResponse {
content := result.Artifact.Body
if len(content) == 0 {
content = []byte(result.RawOutput)
}
profile := artifacts.LLMProfileManifest{
ID: strings.TrimSpace(result.SelectedProfileID),
Provider: scriptoriumProviderName,
Provider: promptKitProviderName,
Model: firstNonEmpty(result.ModelName, result.EffectiveModelParams.Model),
}
if c.recorder != nil {
@@ -149,17 +149,17 @@ func (c *ScriptoriumClient) responseFromResult(result *scriptorium.RunResult, pr
PromptTokens: result.Usage.PromptTokens,
CompletionTokens: result.Usage.CompletionTokens,
TotalTokens: result.Usage.TotalTokens,
Debug: scriptoriumDebugMaterial(prepared, result),
Debug: promptKitDebugMaterial(prepared, result),
}
}
func scriptoriumDebugMaterial(prepared *scriptorium.PreparedRun, result *scriptorium.RunResult) *contracts.LLMDebugMaterial {
func promptKitDebugMaterial(prepared *promptkit.PreparedRun, result *promptkit.RunResult) *contracts.LLMDebugMaterial {
material := &contracts.LLMDebugMaterial{}
if prepared != nil {
material.Prompt = scriptoriumDebugPrompt(prepared)
material.Prompt = promptKitDebugPrompt(prepared)
}
if result != nil {
material.Response = scriptoriumDebugResponse(result)
material.Response = promptKitDebugResponse(result)
}
if material.Prompt == nil && material.Response == nil {
return nil
@@ -167,7 +167,7 @@ func scriptoriumDebugMaterial(prepared *scriptorium.PreparedRun, result *scripto
return material
}
func scriptoriumDebugPrompt(prepared *scriptorium.PreparedRun) *contracts.LLMDebugPrompt {
func promptKitDebugPrompt(prepared *promptkit.PreparedRun) *contracts.LLMDebugPrompt {
if prepared == nil {
return nil
}
@@ -194,7 +194,7 @@ func scriptoriumDebugPrompt(prepared *scriptorium.PreparedRun) *contracts.LLMDeb
}
}
func scriptoriumDebugResponse(result *scriptorium.RunResult) *contracts.LLMDebugResponse {
func promptKitDebugResponse(result *promptkit.RunResult) *contracts.LLMDebugResponse {
if result == nil {
return nil
}
@@ -254,7 +254,7 @@ func cloneStringMap(values map[string]string) map[string]string {
return out
}
func (c *ScriptoriumClient) LLMProfileManifests() []artifacts.LLMProfileManifest {
func (c *PromptKitClient) LLMProfileManifests() []artifacts.LLMProfileManifest {
if c == nil || c.recorder == nil {
return nil
}
@@ -302,11 +302,11 @@ func (r *LLMProfileRecorder) Manifests() []artifacts.LLMProfileManifest {
return out
}
func scriptoriumInputs(inputs contracts.LLMInputSet) map[string]scriptorium.ArtifactRef {
func promptKitInputs(inputs contracts.LLMInputSet) map[string]promptkit.ArtifactRef {
if len(inputs) == 0 {
return nil
}
out := make(map[string]scriptorium.ArtifactRef, len(inputs))
out := make(map[string]promptkit.ArtifactRef, len(inputs))
for key, material := range inputs {
name := strings.TrimSpace(key)
if name == "" {
@@ -320,15 +320,15 @@ func scriptoriumInputs(inputs contracts.LLMInputSet) map[string]scriptorium.Arti
body = " "
}
if origin := strings.TrimSpace(material.OriginURI); origin != "" {
out[name] = scriptorium.InlineWithURI(origin, body)
out[name] = promptkit.InlineWithURI(origin, body)
} else {
out[name] = scriptorium.Inline(body)
out[name] = promptkit.Inline(body)
}
}
return out
}
func scriptoriumVars(req contracts.StructuredCompletionRequest) map[string]string {
func promptKitVars(req contracts.StructuredCompletionRequest) map[string]string {
vars := make(map[string]string, len(req.Vars)+1)
for key, value := range req.Vars {
name := strings.TrimSpace(key)
@@ -346,7 +346,7 @@ func scriptoriumVars(req contracts.StructuredCompletionRequest) map[string]strin
return vars
}
func scriptoriumMetadata(req contracts.StructuredCompletionRequest) map[string]string {
func promptKitMetadata(req contracts.StructuredCompletionRequest) map[string]string {
metadata := map[string]string{}
if stageName := strings.TrimSpace(req.StageName); stageName != "" {
metadata["stage_name"] = stageName
@@ -359,7 +359,7 @@ func scriptoriumMetadata(req contracts.StructuredCompletionRequest) map[string]s
var bearerTokenPattern = regexp.MustCompile(`(?i)Bearer\s+[A-Za-z0-9._~+/=-]+`)
func redactScriptoriumError(err error) error {
func redactPromptKitError(err error) error {
if err == nil {
return nil
}

View File

@@ -4,6 +4,8 @@ import (
"context"
"encoding/json"
"errors"
"io"
"net/http"
"strings"
"sync"
"sync/atomic"
@@ -12,12 +14,12 @@ import (
"time"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/scriptorium"
"gitea.maximumdirect.net/eric/promptkit"
)
func TestScriptoriumClientMapsPromptRequestAndUnmarshalsOutput(t *testing.T) {
fake := &fakeScriptoriumLLM{content: `{"ok":true}`}
client := newTestScriptoriumClient(t, fake)
func TestPromptKitClientMapsPromptRequestAndUnmarshalsOutput(t *testing.T) {
fake := &fakePromptKitLLM{content: `{"ok":true}`}
client := newTestPromptKitClient(t, fake)
var out struct {
OK bool `json:"ok"`
@@ -39,7 +41,7 @@ func TestScriptoriumClientMapsPromptRequestAndUnmarshalsOutput(t *testing.T) {
if !out.OK {
t.Fatalf("decoded output OK = false, want true")
}
if resp.Provider != scriptoriumProviderName || resp.Model != "explicit-model" || resp.ProfileID != "explicit-profile" {
if resp.Provider != "promptkit" || resp.Model != "explicit-model" || resp.ProfileID != "explicit-profile" {
t.Fatalf("response metadata = %#v", resp)
}
if resp.PromptTokens != 11 || resp.CompletionTokens != 7 || resp.TotalTokens != 18 {
@@ -57,6 +59,9 @@ func TestScriptoriumClientMapsPromptRequestAndUnmarshalsOutput(t *testing.T) {
if resp.Debug.Response == nil || resp.Debug.Response.Content != `{"ok":true}` {
t.Fatalf("debug response = %#v, want raw response content", resp.Debug.Response)
}
if resp.Debug.Response.Usage.CachedTokens != 5 || resp.Debug.Response.Usage.CacheWriteTokens != 3 {
t.Fatalf("debug usage = %#v, want cached token counts", resp.Debug.Response.Usage)
}
debugJSON, err := json.Marshal(resp.Debug)
if err != nil {
t.Fatalf("marshal debug material: %v", err)
@@ -71,21 +76,52 @@ func TestScriptoriumClientMapsPromptRequestAndUnmarshalsOutput(t *testing.T) {
if gotReq.Target.Model != "explicit-model" {
t.Fatalf("model = %q, want explicit-model", gotReq.Target.Model)
}
if len(gotReq.Prompt.Messages) != 1 || !strings.Contains(gotReq.Prompt.Messages[0].Content, `{"source":true}`) {
t.Fatalf("rendered messages = %#v, want transcript input content", gotReq.Prompt.Messages)
if len(gotReq.Prompt.Messages) != 1 ||
!strings.Contains(gotReq.Prompt.Messages[0].Content, `{"source":true}`) ||
!strings.Contains(gotReq.Prompt.Messages[0].Content, "value") {
t.Fatalf("rendered messages = %#v, want transcript input and variable content", gotReq.Prompt.Messages)
}
if gotReq.StructuredOutput == nil {
t.Fatalf("structured output = nil, want JSON schema")
}
manifests := client.LLMProfileManifests()
if len(manifests) != 1 || manifests[0].ID != "explicit-profile" || manifests[0].Model != "explicit-model" {
if len(manifests) != 1 ||
manifests[0].ID != "explicit-profile" ||
manifests[0].Provider != "promptkit" ||
manifests[0].Model != "explicit-model" {
t.Fatalf("profile manifests = %#v", manifests)
}
}
func TestScriptoriumClientUsesPromptDefaultProfileWhenRequestProfileEmpty(t *testing.T) {
fake := &fakeScriptoriumLLM{content: `{"ok":true}`}
client := newTestScriptoriumClient(t, fake)
func TestNewPromptKitClientReportsAssetAndEngineConstructionFailures(t *testing.T) {
t.Run("assets", func(t *testing.T) {
registry := NewAssetRegistry()
for _, root := range []string{"one", "two"} {
if err := registry.RegisterPromptFS(fstest.MapFS{
root + "/prompt.yaml": {Data: []byte("id: duplicate")},
}, root); err != nil {
t.Fatalf("RegisterPromptFS() error = %v", err)
}
}
if _, err := NewPromptKitClient(PromptKitClientConfig{Assets: registry}); err == nil ||
!strings.Contains(err.Error(), "duplicate asset path") {
t.Fatalf("NewPromptKitClient() error = %v, want asset construction failure", err)
}
})
t.Run("engine", func(t *testing.T) {
if _, err := NewPromptKitClient(PromptKitClientConfig{
Assets: newTestPromptKitAssets(t),
EngineOptions: []promptkit.Option{promptkit.WithProfileFile("")},
}); err == nil || !strings.Contains(err.Error(), "create PromptKit engine") {
t.Fatalf("NewPromptKitClient() error = %v, want engine construction failure", err)
}
})
}
func TestPromptKitClientUsesPromptDefaultProfileWhenRequestProfileEmpty(t *testing.T) {
fake := &fakePromptKitLLM{content: `{"ok":true}`}
client := newTestPromptKitClient(t, fake)
var out map[string]any
if _, err := client.CompleteStructured(context.Background(), contracts.StructuredCompletionRequest{
@@ -102,8 +138,8 @@ func TestScriptoriumClientUsesPromptDefaultProfileWhenRequestProfileEmpty(t *tes
}
}
func TestScriptoriumClientValidationFailureReturnsError(t *testing.T) {
client := newTestScriptoriumClient(t, &fakeScriptoriumLLM{content: `{"bad":true}`})
func TestPromptKitClientValidationFailureReturnsError(t *testing.T) {
client := newTestPromptKitClient(t, &fakePromptKitLLM{content: `{"bad":true}`})
var out map[string]any
resp, err := client.CompleteStructured(context.Background(), contracts.StructuredCompletionRequest{
@@ -127,8 +163,8 @@ func TestScriptoriumClientValidationFailureReturnsError(t *testing.T) {
}
}
func TestScriptoriumClientDecodeFailureReturnsRawResponse(t *testing.T) {
client := newTestScriptoriumClient(t, &fakeScriptoriumLLM{content: `{"ok":true}`})
func TestPromptKitClientDecodeFailureReturnsRawResponse(t *testing.T) {
client := newTestPromptKitClient(t, &fakePromptKitLLM{content: `{"ok":true}`})
var out []any
resp, err := client.CompleteStructured(context.Background(), contracts.StructuredCompletionRequest{
@@ -138,7 +174,7 @@ func TestScriptoriumClientDecodeFailureReturnsRawResponse(t *testing.T) {
"transcript": contracts.NewLLMInputMaterial("transcript", "application/json", []byte(`{"source":true}`), "", ""),
},
}, &out)
if err == nil || !errors.Is(err, contracts.ErrInvalidStructuredOutput) || !strings.Contains(err.Error(), "decode Scriptorium structured output") {
if err == nil || !errors.Is(err, contracts.ErrInvalidStructuredOutput) || !strings.Contains(err.Error(), "decode PromptKit structured output") {
t.Fatalf("CompleteStructured() error = %v, want decode failure", err)
}
if got := string(resp.Content); got != `{"ok":true}` {
@@ -149,8 +185,8 @@ func TestScriptoriumClientDecodeFailureReturnsRawResponse(t *testing.T) {
}
}
func TestScriptoriumClientProviderFailureIncludesContextAndRedactsBearerToken(t *testing.T) {
client := newTestScriptoriumClient(t, &fakeScriptoriumLLM{err: errors.New("provider failed with Bearer secret-token")})
func TestPromptKitClientProviderFailureIncludesContextAndRedactsBearerToken(t *testing.T) {
client := newTestPromptKitClient(t, &fakePromptKitLLM{err: errors.New("provider failed with Bearer secret-token")})
var out map[string]any
resp, err := client.CompleteStructured(context.Background(), contracts.StructuredCompletionRequest{
@@ -166,7 +202,7 @@ func TestScriptoriumClientProviderFailureIncludesContextAndRedactsBearerToken(t
if errors.Is(err, contracts.ErrInvalidStructuredOutput) {
t.Fatalf("provider error = %v, must not be classified as invalid structured output", err)
}
if !strings.Contains(err.Error(), `run Scriptorium prompt "adapter.test"`) {
if !strings.Contains(err.Error(), `run PromptKit prompt "adapter.test"`) {
t.Fatalf("error = %q, want operation context", err.Error())
}
if strings.Contains(err.Error(), "secret-token") || !strings.Contains(err.Error(), "Bearer [REDACTED]") {
@@ -177,10 +213,10 @@ func TestScriptoriumClientProviderFailureIncludesContextAndRedactsBearerToken(t
}
}
func TestScriptoriumClientContextCancellationIsRespected(t *testing.T) {
func TestPromptKitClientContextCancellationIsRespected(t *testing.T) {
ctx, cancel := context.WithCancel(context.Background())
cancel()
client := newTestScriptoriumClient(t, &fakeScriptoriumLLM{content: `{"ok":true}`})
client := newTestPromptKitClient(t, &fakePromptKitLLM{content: `{"ok":true}`})
var out map[string]any
_, err := client.CompleteStructured(ctx, contracts.StructuredCompletionRequest{
@@ -194,8 +230,57 @@ func TestScriptoriumClientContextCancellationIsRespected(t *testing.T) {
}
}
func TestScriptoriumClientClassifiesEmptyStructuredCompletion(t *testing.T) {
client := newTestScriptoriumClient(t, &fakeScriptoriumLLM{allowEmpty: true})
func TestPromptKitClientForwardsConfiguredTransportTimeout(t *testing.T) {
var remaining time.Duration
transport := roundTripFunc(func(req *http.Request) (*http.Response, error) {
deadline, ok := req.Context().Deadline()
if !ok {
t.Fatal("outbound request context has no deadline")
}
remaining = time.Until(deadline)
return &http.Response{
StatusCode: http.StatusOK,
Status: "200 OK",
Header: make(http.Header),
Body: io.NopCloser(strings.NewReader(
`{"choices":[{"message":{"content":"{\"ok\":true}"}}]}`,
)),
Request: req,
}, nil
})
const configuredTimeout = 2 * time.Second
client, err := NewPromptKitClient(PromptKitClientConfig{
Assets: newTestPromptKitAssets(t),
Timeout: configuredTimeout,
HTTPClient: &http.Client{Transport: transport},
EngineOptions: []promptkit.Option{
promptkit.WithProfiles(promptkit.OpenAICompatibleProfile(promptkit.OpenAICompatibleProfileConfig{
ID: "default-profile",
Endpoint: "http://promptkit.test/v1",
Model: "default-model",
})),
},
})
if err != nil {
t.Fatalf("NewPromptKitClient() error = %v", err)
}
var out map[string]any
if _, err := client.CompleteStructured(context.Background(), contracts.StructuredCompletionRequest{
PromptID: "adapter.test",
SessionID: "session-123",
Inputs: contracts.LLMInputSet{
"transcript": contracts.NewLLMInputMaterial("transcript", "application/json", []byte(`{"source":true}`), "", ""),
},
}, &out); err != nil {
t.Fatalf("CompleteStructured() error = %v", err)
}
if remaining < configuredTimeout-time.Second || remaining > configuredTimeout {
t.Fatalf("transport deadline remaining = %v, want near %v", remaining, configuredTimeout)
}
}
func TestPromptKitClientClassifiesEmptyStructuredCompletion(t *testing.T) {
client := newTestPromptKitClient(t, &fakePromptKitLLM{allowEmpty: true})
var out map[string]any
_, err := client.CompleteStructured(context.Background(), contracts.StructuredCompletionRequest{
@@ -210,12 +295,12 @@ func TestScriptoriumClientClassifiesEmptyStructuredCompletion(t *testing.T) {
}
}
func TestScheduledScriptoriumClientBoundsConcurrentCalls(t *testing.T) {
fake := &fakeScriptoriumLLM{
func TestScheduledPromptKitClientBoundsConcurrentCalls(t *testing.T) {
fake := &fakePromptKitLLM{
content: `{"ok":true}`,
block: make(chan struct{}),
}
client := newTestScriptoriumClient(t, fake)
client := newTestPromptKitClient(t, fake)
scheduler, err := NewScheduler(1)
if err != nil {
t.Fatalf("NewScheduler() error = %v, want nil", err)
@@ -249,8 +334,8 @@ func TestScheduledScriptoriumClientBoundsConcurrentCalls(t *testing.T) {
wg.Wait()
}
func TestScriptoriumClientValidatesRequest(t *testing.T) {
client := newTestScriptoriumClient(t, &fakeScriptoriumLLM{content: `{"ok":true}`})
func TestPromptKitClientValidatesRequest(t *testing.T) {
client := newTestPromptKitClient(t, &fakePromptKitLLM{content: `{"ok":true}`})
var out map[string]any
if _, err := client.CompleteStructured(context.Background(), contracts.StructuredCompletionRequest{}, &out); err == nil || !strings.Contains(err.Error(), "prompt_id") {
t.Fatalf("missing prompt id error = %v, want prompt_id validation", err)
@@ -260,7 +345,34 @@ func TestScriptoriumClientValidatesRequest(t *testing.T) {
}
}
func newTestScriptoriumClient(t *testing.T, fake *fakeScriptoriumLLM) *ScriptoriumClient {
func newTestPromptKitClient(t *testing.T, fake *fakePromptKitLLM) *PromptKitClient {
t.Helper()
registry := newTestPromptKitAssets(t)
client, err := NewPromptKitClient(PromptKitClientConfig{
Assets: registry,
EngineOptions: []promptkit.Option{
promptkit.WithProfiles(
promptkit.OpenAICompatibleProfile(promptkit.OpenAICompatibleProfileConfig{
ID: "default-profile",
Endpoint: "http://127.0.0.1:1/v1",
Model: "default-model",
}),
promptkit.OpenAICompatibleProfile(promptkit.OpenAICompatibleProfileConfig{
ID: "explicit-profile",
Endpoint: "http://127.0.0.1:1/v1",
Model: "explicit-model",
}),
),
promptkit.WithLLMClient(fake),
},
})
if err != nil {
t.Fatalf("NewPromptKitClient() error = %v, want nil", err)
}
return client
}
func newTestPromptKitAssets(t *testing.T) *AssetRegistry {
t.Helper()
registry := NewAssetRegistry()
if err := registry.RegisterPromptFS(fstest.MapFS{
@@ -274,7 +386,7 @@ inputs:
content_type: application/json
messages:
- role: user
content: "Transcript: {{ input \"transcript\" }}"
content: "Transcript: {{ input \"transcript\" }} Custom: {{ index . \"custom\" }}"
output:
format: json
validation_mode: json_schema
@@ -289,43 +401,22 @@ output:
}, "."); err != nil {
t.Fatalf("RegisterSchemaFS() error = %v", err)
}
client, err := NewScriptoriumClient(ScriptoriumClientConfig{
Assets: registry,
EngineOptions: []scriptorium.Option{
scriptorium.WithProfiles(
scriptorium.OpenAICompatibleProfile(scriptorium.OpenAICompatibleProfileConfig{
ID: "default-profile",
Endpoint: "http://127.0.0.1:1/v1",
Model: "default-model",
}),
scriptorium.OpenAICompatibleProfile(scriptorium.OpenAICompatibleProfileConfig{
ID: "explicit-profile",
Endpoint: "http://127.0.0.1:1/v1",
Model: "explicit-model",
}),
),
scriptorium.WithLLMClient(fake),
},
})
if err != nil {
t.Fatalf("NewScriptoriumClient() error = %v, want nil", err)
}
return client
return registry
}
type fakeScriptoriumLLM struct {
type fakePromptKitLLM struct {
content string
allowEmpty bool
err error
block chan struct{}
mu sync.Mutex
last scriptorium.GenerateRequest
last promptkit.GenerateRequest
calls int32
inFlight int32
maxInFlight int32
}
func (f *fakeScriptoriumLLM) Generate(ctx context.Context, req scriptorium.GenerateRequest) (*scriptorium.GenerateResponse, error) {
func (f *fakePromptKitLLM) Generate(ctx context.Context, req promptkit.GenerateRequest) (*promptkit.GenerateResponse, error) {
f.mu.Lock()
f.last = req
f.mu.Unlock()
@@ -355,17 +446,25 @@ func (f *fakeScriptoriumLLM) Generate(ctx context.Context, req scriptorium.Gener
if !f.allowEmpty && !json.Valid([]byte(content)) {
return nil, errors.New("test fake must return JSON content")
}
return &scriptorium.GenerateResponse{
return &promptkit.GenerateResponse{
Content: content,
Usage: scriptorium.TokenUsage{
Usage: promptkit.TokenUsage{
PromptTokens: 11,
CompletionTokens: 7,
TotalTokens: 18,
CachedTokens: 5,
CacheWriteTokens: 3,
},
}, nil
}
func (f *fakeScriptoriumLLM) lastRequest() scriptorium.GenerateRequest {
type roundTripFunc func(*http.Request) (*http.Response, error)
func (f roundTripFunc) RoundTrip(req *http.Request) (*http.Response, error) {
return f(req)
}
func (f *fakePromptKitLLM) lastRequest() promptkit.GenerateRequest {
f.mu.Lock()
defer f.mu.Unlock()
return f.last

View File

@@ -12,14 +12,14 @@ import (
)
// ModulePromptFile maps a module-owned embedded prompt file into the
// Scriptorium-visible module prompt directory.
// PromptKit-visible module prompt directory.
type ModulePromptFile struct {
Name string
Path string
}
// SharedPromptFile maps a caller-owned shared prompt file into a module's
// Scriptorium-visible sharedassets prompt subdirectory.
// PromptKit-visible sharedassets prompt subdirectory.
type SharedPromptFile struct {
Name string
FS fs.FS

View File

@@ -56,7 +56,7 @@ func (c *Chunker) ReferenceSlots() []contracts.ReferenceSlot {
}
func (c *Chunker) ManifestMetadata() map[string]any {
promptSHA, err := scriptoriumPromptMetadata()
promptSHA, err := promptAssetMetadata()
if err != nil {
promptSHA = ""
}

View File

@@ -9,7 +9,7 @@ import (
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
)
const scriptoriumPromptRoot = "assets/prompts"
const promptAssetRoot = "assets/prompts"
var promptAssetManifest = shared.PromptAssetManifest{
ModuleDir: "dnd.scenes",
@@ -30,21 +30,21 @@ func RegisterPromptAssets(registry *llm.AssetRegistry) error {
if err != nil {
return fmt.Errorf("prepare scene prompt assets: %w", err)
}
if err := registry.RegisterPromptFS(promptFS, scriptoriumPromptRoot); err != nil {
if err := registry.RegisterPromptFS(promptFS, promptAssetRoot); err != nil {
return err
}
return registry.RegisterSchemaFS(embeddedAssets, "assets/schemas")
}
func scriptoriumPromptMetadata() (string, error) {
scriptoriumPromptHashOnce.Do(func() {
scriptoriumPromptHash, scriptoriumPromptHashErr = promptAssetManifest.Hash(embeddedAssets)
func promptAssetMetadata() (string, error) {
promptAssetHashOnce.Do(func() {
promptAssetHash, promptAssetHashErr = promptAssetManifest.Hash(embeddedAssets)
})
return scriptoriumPromptHash, scriptoriumPromptHashErr
return promptAssetHash, promptAssetHashErr
}
var (
scriptoriumPromptHashOnce sync.Once
scriptoriumPromptHash string
scriptoriumPromptHashErr error
promptAssetHashOnce sync.Once
promptAssetHash string
promptAssetHashErr error
)

View File

@@ -8,10 +8,10 @@ import (
"time"
"gitea.maximumdirect.net/eric/notarius/internal/framework/llm"
"gitea.maximumdirect.net/eric/scriptorium"
"gitea.maximumdirect.net/eric/promptkit"
)
func TestScriptoriumPromptPreparesTranscriptAndTaskMessages(t *testing.T) {
func TestPromptAssetsPrepareTranscriptAndTaskMessages(t *testing.T) {
transcript := []byte(`{"sentinel":"scene-transcript"}`)
prepared := prepareScenesPrompt(t, transcript, "scene-players", "scene-party", "scene-glossary")
@@ -27,7 +27,7 @@ func TestScriptoriumPromptPreparesTranscriptAndTaskMessages(t *testing.T) {
}
}
for _, index := range []int{1, 4} {
if cache := prepared.Messages[index].CacheControl; cache == nil || cache.Type != scriptorium.CacheControlEphemeral {
if cache := prepared.Messages[index].CacheControl; cache == nil || cache.Type != promptkit.CacheControlEphemeral {
t.Errorf("message %d cache control = %#v, want ephemeral", index, cache)
}
}
@@ -52,7 +52,7 @@ func TestScriptoriumPromptPreparesTranscriptAndTaskMessages(t *testing.T) {
}
}
func TestScriptoriumPromptDiagnosticsOmitRawMaterials(t *testing.T) {
func TestPromptAssetDiagnosticsOmitRawMaterials(t *testing.T) {
transcript := []byte(`{"secret":"source text"}`)
prepared := prepareScenesPrompt(t, transcript, "private player note", "private party note", "private glossary note")
metadata := newChunker(t, &fakeScenesLLMClient{}).ManifestMetadata()
@@ -94,22 +94,22 @@ func TestScriptoriumPromptDiagnosticsOmitRawMaterials(t *testing.T) {
}
}
func prepareScenesPrompt(t *testing.T, transcript []byte, players string, party string, glossary string) *scriptorium.PreparedRun {
func prepareScenesPrompt(t *testing.T, transcript []byte, players string, party string, glossary string) *promptkit.PreparedRun {
t.Helper()
registry := llm.NewAssetRegistry()
if err := RegisterPromptAssets(registry); err != nil {
t.Fatalf("register scene prompt assets: %v", err)
}
engine := newScenesScriptoriumEngine(t, registry)
prepared, err := engine.Prepare(context.Background(), scriptorium.RunRequest{
engine := newScenesPromptEngine(t, registry)
prepared, err := engine.Prepare(context.Background(), promptkit.RunRequest{
PromptID: PromptID,
PromptVersion: ResponseSchemaVersion,
ProfileID: "scene-test-profile",
Inputs: map[string]scriptorium.ArtifactRef{
"transcript": scriptorium.InlineWithURI("file:///session.json", string(transcript)),
"players": scriptorium.Inline(players),
"party": scriptorium.Inline(party),
"glossary": scriptorium.Inline(glossary),
Inputs: map[string]promptkit.ArtifactRef{
"transcript": promptkit.InlineWithURI("file:///session.json", string(transcript)),
"players": promptkit.Inline(players),
"party": promptkit.Inline(party),
"glossary": promptkit.Inline(glossary),
},
})
if err != nil {
@@ -118,18 +118,18 @@ func prepareScenesPrompt(t *testing.T, transcript []byte, players string, party
return prepared
}
func newScenesScriptoriumEngine(t *testing.T, registry *llm.AssetRegistry) *scriptorium.Engine {
func newScenesPromptEngine(t *testing.T, registry *llm.AssetRegistry) *promptkit.Engine {
t.Helper()
options, err := registry.ScriptoriumOptions()
options, err := registry.PromptKitOptions()
if err != nil {
t.Fatalf("ScriptoriumOptions() error = %v, want nil", err)
t.Fatalf("PromptKitOptions() error = %v, want nil", err)
}
options = append(options, scriptorium.WithProfiles(scriptorium.OpenAICompatibleProfile(scriptorium.OpenAICompatibleProfileConfig{
options = append(options, promptkit.WithProfiles(promptkit.OpenAICompatibleProfile(promptkit.OpenAICompatibleProfileConfig{
ID: "scene-test-profile",
Endpoint: "http://127.0.0.1:1/v1",
Model: "scene-test-model",
})))
engine, err := scriptorium.NewEngine(scriptorium.Config{Timeout: time.Second}, options...)
engine, err := promptkit.NewEngine(promptkit.Config{Timeout: time.Second}, options...)
if err != nil {
t.Fatalf("NewEngine() error = %v, want nil", err)
}

View File

@@ -96,7 +96,7 @@ func New(llmClient contracts.StructuredLLMClient, _ Options, references ...contr
if err != nil {
return nil, extractorErrorf("prepare scene eligibility: %w", err)
}
promptSHA, err := scriptoriumPromptMetadata()
promptSHA, err := promptAssetMetadata()
if err != nil {
return nil, extractorErrorf("load prompt metadata: %w", err)
}

View File

@@ -9,7 +9,7 @@ import (
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
)
const scriptoriumPromptRoot = "assets/prompts"
const promptAssetRoot = "assets/prompts"
var promptAssetManifest = shared.PromptAssetManifest{
ModuleDir: "dnd.combat_turns",
@@ -33,21 +33,21 @@ func RegisterPromptAssets(registry *llm.AssetRegistry) error {
if err != nil {
return fmt.Errorf("prepare combat-turn prompt assets: %w", err)
}
if err := registry.RegisterPromptFS(promptFS, scriptoriumPromptRoot); err != nil {
if err := registry.RegisterPromptFS(promptFS, promptAssetRoot); err != nil {
return err
}
return registry.RegisterSchemaFS(embeddedAssets, "assets/schemas")
}
func scriptoriumPromptMetadata() (string, error) {
scriptoriumPromptHashOnce.Do(func() {
scriptoriumPromptHash, scriptoriumPromptHashErr = promptAssetManifest.Hash(embeddedAssets)
func promptAssetMetadata() (string, error) {
promptAssetHashOnce.Do(func() {
promptAssetHash, promptAssetHashErr = promptAssetManifest.Hash(embeddedAssets)
})
return scriptoriumPromptHash, scriptoriumPromptHashErr
return promptAssetHash, promptAssetHashErr
}
var (
scriptoriumPromptHashOnce sync.Once
scriptoriumPromptHash string
scriptoriumPromptHashErr error
promptAssetHashOnce sync.Once
promptAssetHash string
promptAssetHashErr error
)

View File

@@ -8,7 +8,7 @@ import (
"time"
"gitea.maximumdirect.net/eric/notarius/internal/framework/llm"
"gitea.maximumdirect.net/eric/scriptorium"
"gitea.maximumdirect.net/eric/promptkit"
)
func TestRegisterPromptAssetsAndPrepareCombatPrompt(t *testing.T) {
@@ -23,37 +23,37 @@ func TestRegisterPromptAssetsAndPrepareCombatPrompt(t *testing.T) {
if _, err := fs.ReadFile(schemaFS, "dnd_combat_turns_llm.v1.json"); err != nil {
t.Fatalf("response schema asset: %v", err)
}
hash, err := scriptoriumPromptMetadata()
hash, err := promptAssetMetadata()
if err != nil || !strings.HasPrefix(hash, "sha256:") {
t.Fatalf("scriptoriumPromptMetadata() = %q, %v; want digest", hash, err)
t.Fatalf("promptAssetMetadata() = %q, %v; want digest", hash, err)
}
}
func TestScriptoriumPromptPreparesRequiredInputs(t *testing.T) {
func TestPromptAssetsPrepareRequiredInputs(t *testing.T) {
registry := llm.NewAssetRegistry()
if err := RegisterPromptAssets(registry); err != nil {
t.Fatalf("RegisterPromptAssets() error = %v, want nil", err)
}
options, err := registry.ScriptoriumOptions()
options, err := registry.PromptKitOptions()
if err != nil {
t.Fatalf("ScriptoriumOptions() error = %v, want nil", err)
t.Fatalf("PromptKitOptions() error = %v, want nil", err)
}
options = append(options, scriptorium.WithProfiles(scriptorium.OpenAICompatibleProfile(scriptorium.OpenAICompatibleProfileConfig{
options = append(options, promptkit.WithProfiles(promptkit.OpenAICompatibleProfile(promptkit.OpenAICompatibleProfileConfig{
ID: "combat-test-profile", Endpoint: "http://127.0.0.1:1/v1", Model: "combat-test-model",
})))
engine, err := scriptorium.NewEngine(scriptorium.Config{Timeout: time.Second}, options...)
engine, err := promptkit.NewEngine(promptkit.Config{Timeout: time.Second}, options...)
if err != nil {
t.Fatalf("NewEngine() error = %v, want nil", err)
}
transcript := `{"units":[1]}`
prepared, err := engine.Prepare(context.Background(), scriptorium.RunRequest{
prepared, err := engine.Prepare(context.Background(), promptkit.RunRequest{
PromptID: PromptID, PromptVersion: SchemaVersion, ProfileID: "combat-test-profile",
Inputs: map[string]scriptorium.ArtifactRef{
"transcript": scriptorium.InlineWithURI("file:///session.json", transcript),
"players": scriptorium.Inline(" "),
"party": scriptorium.Inline(" "),
"glossary": scriptorium.Inline(" "),
"npcs": scriptorium.Inline(" "),
Inputs: map[string]promptkit.ArtifactRef{
"transcript": promptkit.InlineWithURI("file:///session.json", transcript),
"players": promptkit.Inline(" "),
"party": promptkit.Inline(" "),
"glossary": promptkit.Inline(" "),
"npcs": promptkit.Inline(" "),
},
})
if err != nil {

View File

@@ -54,7 +54,7 @@ func New(llmClient contracts.StructuredLLMClient, _ Options, references ...contr
if len(references) > 1 {
return nil, extractorErrorf("at most one reference set may be supplied")
}
promptSHA, err := scriptoriumPromptMetadata()
promptSHA, err := promptAssetMetadata()
if err != nil {
return nil, extractorErrorf("load prompt metadata: %w", err)
}

View File

@@ -9,7 +9,7 @@ import (
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
)
const scriptoriumPromptRoot = "assets/prompts"
const promptAssetRoot = "assets/prompts"
var promptAssetManifest = shared.PromptAssetManifest{
ModuleDir: "dnd.item_events",
@@ -32,21 +32,21 @@ func RegisterPromptAssets(registry *llm.AssetRegistry) error {
if err != nil {
return fmt.Errorf("prepare item event prompt assets: %w", err)
}
if err := registry.RegisterPromptFS(promptFS, scriptoriumPromptRoot); err != nil {
if err := registry.RegisterPromptFS(promptFS, promptAssetRoot); err != nil {
return err
}
return registry.RegisterSchemaFS(embeddedAssets, "assets/schemas")
}
func scriptoriumPromptMetadata() (string, error) {
scriptoriumPromptHashOnce.Do(func() {
scriptoriumPromptHash, scriptoriumPromptHashErr = promptAssetManifest.Hash(embeddedAssets)
func promptAssetMetadata() (string, error) {
promptAssetHashOnce.Do(func() {
promptAssetHash, promptAssetHashErr = promptAssetManifest.Hash(embeddedAssets)
})
return scriptoriumPromptHash, scriptoriumPromptHashErr
return promptAssetHash, promptAssetHashErr
}
var (
scriptoriumPromptHashOnce sync.Once
scriptoriumPromptHash string
scriptoriumPromptHashErr error
promptAssetHashOnce sync.Once
promptAssetHash string
promptAssetHashErr error
)

View File

@@ -9,7 +9,7 @@ import (
"time"
"gitea.maximumdirect.net/eric/notarius/internal/framework/llm"
"gitea.maximumdirect.net/eric/scriptorium"
"gitea.maximumdirect.net/eric/promptkit"
)
func TestPromptAssetsPrepareItemEventPrompt(t *testing.T) {
@@ -17,24 +17,24 @@ func TestPromptAssetsPrepareItemEventPrompt(t *testing.T) {
if err := RegisterPromptAssets(registry); err != nil {
t.Fatal(err)
}
options, err := registry.ScriptoriumOptions()
options, err := registry.PromptKitOptions()
if err != nil {
t.Fatal(err)
}
options = append(options, scriptorium.WithProfiles(scriptorium.OpenAICompatibleProfile(scriptorium.OpenAICompatibleProfileConfig{
options = append(options, promptkit.WithProfiles(promptkit.OpenAICompatibleProfile(promptkit.OpenAICompatibleProfileConfig{
ID: "item-events-test-profile", Endpoint: "http://127.0.0.1:1/v1", Model: "item-events-test-model",
})))
engine, err := scriptorium.NewEngine(scriptorium.Config{Timeout: time.Second}, options...)
engine, err := promptkit.NewEngine(promptkit.Config{Timeout: time.Second}, options...)
if err != nil {
t.Fatal(err)
}
prepared, err := engine.Prepare(context.Background(), scriptorium.RunRequest{
prepared, err := engine.Prepare(context.Background(), promptkit.RunRequest{
PromptID: PromptID, PromptVersion: SchemaVersion, ProfileID: "item-events-test-profile",
Inputs: map[string]scriptorium.ArtifactRef{
"transcript": scriptorium.InlineWithURI("file:///session.json", `{"segments":[1]}`),
"players": scriptorium.Inline(" "),
"party": scriptorium.Inline(" "),
"glossary": scriptorium.Inline(" "),
Inputs: map[string]promptkit.ArtifactRef{
"transcript": promptkit.InlineWithURI("file:///session.json", `{"segments":[1]}`),
"players": promptkit.Inline(" "),
"party": promptkit.Inline(" "),
"glossary": promptkit.Inline(" "),
},
})
if err != nil {
@@ -46,9 +46,9 @@ func TestPromptAssetsPrepareItemEventPrompt(t *testing.T) {
}
func TestPromptAssetsDoNotLeakIntoMetadata(t *testing.T) {
hash, err := scriptoriumPromptMetadata()
hash, err := promptAssetMetadata()
if err != nil || !strings.HasPrefix(hash, "sha256:") {
t.Fatalf("scriptoriumPromptMetadata() = %q, %v", hash, err)
t.Fatalf("promptAssetMetadata() = %q, %v", hash, err)
}
metadata := newExtractor(t, &fakeItemEventsLLMClient{}).ManifestMetadata()
payload, err := json.Marshal(metadata)

View File

@@ -80,7 +80,7 @@ func New(llmClient contracts.StructuredLLMClient, _ Options, references ...contr
if err != nil {
return nil, extractorErrorf("prepare NPC registry prompt input: %w", err)
}
promptSHA, err := scriptoriumPromptMetadata()
promptSHA, err := promptAssetMetadata()
if err != nil {
return nil, extractorErrorf("load prompt metadata: %w", err)
}

View File

@@ -9,7 +9,7 @@ import (
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
)
const scriptoriumPromptRoot = "assets/prompts"
const promptAssetRoot = "assets/prompts"
var promptAssetManifest = shared.PromptAssetManifest{
ModuleDir: "dnd.npc_interactions",
@@ -33,21 +33,21 @@ func RegisterPromptAssets(registry *llm.AssetRegistry) error {
if err != nil {
return fmt.Errorf("prepare NPC-interaction prompt assets: %w", err)
}
if err := registry.RegisterPromptFS(promptFS, scriptoriumPromptRoot); err != nil {
if err := registry.RegisterPromptFS(promptFS, promptAssetRoot); err != nil {
return err
}
return registry.RegisterSchemaFS(embeddedAssets, "assets/schemas")
}
func scriptoriumPromptMetadata() (string, error) {
scriptoriumPromptHashOnce.Do(func() {
scriptoriumPromptHash, scriptoriumPromptHashErr = promptAssetManifest.Hash(embeddedAssets)
func promptAssetMetadata() (string, error) {
promptAssetHashOnce.Do(func() {
promptAssetHash, promptAssetHashErr = promptAssetManifest.Hash(embeddedAssets)
})
return scriptoriumPromptHash, scriptoriumPromptHashErr
return promptAssetHash, promptAssetHashErr
}
var (
scriptoriumPromptHashOnce sync.Once
scriptoriumPromptHash string
scriptoriumPromptHashErr error
promptAssetHashOnce sync.Once
promptAssetHash string
promptAssetHashErr error
)

View File

@@ -8,7 +8,7 @@ import (
"time"
"gitea.maximumdirect.net/eric/notarius/internal/framework/llm"
"gitea.maximumdirect.net/eric/scriptorium"
"gitea.maximumdirect.net/eric/promptkit"
)
func TestRegisterPromptAssetsAndPrepareInteractionPrompt(t *testing.T) {
@@ -23,26 +23,26 @@ func TestRegisterPromptAssetsAndPrepareInteractionPrompt(t *testing.T) {
if _, err := fs.ReadFile(schemaFS, "dnd_npc_interactions_llm.v1.json"); err != nil {
t.Fatalf("response schema asset: %v", err)
}
options, err := registry.ScriptoriumOptions()
options, err := registry.PromptKitOptions()
if err != nil {
t.Fatal(err)
}
options = append(options, scriptorium.WithProfiles(scriptorium.OpenAICompatibleProfile(scriptorium.OpenAICompatibleProfileConfig{
options = append(options, promptkit.WithProfiles(promptkit.OpenAICompatibleProfile(promptkit.OpenAICompatibleProfileConfig{
ID: "npc-interactions-test-profile", Endpoint: "http://127.0.0.1:1/v1", Model: "npc-interactions-test-model",
})))
engine, err := scriptorium.NewEngine(scriptorium.Config{Timeout: time.Second}, options...)
engine, err := promptkit.NewEngine(promptkit.Config{Timeout: time.Second}, options...)
if err != nil {
t.Fatal(err)
}
transcript := `{"units":[1]}`
prepared, err := engine.Prepare(context.Background(), scriptorium.RunRequest{
prepared, err := engine.Prepare(context.Background(), promptkit.RunRequest{
PromptID: PromptID, PromptVersion: SchemaVersion, ProfileID: "npc-interactions-test-profile",
Inputs: map[string]scriptorium.ArtifactRef{
"transcript": scriptorium.InlineWithURI("file:///session.json", transcript),
"players": scriptorium.Inline("Dana: Mira"),
"party": scriptorium.Inline("Mira: ranger"),
"glossary": scriptorium.Inline("Greencloak: title"),
"npcs": scriptorium.Inline(`{"npcs":[{"name":"Mira Thorn"}]}`),
Inputs: map[string]promptkit.ArtifactRef{
"transcript": promptkit.InlineWithURI("file:///session.json", transcript),
"players": promptkit.Inline("Dana: Mira"),
"party": promptkit.Inline("Mira: ranger"),
"glossary": promptkit.Inline("Greencloak: title"),
"npcs": promptkit.Inline(`{"npcs":[{"name":"Mira Thorn"}]}`),
},
})
if err != nil {
@@ -54,9 +54,9 @@ func TestRegisterPromptAssetsAndPrepareInteractionPrompt(t *testing.T) {
}
func TestPromptMetadataDoesNotExposeAssetContent(t *testing.T) {
hash, err := scriptoriumPromptMetadata()
hash, err := promptAssetMetadata()
if err != nil || !strings.HasPrefix(hash, "sha256:") {
t.Fatalf("scriptoriumPromptMetadata() = %q, %v", hash, err)
t.Fatalf("promptAssetMetadata() = %q, %v", hash, err)
}
metadata := newExtractor(t, &fakeInteractionsLLMClient{}).ManifestMetadata()
for _, forbidden := range []string{"common-dnd-system", "dnd_npc_interactions_llm.v1.json"} {

View File

@@ -54,7 +54,7 @@ func New(llmClient contracts.StructuredLLMClient, _ Options, references ...contr
if len(references) > 1 {
return nil, extractorErrorf("at most one reference set may be supplied")
}
promptSHA, err := scriptoriumPromptMetadata()
promptSHA, err := promptAssetMetadata()
if err != nil {
return nil, extractorErrorf("load prompt metadata: %w", err)
}

View File

@@ -9,7 +9,7 @@ import (
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
)
const scriptoriumPromptRoot = "assets/prompts"
const promptAssetRoot = "assets/prompts"
var promptAssetManifest = shared.PromptAssetManifest{
ModuleDir: "dnd.npcs",
@@ -32,21 +32,21 @@ func RegisterPromptAssets(registry *llm.AssetRegistry) error {
if err != nil {
return fmt.Errorf("prepare NPC prompt assets: %w", err)
}
if err := registry.RegisterPromptFS(promptFS, scriptoriumPromptRoot); err != nil {
if err := registry.RegisterPromptFS(promptFS, promptAssetRoot); err != nil {
return err
}
return registry.RegisterSchemaFS(embeddedAssets, "assets/schemas")
}
func scriptoriumPromptMetadata() (string, error) {
scriptoriumPromptHashOnce.Do(func() {
scriptoriumPromptHash, scriptoriumPromptHashErr = promptAssetManifest.Hash(embeddedAssets)
func promptAssetMetadata() (string, error) {
promptAssetHashOnce.Do(func() {
promptAssetHash, promptAssetHashErr = promptAssetManifest.Hash(embeddedAssets)
})
return scriptoriumPromptHash, scriptoriumPromptHashErr
return promptAssetHash, promptAssetHashErr
}
var (
scriptoriumPromptHashOnce sync.Once
scriptoriumPromptHash string
scriptoriumPromptHashErr error
promptAssetHashOnce sync.Once
promptAssetHash string
promptAssetHashErr error
)

View File

@@ -8,7 +8,7 @@ import (
"time"
"gitea.maximumdirect.net/eric/notarius/internal/framework/llm"
"gitea.maximumdirect.net/eric/scriptorium"
"gitea.maximumdirect.net/eric/promptkit"
)
func TestRegisterPromptAssetsAndPrepareNPCPrompt(t *testing.T) {
@@ -16,24 +16,24 @@ func TestRegisterPromptAssetsAndPrepareNPCPrompt(t *testing.T) {
if err := RegisterPromptAssets(registry); err != nil {
t.Fatalf("RegisterPromptAssets() error = %v, want nil", err)
}
options, err := registry.ScriptoriumOptions()
options, err := registry.PromptKitOptions()
if err != nil {
t.Fatalf("ScriptoriumOptions() error = %v, want nil", err)
t.Fatalf("PromptKitOptions() error = %v, want nil", err)
}
options = append(options, scriptorium.WithProfiles(scriptorium.OpenAICompatibleProfile(scriptorium.OpenAICompatibleProfileConfig{
options = append(options, promptkit.WithProfiles(promptkit.OpenAICompatibleProfile(promptkit.OpenAICompatibleProfileConfig{
ID: "npc-test-profile", Endpoint: "http://127.0.0.1:1/v1", Model: "npc-test-model",
})))
engine, err := scriptorium.NewEngine(scriptorium.Config{Timeout: time.Second}, options...)
engine, err := promptkit.NewEngine(promptkit.Config{Timeout: time.Second}, options...)
if err != nil {
t.Fatalf("NewEngine() error = %v, want nil", err)
}
prepared, err := engine.Prepare(context.Background(), scriptorium.RunRequest{
prepared, err := engine.Prepare(context.Background(), promptkit.RunRequest{
PromptID: PromptID, PromptVersion: SchemaVersion, ProfileID: "npc-test-profile",
Inputs: map[string]scriptorium.ArtifactRef{
"transcript": scriptorium.InlineWithURI("file:///session.json", `{"units":[1]}`),
"players": scriptorium.Inline(" "),
"party": scriptorium.Inline(" "),
"glossary": scriptorium.Inline(" "),
Inputs: map[string]promptkit.ArtifactRef{
"transcript": promptkit.InlineWithURI("file:///session.json", `{"units":[1]}`),
"players": promptkit.Inline(" "),
"party": promptkit.Inline(" "),
"glossary": promptkit.Inline(" "),
},
})
if err != nil {
@@ -45,9 +45,9 @@ func TestRegisterPromptAssetsAndPrepareNPCPrompt(t *testing.T) {
}
func TestPromptMetadataAndDiagnosticsDoNotContainRawAssets(t *testing.T) {
hash, err := scriptoriumPromptMetadata()
hash, err := promptAssetMetadata()
if err != nil || !strings.HasPrefix(hash, "sha256:") {
t.Fatalf("scriptoriumPromptMetadata() = %q, %v; want hash", hash, err)
t.Fatalf("promptAssetMetadata() = %q, %v; want hash", hash, err)
}
metadata := newExtractor(t, &fakeNPCsLLMClient{}).ManifestMetadata()
payload, err := json.Marshal(metadata)

View File

@@ -59,7 +59,7 @@ func New(llmClient contracts.StructuredLLMClient, _ Options, references ...contr
if len(references) > 1 {
return nil, extractorErrorf("at most one reference set may be supplied")
}
promptSHA, err := scriptoriumPromptMetadata()
promptSHA, err := promptAssetMetadata()
if err != nil {
return nil, extractorErrorf("load prompt metadata: %w", err)
}

View File

@@ -9,7 +9,7 @@ import (
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
)
const scriptoriumPromptRoot = "assets/prompts"
const promptAssetRoot = "assets/prompts"
var promptAssetManifest = shared.PromptAssetManifest{
ModuleDir: "dnd.scene_descriptions",
@@ -31,21 +31,21 @@ func RegisterPromptAssets(registry *llm.AssetRegistry) error {
if err != nil {
return fmt.Errorf("prepare scene-description prompt assets: %w", err)
}
if err := registry.RegisterPromptFS(promptFS, scriptoriumPromptRoot); err != nil {
if err := registry.RegisterPromptFS(promptFS, promptAssetRoot); err != nil {
return err
}
return registry.RegisterSchemaFS(embeddedAssets, "assets/schemas")
}
func scriptoriumPromptMetadata() (string, error) {
scriptoriumPromptHashOnce.Do(func() {
scriptoriumPromptHash, scriptoriumPromptHashErr = promptAssetManifest.Hash(embeddedAssets)
func promptAssetMetadata() (string, error) {
promptAssetHashOnce.Do(func() {
promptAssetHash, promptAssetHashErr = promptAssetManifest.Hash(embeddedAssets)
})
return scriptoriumPromptHash, scriptoriumPromptHashErr
return promptAssetHash, promptAssetHashErr
}
var (
scriptoriumPromptHashOnce sync.Once
scriptoriumPromptHash string
scriptoriumPromptHashErr error
promptAssetHashOnce sync.Once
promptAssetHash string
promptAssetHashErr error
)

View File

@@ -8,7 +8,7 @@ import (
"time"
"gitea.maximumdirect.net/eric/notarius/internal/framework/llm"
"gitea.maximumdirect.net/eric/scriptorium"
"gitea.maximumdirect.net/eric/promptkit"
)
func TestRegisterPromptAssetsPreparesSceneDescriptionPrompt(t *testing.T) {
@@ -16,24 +16,24 @@ func TestRegisterPromptAssetsPreparesSceneDescriptionPrompt(t *testing.T) {
if err := RegisterPromptAssets(registry); err != nil {
t.Fatalf("RegisterPromptAssets() error = %v, want nil", err)
}
options, err := registry.ScriptoriumOptions()
options, err := registry.PromptKitOptions()
if err != nil {
t.Fatalf("ScriptoriumOptions() error = %v, want nil", err)
t.Fatalf("PromptKitOptions() error = %v, want nil", err)
}
options = append(options, scriptorium.WithProfiles(scriptorium.OpenAICompatibleProfile(scriptorium.OpenAICompatibleProfileConfig{
options = append(options, promptkit.WithProfiles(promptkit.OpenAICompatibleProfile(promptkit.OpenAICompatibleProfileConfig{
ID: "scene-description-test-profile", Endpoint: "http://127.0.0.1:1/v1", Model: "scene-description-test-model",
})))
engine, err := scriptorium.NewEngine(scriptorium.Config{Timeout: time.Second}, options...)
engine, err := promptkit.NewEngine(promptkit.Config{Timeout: time.Second}, options...)
if err != nil {
t.Fatalf("NewEngine() error = %v, want nil", err)
}
prepared, err := engine.Prepare(context.Background(), scriptorium.RunRequest{
prepared, err := engine.Prepare(context.Background(), promptkit.RunRequest{
PromptID: PromptID, PromptVersion: SchemaVersion, ProfileID: "scene-description-test-profile",
Inputs: map[string]scriptorium.ArtifactRef{
"transcript": scriptorium.InlineWithURI("file:///session.json", `{"units":[1]}`),
"players": scriptorium.Inline(" "),
"party": scriptorium.Inline(" "),
"glossary": scriptorium.Inline(" "),
Inputs: map[string]promptkit.ArtifactRef{
"transcript": promptkit.InlineWithURI("file:///session.json", `{"units":[1]}`),
"players": promptkit.Inline(" "),
"party": promptkit.Inline(" "),
"glossary": promptkit.Inline(" "),
},
})
if err != nil {
@@ -45,9 +45,9 @@ func TestRegisterPromptAssetsPreparesSceneDescriptionPrompt(t *testing.T) {
}
func TestPromptMetadataAndDiagnosticsDoNotContainRawAssets(t *testing.T) {
hash, err := scriptoriumPromptMetadata()
hash, err := promptAssetMetadata()
if err != nil || !strings.HasPrefix(hash, "sha256:") {
t.Fatalf("scriptoriumPromptMetadata() = %q, %v; want hash", hash, err)
t.Fatalf("promptAssetMetadata() = %q, %v; want hash", hash, err)
}
payload, err := json.Marshal(newExtractor(t, &fakeSceneDescriptionsLLMClient{}).ManifestMetadata())
if err != nil {

View File

@@ -94,7 +94,7 @@ func New(llmClient contracts.StructuredLLMClient, _ Options, references ...contr
if err != nil {
return nil, extractorErrorf("prepare NPC registry prompt input: %w", err)
}
promptSHA, err := scriptoriumPromptMetadata()
promptSHA, err := promptAssetMetadata()
if err != nil {
return nil, extractorErrorf("load prompt metadata: %w", err)
}

View File

@@ -9,7 +9,7 @@ import (
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
)
const scriptoriumPromptRoot = "assets/prompts"
const promptAssetRoot = "assets/prompts"
var promptAssetManifest = shared.PromptAssetManifest{
ModuleDir: "dnd.spells",
@@ -34,21 +34,21 @@ func RegisterPromptAssets(registry *llm.AssetRegistry) error {
if err != nil {
return fmt.Errorf("prepare spell prompt assets: %w", err)
}
if err := registry.RegisterPromptFS(promptFS, scriptoriumPromptRoot); err != nil {
if err := registry.RegisterPromptFS(promptFS, promptAssetRoot); err != nil {
return err
}
return registry.RegisterSchemaFS(embeddedAssets, "assets/schemas")
}
func scriptoriumPromptMetadata() (string, error) {
scriptoriumPromptHashOnce.Do(func() {
scriptoriumPromptHash, scriptoriumPromptHashErr = promptAssetManifest.Hash(embeddedAssets)
func promptAssetMetadata() (string, error) {
promptAssetHashOnce.Do(func() {
promptAssetHash, promptAssetHashErr = promptAssetManifest.Hash(embeddedAssets)
})
return scriptoriumPromptHash, scriptoriumPromptHashErr
return promptAssetHash, promptAssetHashErr
}
var (
scriptoriumPromptHashOnce sync.Once
scriptoriumPromptHash string
scriptoriumPromptHashErr error
promptAssetHashOnce sync.Once
promptAssetHash string
promptAssetHashErr error
)

View File

@@ -8,10 +8,10 @@ import (
"time"
"gitea.maximumdirect.net/eric/notarius/internal/framework/llm"
"gitea.maximumdirect.net/eric/scriptorium"
"gitea.maximumdirect.net/eric/promptkit"
)
func TestScriptoriumPromptPreparesSpellPrompt(t *testing.T) {
func TestPromptAssetsPrepareSpellPrompt(t *testing.T) {
transcript := []byte(`{"id":"session-1","segments":[{"id":"u1","text":"Mira casts shield."}]}`)
prepared := prepareSpellsPrompt(t, transcript, "Dana: Mira", "Mira: wizard", "Shield: abjuration")
@@ -23,7 +23,7 @@ func TestScriptoriumPromptPreparesSpellPrompt(t *testing.T) {
}
}
func TestScriptoriumPromptPreparesWithMissingOptionalReferences(t *testing.T) {
func TestPromptAssetsPrepareWithMissingOptionalReferences(t *testing.T) {
transcript := []byte(`{"id":"session-1","segments":[]}`)
prepared := prepareSpellsPrompt(t, transcript, " ", " ", " ")
@@ -35,7 +35,7 @@ func TestScriptoriumPromptPreparesWithMissingOptionalReferences(t *testing.T) {
t.Fatalf("reference message did not render empty optional reference placeholders")
}
func TestScriptoriumPromptDiagnosticsOmitRawMaterials(t *testing.T) {
func TestPromptAssetDiagnosticsOmitRawMaterials(t *testing.T) {
transcript := []byte(`{"secret":"source text"}`)
reference := "private party note"
prepared := prepareSpellsPrompt(t, transcript, "private player note", reference, " ")
@@ -82,36 +82,36 @@ func TestScriptoriumPromptDiagnosticsOmitRawMaterials(t *testing.T) {
}
}
func prepareSpellsPrompt(t *testing.T, transcript []byte, players string, party string, glossary string) *scriptorium.PreparedRun {
func prepareSpellsPrompt(t *testing.T, transcript []byte, players string, party string, glossary string) *promptkit.PreparedRun {
t.Helper()
registry := llm.NewAssetRegistry()
if err := RegisterPromptAssets(registry); err != nil {
t.Fatalf("register spell prompt assets: %v", err)
}
options, err := registry.ScriptoriumOptions()
options, err := registry.PromptKitOptions()
if err != nil {
t.Fatalf("ScriptoriumOptions() error = %v, want nil", err)
t.Fatalf("PromptKitOptions() error = %v, want nil", err)
}
options = append(options, scriptorium.WithProfiles(scriptorium.OpenAICompatibleProfile(scriptorium.OpenAICompatibleProfileConfig{
options = append(options, promptkit.WithProfiles(promptkit.OpenAICompatibleProfile(promptkit.OpenAICompatibleProfileConfig{
ID: "spell-test-profile",
Endpoint: "http://127.0.0.1:1/v1",
Model: "spell-test-model",
})))
engine, err := scriptorium.NewEngine(scriptorium.Config{Timeout: time.Second}, options...)
engine, err := promptkit.NewEngine(promptkit.Config{Timeout: time.Second}, options...)
if err != nil {
t.Fatalf("NewEngine() error = %v, want nil", err)
}
prepared, err := engine.Prepare(context.Background(), scriptorium.RunRequest{
prepared, err := engine.Prepare(context.Background(), promptkit.RunRequest{
PromptID: PromptID,
PromptVersion: SchemaVersion,
ProfileID: "spell-test-profile",
Inputs: map[string]scriptorium.ArtifactRef{
"transcript": scriptorium.InlineWithURI("file:///session.json", string(transcript)),
"spell_catalog": scriptorium.Inline(`{"spell_names":["Cure Wounds"]}`),
"npcs": scriptorium.Inline(`{"npcs":[]}`),
"players": scriptorium.Inline(players),
"party": scriptorium.Inline(party),
"glossary": scriptorium.Inline(glossary),
Inputs: map[string]promptkit.ArtifactRef{
"transcript": promptkit.InlineWithURI("file:///session.json", string(transcript)),
"spell_catalog": promptkit.Inline(`{"spell_names":["Cure Wounds"]}`),
"npcs": promptkit.Inline(`{"npcs":[]}`),
"players": promptkit.Inline(players),
"party": promptkit.Inline(party),
"glossary": promptkit.Inline(glossary),
},
})
if err != nil {

View File

@@ -53,7 +53,7 @@ func New(llmClient contracts.StructuredLLMClient, _ Options) (*Normalizer, error
if llmClient == nil {
return nil, normalizerErrorf("LLM client must not be nil")
}
promptSHA, err := scriptoriumPromptMetadata()
promptSHA, err := promptAssetMetadata()
if err != nil {
return nil, normalizerErrorf("load prompt metadata: %w", err)
}

View File

@@ -9,7 +9,7 @@ import (
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
)
const scriptoriumPromptRoot = "assets/prompts"
const promptAssetRoot = "assets/prompts"
var promptAssetManifest = shared.PromptAssetManifest{
ModuleDir: PromptID,
@@ -30,21 +30,21 @@ func RegisterPromptAssets(registry *llm.AssetRegistry) error {
if err != nil {
return fmt.Errorf("prepare NPC normalization prompt assets: %w", err)
}
if err := registry.RegisterPromptFS(promptFS, scriptoriumPromptRoot); err != nil {
if err := registry.RegisterPromptFS(promptFS, promptAssetRoot); err != nil {
return err
}
return registry.RegisterSchemaFS(embeddedAssets, "assets/schemas")
}
func scriptoriumPromptMetadata() (string, error) {
scriptoriumPromptHashOnce.Do(func() {
scriptoriumPromptHash, scriptoriumPromptHashErr = promptAssetManifest.Hash(embeddedAssets)
func promptAssetMetadata() (string, error) {
promptAssetHashOnce.Do(func() {
promptAssetHash, promptAssetHashErr = promptAssetManifest.Hash(embeddedAssets)
})
return scriptoriumPromptHash, scriptoriumPromptHashErr
return promptAssetHash, promptAssetHashErr
}
var (
scriptoriumPromptHashOnce sync.Once
scriptoriumPromptHash string
scriptoriumPromptHashErr error
promptAssetHashOnce sync.Once
promptAssetHash string
promptAssetHashErr error
)

View File

@@ -8,36 +8,36 @@ import (
"time"
"gitea.maximumdirect.net/eric/notarius/internal/framework/llm"
"gitea.maximumdirect.net/eric/scriptorium"
"gitea.maximumdirect.net/eric/promptkit"
)
func TestRegisterPromptAssetsPreparesNormalizationPrompt(t *testing.T) {
if want := []string{"common-dnd-system.md", "common-dnd-transcript.md"}; !reflect.DeepEqual(promptAssetManifest.SharedFiles, want) {
t.Fatalf("shared prompt assets = %#v, want %#v", promptAssetManifest.SharedFiles, want)
}
if promptHash, err := scriptoriumPromptMetadata(); err != nil || promptHash == "" {
t.Fatalf("scriptoriumPromptMetadata() = %q, %v; want prompt fingerprint", promptHash, err)
if promptHash, err := promptAssetMetadata(); err != nil || promptHash == "" {
t.Fatalf("promptAssetMetadata() = %q, %v; want prompt fingerprint", promptHash, err)
}
registry := llm.NewAssetRegistry()
if err := RegisterPromptAssets(registry); err != nil {
t.Fatalf("RegisterPromptAssets() error = %v", err)
}
options, err := registry.ScriptoriumOptions()
options, err := registry.PromptKitOptions()
if err != nil {
t.Fatalf("ScriptoriumOptions() error = %v", err)
t.Fatalf("PromptKitOptions() error = %v", err)
}
options = append(options, scriptorium.WithProfiles(scriptorium.OpenAICompatibleProfile(scriptorium.OpenAICompatibleProfileConfig{
options = append(options, promptkit.WithProfiles(promptkit.OpenAICompatibleProfile(promptkit.OpenAICompatibleProfileConfig{
ID: "normalize-test-profile", Endpoint: "http://127.0.0.1:1/v1", Model: "normalize-test-model",
})))
engine, err := scriptorium.NewEngine(scriptorium.Config{Timeout: time.Second}, options...)
engine, err := promptkit.NewEngine(promptkit.Config{Timeout: time.Second}, options...)
if err != nil {
t.Fatalf("NewEngine() error = %v", err)
}
prepared, err := engine.Prepare(context.Background(), scriptorium.RunRequest{
prepared, err := engine.Prepare(context.Background(), promptkit.RunRequest{
PromptID: PromptID, PromptVersion: SchemaVersion, ProfileID: "normalize-test-profile",
Inputs: map[string]scriptorium.ArtifactRef{
"candidates": scriptorium.Inline(`{"npcs":[{"name":"Mira","source_refs":[]}]}`),
"transcript": scriptorium.Inline(`{"windows":[{"units":[]}]}`),
Inputs: map[string]promptkit.ArtifactRef{
"candidates": promptkit.Inline(`{"npcs":[{"name":"Mira","source_refs":[]}]}`),
"transcript": promptkit.Inline(`{"windows":[{"units":[]}]}`),
},
})
if err != nil {
@@ -55,7 +55,7 @@ func TestRegisterPromptAssetsPreparesNormalizationPrompt(t *testing.T) {
}
}
for _, index := range []int{2, 4} {
if cache := prepared.Messages[index].CacheControl; cache == nil || cache.Type != scriptorium.CacheControlEphemeral {
if cache := prepared.Messages[index].CacheControl; cache == nil || cache.Type != promptkit.CacheControlEphemeral {
t.Errorf("message %d cache control = %#v, want ephemeral", index, cache)
}
}

View File

@@ -14,7 +14,7 @@ import (
npcextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/npcs"
scenedescriptionextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/scenedescriptions"
spellextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/spells"
"gitea.maximumdirect.net/eric/scriptorium"
"gitea.maximumdirect.net/eric/promptkit"
)
func TestExtractionPromptsShareRenderedPrefix(t *testing.T) {
@@ -31,17 +31,17 @@ func TestExtractionPromptsShareRenderedPrefix(t *testing.T) {
t.Fatalf("registerPromptAssets() error = %v", err)
}
engine := newPromptCacheEngine(t, registry)
commonInputs := map[string]scriptorium.ArtifactRef{
"transcript": scriptorium.InlineWithURI("file:///session.json", `{"sentinel":"`+transcriptSentinel+`"}`),
"players": scriptorium.Inline(playersSentinel),
"party": scriptorium.Inline(partySentinel),
"glossary": scriptorium.Inline(glossarySentinel),
commonInputs := map[string]promptkit.ArtifactRef{
"transcript": promptkit.InlineWithURI("file:///session.json", `{"sentinel":"`+transcriptSentinel+`"}`),
"players": promptkit.Inline(playersSentinel),
"party": promptkit.Inline(partySentinel),
"glossary": promptkit.Inline(glossarySentinel),
}
cases := []struct {
name string
promptID string
promptVersion string
inputs map[string]scriptorium.ArtifactRef
inputs map[string]promptkit.ArtifactRef
npcInput bool
spellCatalogInput bool
}{
@@ -52,8 +52,8 @@ func TestExtractionPromptsShareRenderedPrefix(t *testing.T) {
name: "combat turns",
promptID: combatextract.PromptID,
promptVersion: combatextract.SchemaVersion,
inputs: withPromptInputs(commonInputs, map[string]scriptorium.ArtifactRef{
"npcs": scriptorium.Inline(`{"sentinel":"` + npcSentinel + `"}`),
inputs: withPromptInputs(commonInputs, map[string]promptkit.ArtifactRef{
"npcs": promptkit.Inline(`{"sentinel":"` + npcSentinel + `"}`),
}),
npcInput: true,
},
@@ -61,8 +61,8 @@ func TestExtractionPromptsShareRenderedPrefix(t *testing.T) {
name: "npc interactions",
promptID: interactionextract.PromptID,
promptVersion: interactionextract.SchemaVersion,
inputs: withPromptInputs(commonInputs, map[string]scriptorium.ArtifactRef{
"npcs": scriptorium.Inline(`{"sentinel":"` + npcSentinel + `"}`),
inputs: withPromptInputs(commonInputs, map[string]promptkit.ArtifactRef{
"npcs": promptkit.Inline(`{"sentinel":"` + npcSentinel + `"}`),
}),
npcInput: true,
},
@@ -70,19 +70,19 @@ func TestExtractionPromptsShareRenderedPrefix(t *testing.T) {
name: "spells",
promptID: spellextract.PromptID,
promptVersion: spellextract.SchemaVersion,
inputs: withPromptInputs(commonInputs, map[string]scriptorium.ArtifactRef{
"npcs": scriptorium.Inline(`{"sentinel":"` + npcSentinel + `"}`),
"spell_catalog": scriptorium.Inline(`{"sentinel":"` + catalogSentinel + `"}`),
inputs: withPromptInputs(commonInputs, map[string]promptkit.ArtifactRef{
"npcs": promptkit.Inline(`{"sentinel":"` + npcSentinel + `"}`),
"spell_catalog": promptkit.Inline(`{"sentinel":"` + catalogSentinel + `"}`),
}),
npcInput: true,
spellCatalogInput: true,
},
}
var sharedPrefix []scriptorium.RenderedMessage
var sharedPrefix []promptkit.RenderedMessage
for _, testCase := range cases {
t.Run(testCase.name, func(t *testing.T) {
prepared, err := engine.Prepare(context.Background(), scriptorium.RunRequest{
prepared, err := engine.Prepare(context.Background(), promptkit.RunRequest{
PromptID: testCase.promptID,
PromptVersion: testCase.promptVersion,
ProfileID: "prompt-cache-test-profile",
@@ -100,7 +100,7 @@ func TestExtractionPromptsShareRenderedPrefix(t *testing.T) {
t.Fatalf("prepared prompt has %d messages, want lane-specific suffix after transcript", len(prepared.Messages))
}
if sharedPrefix == nil {
sharedPrefix = append([]scriptorium.RenderedMessage(nil), prefix...)
sharedPrefix = append([]promptkit.RenderedMessage(nil), prefix...)
} else if !reflect.DeepEqual(prefix, sharedPrefix) {
t.Fatalf("rendered prefix = %#v, want %#v", prefix, sharedPrefix)
}
@@ -114,24 +114,24 @@ func TestExtractionPromptsShareRenderedPrefix(t *testing.T) {
}
}
func newPromptCacheEngine(t *testing.T, registry *llm.AssetRegistry) *scriptorium.Engine {
func newPromptCacheEngine(t *testing.T, registry *llm.AssetRegistry) *promptkit.Engine {
t.Helper()
options, err := registry.ScriptoriumOptions()
options, err := registry.PromptKitOptions()
if err != nil {
t.Fatalf("ScriptoriumOptions() error = %v", err)
t.Fatalf("PromptKitOptions() error = %v", err)
}
options = append(options, scriptorium.WithProfiles(scriptorium.OpenAICompatibleProfile(scriptorium.OpenAICompatibleProfileConfig{
options = append(options, promptkit.WithProfiles(promptkit.OpenAICompatibleProfile(promptkit.OpenAICompatibleProfileConfig{
ID: "prompt-cache-test-profile", Endpoint: "http://127.0.0.1:1/v1", Model: "prompt-cache-test-model",
})))
engine, err := scriptorium.NewEngine(scriptorium.Config{Timeout: time.Second}, options...)
engine, err := promptkit.NewEngine(promptkit.Config{Timeout: time.Second}, options...)
if err != nil {
t.Fatalf("NewEngine() error = %v", err)
}
return engine
}
func withPromptInputs(inputs, extras map[string]scriptorium.ArtifactRef) map[string]scriptorium.ArtifactRef {
merged := make(map[string]scriptorium.ArtifactRef, len(inputs)+len(extras))
func withPromptInputs(inputs, extras map[string]promptkit.ArtifactRef) map[string]promptkit.ArtifactRef {
merged := make(map[string]promptkit.ArtifactRef, len(inputs)+len(extras))
for name, input := range inputs {
merged[name] = input
}
@@ -141,14 +141,14 @@ func withPromptInputs(inputs, extras map[string]scriptorium.ArtifactRef) map[str
return merged
}
func assertRenderedInputAfter(t *testing.T, messages []scriptorium.RenderedMessage, sentinel string, index int) {
func assertRenderedInputAfter(t *testing.T, messages []promptkit.RenderedMessage, sentinel string, index int) {
t.Helper()
if inputIndex := renderedInputMessageIndex(t, messages, sentinel); inputIndex <= index {
t.Fatalf("input sentinel %q rendered at message %d, want after transcript message %d", sentinel, inputIndex, index)
}
}
func renderedInputMessageIndex(t *testing.T, messages []scriptorium.RenderedMessage, sentinel string) int {
func renderedInputMessageIndex(t *testing.T, messages []promptkit.RenderedMessage, sentinel string) int {
t.Helper()
index := -1
occurrences := 0

View File

@@ -1,4 +1,4 @@
version: 3
version: 4
output:
directory: ./notarius-output
cache:

View File

@@ -1,4 +1,4 @@
version: 3
version: 4
output:
directory: ./notarius-output
cache:

View File

@@ -1,4 +1,4 @@
version: 3
version: 4
output:
directory: ./notarius-output
cache:

View File

@@ -1,4 +1,4 @@
version: 3
version: 4
output:
directory: ./notarius-output
cache: