Document consumer safety and offline execution
This commit is contained in:
15
doc.go
15
doc.go
@@ -18,6 +18,21 @@
|
|||||||
// isolated from engine state. Callers own those copies and may mutate them
|
// isolated from engine state. Callers own those copies and may mutate them
|
||||||
// after the call that supplied or returned them.
|
// after the call that supplied or returned them.
|
||||||
//
|
//
|
||||||
|
// # Security and sensitive data
|
||||||
|
//
|
||||||
|
// The default artifact reader treats [File] paths as caller-selected operating
|
||||||
|
// system paths. It does not restrict them to an application root or impose an
|
||||||
|
// inbound request-size policy. Promptkit is not an inbound request or
|
||||||
|
// untrusted-input security boundary. Applications must validate and restrict
|
||||||
|
// untrusted input before constructing a request, or install an [ArtifactReader]
|
||||||
|
// that enforces their filesystem, authorization, and size policies.
|
||||||
|
//
|
||||||
|
// Rendered messages, input and output [Artifact] bodies, [RunResult.RawOutput],
|
||||||
|
// and [ValidationResult.Errors] may contain sensitive data. Credential
|
||||||
|
// exclusion and redaction do not sanitize those values. Applications and
|
||||||
|
// injected collaborators are responsible for access control, retention,
|
||||||
|
// logging, and secret handling appropriate to their data.
|
||||||
|
//
|
||||||
// # JSON
|
// # JSON
|
||||||
//
|
//
|
||||||
// Stable JSON representations are provided for [PreparedRun], [RunResult],
|
// Stable JSON representations are provided for [PreparedRun], [RunResult],
|
||||||
|
|||||||
@@ -16,8 +16,8 @@ import "gitea.maximumdirect.net/eric/promptkit"
|
|||||||
```
|
```
|
||||||
|
|
||||||
The following Go fragments are illustrative and omit surrounding package,
|
The following Go fragments are illustrative and omit surrounding package,
|
||||||
import, and error-handling code. Use the maintained example for a complete
|
import, and error-handling code. Use the maintained examples for complete
|
||||||
program.
|
programs.
|
||||||
|
|
||||||
## Construct An Engine
|
## Construct An Engine
|
||||||
|
|
||||||
@@ -68,6 +68,13 @@ configured model client, classifies the generated artifact, and validates the
|
|||||||
content. A completed content check may return `ValidationFailed` in the result;
|
content. A completed content check may return `ValidationFailed` in the result;
|
||||||
an operational inability to validate returns an error.
|
an operational inability to validate returns an error.
|
||||||
|
|
||||||
|
The maintained
|
||||||
|
[offline execution example](../../examples/go-library/run/main.go) injects a
|
||||||
|
deterministic model client and exercises `Run` without credentials, network
|
||||||
|
access, or paid calls. It is intentionally separate from the preparation
|
||||||
|
example so each workflow and its small prompt fixture can be copied and run on
|
||||||
|
its own.
|
||||||
|
|
||||||
Use the [`RunResult` and `ValidationResult` GoDoc](../../types.go) for the
|
Use the [`RunResult` and `ValidationResult` GoDoc](../../types.go) for the
|
||||||
returned data and the `Engine.Run` GoDoc for failure and cancellation
|
returned data and the `Engine.Run` GoDoc for failure and cancellation
|
||||||
semantics. The
|
semantics. The
|
||||||
@@ -98,12 +105,29 @@ from supported JSON values and the package's `String` and `GoString`
|
|||||||
summaries. The exact precedence and redaction guarantees belong to
|
summaries. The exact precedence and redaction guarantees belong to
|
||||||
[`RunRequest`, `GenerateRequest`, and the profile GoDoc](../../types.go).
|
[`RunRequest`, `GenerateRequest`, and the profile GoDoc](../../types.go).
|
||||||
|
|
||||||
|
## Protect Files And Generated Data
|
||||||
|
|
||||||
|
The default artifact reader opens a `File` reference as a caller-selected
|
||||||
|
operating-system path. It does not constrain paths to an application root,
|
||||||
|
impose an inbound request-size policy, or establish an untrusted-input security
|
||||||
|
boundary. Applications must validate and restrict untrusted paths and payloads
|
||||||
|
before constructing a request, or inject an artifact reader that enforces
|
||||||
|
their filesystem, authorization, and size policies.
|
||||||
|
|
||||||
|
Rendered messages, input and output artifact bodies, raw model output, and
|
||||||
|
validation diagnostics can contain sensitive data. API-key redaction does not
|
||||||
|
sanitize those values. Treat prepared values, results, collaborator requests,
|
||||||
|
errors, and logs according to the application's data-access, retention, and
|
||||||
|
secret-handling policies.
|
||||||
|
|
||||||
## Extension Interfaces
|
## Extension Interfaces
|
||||||
|
|
||||||
Inject an [`LLMClient` or `ArtifactReader`](../../types.go) when the built-in
|
Inject an [`LLMClient` or `ArtifactReader`](../../types.go) when the built-in
|
||||||
behavior does not fit the application. Their GoDoc defines concurrent use,
|
behavior does not fit the application. Their GoDoc defines concurrent use,
|
||||||
context handling, ownership of copied values, nil responses, and preservation
|
context handling, ownership of copied values, nil responses, and preservation
|
||||||
of collaborator errors.
|
of collaborator errors. Implementations must honor cancellation, safely manage
|
||||||
|
copies they retain, avoid unsafe logging of content or credentials, and enforce
|
||||||
|
the application policy that motivated the injection.
|
||||||
|
|
||||||
## Handle Errors
|
## Handle Errors
|
||||||
|
|
||||||
@@ -119,4 +143,5 @@ documented.
|
|||||||
|
|
||||||
Promptkit is an importable library. It does not own a command, inbound HTTP
|
Promptkit is an importable library. It does not own a command, inbound HTTP
|
||||||
API, process configuration, or deployment policy. Applications map the root
|
API, process configuration, or deployment policy. Applications map the root
|
||||||
package's results and errors into those concerns.
|
package's results and errors into those concerns, including inbound size and
|
||||||
|
trust policy.
|
||||||
|
|||||||
@@ -13,6 +13,7 @@ contributor workflow and validation.
|
|||||||
| --- | --- | --- |
|
| --- | --- | --- |
|
||||||
| Root `promptkit` package | Provides the supported engine facade, source and injection options, public request and result values, built-in profile construction, extension interfaces, value conversion, redacted formatting, and public error mapping. | [Package GoDoc](../../doc.go), [engine assembly](../../engine.go) |
|
| Root `promptkit` package | Provides the supported engine facade, source and injection options, public request and result values, built-in profile construction, extension interfaces, value conversion, redacted formatting, and public error mapping. | [Package GoDoc](../../doc.go), [engine assembly](../../engine.go) |
|
||||||
| `examples/go-library/prepare` | Demonstrates an offline downstream consumer using a prompt file, in-memory profile, inline input, and `Prepare`. It is not a public library package. | [Example program](../../examples/go-library/prepare/main.go) |
|
| `examples/go-library/prepare` | Demonstrates an offline downstream consumer using a prompt file, in-memory profile, inline input, and `Prepare`. It is not a public library package. | [Example program](../../examples/go-library/prepare/main.go) |
|
||||||
|
| `examples/go-library/run` | Demonstrates an offline downstream consumer using a prompt file, in-memory profile, inline input, an injected deterministic model client, and `Run`. It is not a public library package. | [Example program](../../examples/go-library/run/main.go) |
|
||||||
| `internal/domain` | Defines internal framework values for requests, artifacts, prompt definitions, profiles, execution targets, rendering, generation, and validation. | [Domain declarations](../../internal/domain/domain.go) |
|
| `internal/domain` | Defines internal framework values for requests, artifacts, prompt definitions, profiles, execution targets, rendering, generation, and validation. | [Domain declarations](../../internal/domain/domain.go) |
|
||||||
| `internal/defaults` | Defines application-neutral framework constants and constructs the default execution target. It contains no CLI, server, or inbound HTTP limits. | [Framework defaults](../../internal/defaults/defaults.go) |
|
| `internal/defaults` | Defines application-neutral framework constants and constructs the default execution target. It contains no CLI, server, or inbound HTTP limits. | [Framework defaults](../../internal/defaults/defaults.go) |
|
||||||
| `internal/filecatalog` | Provides deterministic YAML discovery and path helpers for operating-system filesystems and `fs.FS` sources. | [File catalog](../../internal/filecatalog/catalog.go) |
|
| `internal/filecatalog` | Provides deterministic YAML discovery and path helpers for operating-system filesystems and `fs.FS` sources. | [File catalog](../../internal/filecatalog/catalog.go) |
|
||||||
|
|||||||
@@ -41,9 +41,9 @@ The implemented internal components consist of:
|
|||||||
- `internal/usecase`, which coordinates preparation and execution across the
|
- `internal/usecase`, which coordinates preparation and execution across the
|
||||||
internal framework components.
|
internal framework components.
|
||||||
|
|
||||||
The `examples/go-library/prepare` package is a maintained downstream consumer
|
The `examples/go-library/prepare` and `examples/go-library/run` packages are
|
||||||
of the root facade. It does not expose a library package or participate in
|
maintained downstream consumers of the root facade. They do not expose library
|
||||||
internal assembly.
|
packages or participate in internal assembly.
|
||||||
|
|
||||||
The root facade assembles the internal repositories, renderer, validator,
|
The root facade assembles the internal repositories, renderer, validator,
|
||||||
outbound client, and use-case runner while translating public values and
|
outbound client, and use-case runner while translating public values and
|
||||||
|
|||||||
@@ -106,21 +106,25 @@ documented behavior whose compatibility risk warrants durable coverage.
|
|||||||
Move consumer-relevant security boundaries to the places where consumers will
|
Move consumer-relevant security boundaries to the places where consumers will
|
||||||
encounter them and add one representative execution workflow.
|
encounter them and add one representative execution workflow.
|
||||||
|
|
||||||
- [ ] Explain in public GoDoc and the consumer guide that the default file
|
- [x] Explain in public GoDoc and the consumer guide that the default file
|
||||||
artifact reader accepts unrestricted caller-selected paths.
|
artifact reader accepts unrestricted caller-selected paths.
|
||||||
- [ ] Make clear that Promptkit does not impose an application root, inbound
|
- [x] Make clear that Promptkit does not impose an application root, inbound
|
||||||
request-size policy, or untrusted-input security boundary.
|
request-size policy, or untrusted-input security boundary.
|
||||||
- [ ] Explain that rendered messages, artifact bodies, raw model output, and
|
- [x] Explain that rendered messages, artifact bodies, raw model output, and
|
||||||
validation details may be sensitive even when credentials are redacted.
|
validation details may be sensitive even when credentials are redacted.
|
||||||
- [ ] Clarify the responsibilities of injected artifact readers and model
|
- [x] Clarify the responsibilities of injected artifact readers and model
|
||||||
clients for cancellation, copying, logging, and secret handling.
|
clients for cancellation, copying, logging, and secret handling.
|
||||||
- [ ] Add a maintained offline `Run` example using an injected deterministic
|
- [x] Add a maintained offline `Run` example using an injected deterministic
|
||||||
model client, without credentials, live network access, or paid calls.
|
model client, without credentials, live network access, or paid calls.
|
||||||
- [ ] Link the consumer guide to the execution example and keep embedded
|
- [x] Link the consumer guide to the execution example and keep embedded
|
||||||
snippets smaller than the maintained artifact.
|
snippets smaller than the maintained artifact.
|
||||||
- [ ] Decide whether the existing preparation example should remain separate
|
- [x] Decide whether the existing preparation example should remain separate
|
||||||
or share reusable fixtures without obscuring either workflow.
|
or share reusable fixtures without obscuring either workflow.
|
||||||
|
|
||||||
|
The preparation and execution examples remain separate, self-contained
|
||||||
|
workflows. Each keeps its own small prompt fixture so consumers can copy or run
|
||||||
|
one example without depending on the other.
|
||||||
|
|
||||||
**Gate:** Both preparation and execution have complete, secret-free,
|
**Gate:** Both preparation and execution have complete, secret-free,
|
||||||
deterministic consumer examples, and the consumer guide exposes the important
|
deterministic consumer examples, and the consumer guide exposes the important
|
||||||
filesystem and data-sensitivity boundaries without leaking internal mechanics.
|
filesystem and data-sensitivity boundaries without leaking internal mechanics.
|
||||||
|
|||||||
81
examples/go-library/run/main.go
Normal file
81
examples/go-library/run/main.go
Normal file
@@ -0,0 +1,81 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/promptkit"
|
||||||
|
)
|
||||||
|
|
||||||
|
type deterministicClient struct{}
|
||||||
|
|
||||||
|
func (deterministicClient) Generate(
|
||||||
|
ctx context.Context,
|
||||||
|
_ promptkit.GenerateRequest,
|
||||||
|
) (*promptkit.GenerateResponse, error) {
|
||||||
|
if err := ctx.Err(); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
return &promptkit.GenerateResponse{
|
||||||
|
Content: "Ada finished the migration review.",
|
||||||
|
Usage: promptkit.TokenUsage{
|
||||||
|
PromptTokens: 12,
|
||||||
|
CompletionTokens: 6,
|
||||||
|
TotalTokens: 18,
|
||||||
|
},
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
type summary struct {
|
||||||
|
Output string `json:"output"`
|
||||||
|
ValidationStatus promptkit.ValidationStatus `json:"validation_status"`
|
||||||
|
IsValid bool `json:"is_valid"`
|
||||||
|
Model string `json:"model"`
|
||||||
|
TotalTokens int `json:"total_tokens"`
|
||||||
|
}
|
||||||
|
|
||||||
|
func main() {
|
||||||
|
engine, err := promptkit.NewEngine(
|
||||||
|
promptkit.Config{},
|
||||||
|
promptkit.WithPromptFile("examples/go-library/run/prompt.yaml"),
|
||||||
|
promptkit.WithProfiles(promptkit.Profile{
|
||||||
|
ID: "offline-example",
|
||||||
|
Endpoint: "https://example.invalid/v1",
|
||||||
|
Model: "offline-model",
|
||||||
|
}),
|
||||||
|
promptkit.WithLLMClient(deterministicClient{}),
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
exit(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
result, err := engine.Run(context.Background(), promptkit.RunRequest{
|
||||||
|
PromptID: "example.run",
|
||||||
|
Inputs: map[string]promptkit.ArtifactRef{
|
||||||
|
"note": promptkit.Inline("Ada finished the migration review."),
|
||||||
|
},
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
exit(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
encoder := json.NewEncoder(os.Stdout)
|
||||||
|
encoder.SetIndent("", " ")
|
||||||
|
if err := encoder.Encode(summary{
|
||||||
|
Output: result.RawOutput,
|
||||||
|
ValidationStatus: result.Validation.Status,
|
||||||
|
IsValid: result.Validation.IsValid,
|
||||||
|
Model: result.ModelName,
|
||||||
|
TotalTokens: result.Usage.TotalTokens,
|
||||||
|
}); err != nil {
|
||||||
|
exit(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func exit(err error) {
|
||||||
|
fmt.Fprintln(os.Stderr, err)
|
||||||
|
os.Exit(1)
|
||||||
|
}
|
||||||
16
examples/go-library/run/prompt.yaml
Normal file
16
examples/go-library/run/prompt.yaml
Normal file
@@ -0,0 +1,16 @@
|
|||||||
|
id: example.run
|
||||||
|
version: "1.0.0"
|
||||||
|
default_profile: offline-example
|
||||||
|
description: Run a prompt with a deterministic injected model client.
|
||||||
|
inputs:
|
||||||
|
- name: note
|
||||||
|
required: true
|
||||||
|
content_type: text/plain
|
||||||
|
messages:
|
||||||
|
- role: system
|
||||||
|
content: Summarize the note in one sentence.
|
||||||
|
- role: user
|
||||||
|
content: '{{input "note"}}'
|
||||||
|
output:
|
||||||
|
format: text
|
||||||
|
validation_mode: basic
|
||||||
15
types.go
15
types.go
@@ -244,6 +244,11 @@ type Artifact struct {
|
|||||||
// value. Readers supply artifact metadata, and the engine assigns an input-map
|
// value. Readers supply artifact metadata, and the engine assigns an input-map
|
||||||
// name only when the returned artifact name is empty.
|
// name only when the returned artifact name is empty.
|
||||||
//
|
//
|
||||||
|
// An injected reader owns any application-specific path containment,
|
||||||
|
// authorization, content-size, and content-type policy. It must protect
|
||||||
|
// sensitive references and bodies in its logging and in any copies it retains.
|
||||||
|
// It may reuse or mutate the returned artifact and body after Read returns.
|
||||||
|
//
|
||||||
// Returning a non-nil error makes the engine return an error matching
|
// Returning a non-nil error makes the engine return an error matching
|
||||||
// ErrArtifactLoad while preserving the reader error through errors.Is.
|
// ErrArtifactLoad while preserving the reader error through errors.Is.
|
||||||
// Returning a nil artifact with a nil error also produces ErrArtifactLoad.
|
// Returning a nil artifact with a nil error also produces ErrArtifactLoad.
|
||||||
@@ -519,6 +524,11 @@ type StructuredOutputJSONSpec struct {
|
|||||||
// slices, and pointers are client-owned copies and may be mutated or retained
|
// slices, and pointers are client-owned copies and may be mutated or retained
|
||||||
// without affecting engine state.
|
// without affecting engine state.
|
||||||
//
|
//
|
||||||
|
// Generate receives rendered messages and may receive a direct API key. A
|
||||||
|
// client must protect those values and any raw output in its logging, storage,
|
||||||
|
// and retained copies. It is responsible for the cancellation behavior of any
|
||||||
|
// work it starts and for synchronizing access to retained or shared data.
|
||||||
|
//
|
||||||
// A returned error makes Run return ErrLLMGenerate while preserving the client
|
// A returned error makes Run return ErrLLMGenerate while preserving the client
|
||||||
// error through errors.Is. A nil response with a nil error also produces
|
// error through errors.Is. A nil response with a nil error also produces
|
||||||
// ErrLLMGenerate. Promptkit copies the non-nil response before returning from
|
// ErrLLMGenerate. Promptkit copies the non-nil response before returning from
|
||||||
@@ -557,6 +567,11 @@ type GenerateResponse struct {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// File returns a file-backed artifact reference whose URI is path.
|
// File returns a file-backed artifact reference whose URI is path.
|
||||||
|
//
|
||||||
|
// The default artifact reader opens path as a caller-selected operating-system
|
||||||
|
// path without restricting it to an application root or imposing a size limit.
|
||||||
|
// Applications accepting untrusted paths must validate them before calling
|
||||||
|
// Promptkit or use [WithArtifactReader] to enforce application policy.
|
||||||
func File(path string) ArtifactRef {
|
func File(path string) ArtifactRef {
|
||||||
return ArtifactRef{Type: ArtifactRefFile, URI: path}
|
return ArtifactRef{Type: ArtifactRefFile, URI: path}
|
||||||
}
|
}
|
||||||
|
|||||||
Reference in New Issue
Block a user