eric/audita
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 6 Wiki Activity

Document the PromptKit configuration migration

Browse Source
This commit is contained in:
Eric Rakestraw
2026-10-04 01:14:53 +00:00
parent 5525fe875e
commit 9bcf18331e
9 changed files with 160 additions and 47 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
README.md
View File

@@ -15,7 +15,7 @@ go build -o ./bin/audita ./cmd/audita
Processing requires a reachable [compatible LLM endpoint](docs/integrations/openai-compatible-llm.md) Processing requires a reachable [compatible LLM endpoint](docs/integrations/openai-compatible-llm.md)
and access to a model you select. The [minimal config](examples/minimal-config.yml) and access to a model you select. The [minimal config](examples/minimal-config.yml)
targets an unauthenticated local endpoint; edit its endpoint and targets an unauthenticated local endpoint; edit its endpoint and
[local profiles](examples/profiles/) to match your service and models. The [local profiles](examples/profiles/local/) to match your service and models. The
[service example](examples/production-config.yml) demonstrates a named credential [service example](examples/production-config.yml) demonstrates a named credential
reference and a shared backend. Set its referenced environment variable before reference and a shared backend. Set its referenced environment variable before
validating it. The [testing overlay](examples/overlays/testing.yml) selects both validating it. The [testing overlay](examples/overlays/testing.yml) selects both

100
docs/config.md
View File

@@ -2,8 +2,10 @@
This reference describes the version-2 configuration loader in `internal/core/config`. This reference describes the version-2 configuration loader in `internal/core/config`.
Use the maintained [local](../examples/minimal-config.yml) and Use the maintained [local](../examples/minimal-config.yml) and
[service](../examples/production-config.yml) examples with their PromptKit [service](../examples/production-config.yml) examples with their separately
[profiles](../examples/profiles/) and optional [testing overlay](../examples/overlays/testing.yml). selected [local profiles](../examples/profiles/local/) and
[service profiles](../examples/profiles/service/), plus the optional
[testing overlay](../examples/overlays/testing.yml).
The [ConfigKit inheritance example](../examples/configkit-inheritance.yml) The [ConfigKit inheritance example](../examples/configkit-inheritance.yml)
shows a local child backend reusing and overriding sibling settings. shows a local child backend reusing and overriding sibling settings.
Choose real models for your endpoint; a testing profile makes no cost or quality Choose real models for your endpoint; a testing profile makes no cost or quality
@@ -36,9 +38,7 @@ CLI setting overrides. ConfigKit consumes it before typed decoding. A previously
valid backend whose ID is literally `extends` cannot be represented under this valid backend whose ID is literally `extends` cannot be represented under this
rule. Rename that backend to a non-conflicting ID and update every PromptKit rule. Rename that backend to a non-conflicting ID and update every PromptKit
profile that selects it. This compatibility impact means the upgrade does not profile that selects it. This compatibility impact means the upgrade does not
promise that every previously valid configuration remains valid. PromptKit promise that every previously valid configuration remains valid.
profile documents keep their existing syntax and do not gain ConfigKit
inheritance.
Inherited values count as explicit input, including zero, false, empty, and Inherited values count as explicit input, including zero, false, empty, and
null. Existing field rules still apply: for example, inherited backend null. Existing field rules still apply: for example, inherited backend
@@ -55,20 +55,92 @@ when that setting can be decoded. ConfigKit retains field origins and derivation
for effective-configuration inspection. Its effective YAML redacts resolved for effective-configuration inspection. Its effective YAML redacts resolved
credentials. credentials.
Profile files are resolved against the selected main file's directory, or may ### PromptKit profiles and operator prompts
use an absolute `profiles_dir`. Both selected profiles must exist and select a
registered backend. Their effective endpoints must match that backend's endpoint `profiles_dir` selects a PromptKit configuration root, resolved against the
after surrounding whitespace and trailing slashes are removed. Profiles cannot selected main file's directory unless it is absolute. The root may contain
supply a competing `api_key_env` lookup. The backend credential supplies the keyed `profiles`, `backends`, and `prompts` definitions; it is not limited to
runtime key when required; inspection makes no provider call. The profile-only files. IDs come from literal YAML mapping keys, never filenames.
[CLI inspection views](cli.md#streams-and-exit-status) show selected controls and Multiple definitions may share a document, and a document may contribute to any
field provenance separately from ConfigKit's redacted settings YAML. of the supported mappings. Directory discovery is recursive, but discovery
order does not establish field precedence. Contributions within each source
group must compose without conflicting values.
PromptKit assembles sources in this order: its selected catalogs and owned
defaults, Audita's embedded prompt/schema inputs, operator documents under
`profiles_dir`, then the application backend mapping. The application mapping
owns each configured endpoint, concurrency limit, queue capacity, and credential
environment lookup. Other supported provider fields can be contributed by the
operator documents. Imports remain disabled. The current maintained examples
select separate roots: [local](../examples/profiles/local/) and
[service](../examples/profiles/service/), so definitions for one backend do not
enter the other example's final configuration.
Both selected roles must resolve to an existing profile, registered backend,
nonblank effective model, and nonblank endpoint. The normalized effective
endpoint must match the application backend endpoint. A selected profile cannot
introduce a competing `api_key_env` lookup; the application backend credential
supplies the runtime key when required. Other unselected profiles may be
partial, but every final backend, profile, and prompt definition is validated
when the engine is constructed. A malformed or conflicting unused definition
can therefore make the selected root invalid. A missing selected root is an
operator configuration error; an empty root may contribute no definitions.
Profile IDs and parent IDs are exact, literal keys. PromptKit profiles can use
native `extends` inheritance; child fields override inherited fields, while
unmentioned controls survive changes to model or backend. For example:
```yaml
profiles:
"shared~base":
backend: service
model: example-model
temperature: 0.4
top_p: 0.8
"proposal fast":
extends: "shared~base"
model: faster-model
temperature: 0
```
Presence matters. Explicit `temperature: 0` is sent as zero; omission leaves it
unspecified; null suppresses an inherited temperature. Other fields have their
own native null and omission rules. Provider `extra_params` maps merge
recursively, and a null provider value is sent as JSON null. See the pinned
[PromptKit format reference](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.12.0/docs/formats.md)
for the complete field semantics.
Operator documents can also override an embedded prompt/version, including its
messages and definition-level output settings. Shared variables supplied by the
adapter are rendered only when an effective message references them; Audita does
not append omitted shared text. Audita's per-call schemas, structural repair
budget, and downstream domain validators remain in force. The engine captures
effective definitions and message-file contents when constructed, so edits are
observed only by a newly constructed runtime.
### Migrating PromptKit profile documents
For the PromptKit v0.11 format, wrap each profile's fields under
`profiles: {<literal-id>: ...}`, remove inner `id`, and replace
`base_profile` with `extends`. Profile and prompt IDs are mapping keys, so they
need not match source filenames. Review zeros, omitted values, and nulls against
the native presence rules before deploying. Select a root whose final
definitions have valid backend references; eager validation applies even to
unselected definitions. See the complete maintained [local](../examples/minimal-config.yml)
and [service](../examples/production-config.yml) examples and the pinned
[consumer migration guide](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.12.0/docs/consumers/pkg-promptkit.md#migrate-from-the-previous-configuration-api).
This incompatible profile-format migration is intended for an Audita major
release; no automatic file conversion is provided.
The [CLI inspection views](cli.md#streams-and-exit-status) show selected
controls and field provenance separately from ConfigKit's redacted settings
YAML. Inspection resolves both roles without provider calls.
## Settings ## Settings
| Field | Default | Rule | | Field | Default | Rule |
| --- | --- | --- | | --- | --- | --- |
| `profiles_dir` | `profiles` | Nonempty; profile files are resolved relative to the selected main file. | | `profiles_dir` | `profiles` | Nonempty PromptKit document root; resolved relative to the selected main file, or absolute. |
| `backends.<id>.endpoint` | None | Required nonempty endpoint for each backend. At least one backend is required. | | `backends.<id>.endpoint` | None | Required nonempty endpoint for each backend. At least one backend is required. |
| `backends.<id>.concurrency` | `1` | Positive finite capacity; shared by both roles using that backend. | | `backends.<id>.concurrency` | `1` | Positive finite capacity; shared by both roles using that backend. |
| `backends.<id>.queue_capacity` | `1024` | Nonnegative; zero disables waiting admission. | | `backends.<id>.queue_capacity` | `1024` | Nonnegative; zero disables waiting admission. |

2
docs/integrations/openai-compatible-llm.md
View File

@@ -19,7 +19,7 @@ failures; explicit zero makes one attempt. `llm.repair_attempts` counts
additional generations for schema-invalid model output. The operation timeout additional generations for schema-invalid model output. The operation timeout
covers preparation, capacity waiting, retries, generation, and repair. Caller covers preparation, capacity waiting, retries, generation, and repair. Caller
cancellation or an earlier deadline also applies. See [configuration](../config.md#settings) cancellation or an earlier deadline also applies. See [configuration](../config.md#settings)
for defaults and [PromptKit's consumer contract](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.11.2/docs/consumers/pkg-promptkit.md) for defaults and [PromptKit v0.12.0's consumer contract](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.12.0/docs/consumers/pkg-promptkit.md)
for the pinned protocol and retry details. for the pinned protocol and retry details.
A model operation that exhausts structural repair, a live-parent local operation A model operation that exhausts structural repair, a live-parent local operation

5
docs/internal/cli-config.md
View File

@@ -42,6 +42,11 @@ go run ./cmd/audita config print-effective --config examples/minimal-config.yml
go run ./cmd/audita config print-effective --config examples/configkit-inheritance.yml --view explain go run ./cmd/audita config print-effective --config examples/configkit-inheritance.yml --view explain
``` ```
The local and ConfigKit inheritance files select `profiles/local`; the service
file selects `profiles/service`, including the service testing profiles. Keep
these roots separate because PromptKit validates every final definition in a
selected root, including definitions that are not selected for a role.
Only referenced credentials read environment values; use a controlled synthetic Only referenced credentials read environment values; use a controlled synthetic
value when validating the service example. Check affected CLI examples against the value when validating the service example. Check affected CLI examples against the
parser and use schema tests for maintained transcript/glossary inputs. Example parser and use schema tests for maintained transcript/glossary inputs. Example

27
docs/internal/llm-runtime.md
View File

@@ -2,11 +2,28 @@
## Client Boundary ## Client Boundary
`llm.NewRuntime` assembles one PromptKit engine for both roles. It loads embedded `llm.NewRuntime` assembles one PromptKit engine for both roles. PromptKit's
prompt and schema assets, registers application backends, and inspects the catalogs and defaults are followed by Audita's embedded prompt and schema
selected proposal and validation profiles offline. A role-bound `llm.RoleClient` inputs, operator documents from `profiles_dir`, and the highest-priority
implements `structuredoutput.Client`; proposal and validator contracts alias that application backend mapping. Audita owns backend endpoints, capacity settings,
interface. Both roles share engine-local backend capacity. and resolved credentials; native PromptKit layering owns profile inheritance,
prompt overrides, and provider-specific fields. Imports are disabled. The
runtime inspects both selected roles offline, requiring complete model and
endpoint values and rejecting competing profile credential lookups. A
role-bound `llm.RoleClient` implements `structuredoutput.Client`; proposal and
validator contracts alias that interface. Both roles share engine-local backend
capacity.
Successful construction captures the final definitions and referenced message
contents. The runtime does not reload operator documents or message files during
calls. If final engine construction fails, an embedded-only diagnostic
construction distinguishes shipped asset failures from invalid operator input.
An asset failure remains an asset error; when embedded construction succeeds,
the original failure remains a configuration error. The diagnostic engine
never replaces or executes the requested configuration, and caller cancellation
skips this path. The captured inspection also supplies an internal source
ledger: all loaded host YAML plus effective, source-relative host
`content_file` resources are tracked for output-collision checks.
For each logical call, the adapter starts an operation context and a diagnostic For each logical call, the adapter starts an operation context and a diagnostic
record before preparation. It supplies the reserved hardening text, the selected record before preparation. It supplies the reserved hardening text, the selected

31
docs/internal/prompts.md
View File

@@ -11,18 +11,27 @@
- `validators.meaning_reversal_review` (`v1`) - `validators.meaning_reversal_review` (`v1`)
- `validators.editorial_review` (`v1`) - `validators.editorial_review` (`v1`)
PromptKit reads the embedded definitions and renders their system and user PromptKit reads the embedded definitions as shipped defaults and renders their
messages from named string variables. The role-bound adapter supplies messages from named string variables. Operator documents in `profiles_dir` may
contribute native keyed prompt definitions and override an embedded
prompt/version, including its messages and definition-level output settings.
The effective operator text is captured when the runtime is constructed; the
embedded assets remain the source of Audita's shipped task instructions. The
role-bound adapter supplies
`HardeningPolicy` alongside `ProposalContract` and `ValidationContract` from `HardeningPolicy` alongside `ProposalContract` and `ValidationContract` from
embedded fragments under `assets/shared`, after caller variables so callers cannot replace those embedded fragments under `assets/shared`, after caller variables so callers
instructions. Common literal-span and identity guidance lives in the proposal cannot replace those instructions. Common literal-span and identity guidance
fragment; shared candidate-reference, context, and confidence guidance lives in lives in the proposal fragment; shared candidate-reference, context, and
the validation fragment. Module-specific policies and examples remain in YAML. confidence guidance lives in the validation fragment. Module-specific policies
Proposal prompts receive the section JSON, glossary JSON where used, and transcript description block. and examples remain in YAML. Proposal prompts receive the section JSON, glossary
Validator prompts receive the correction payload and description block. JSON where used, and transcript description block. Validator prompts receive
The prompt assets retain the prior task instructions and output-schema the correction payload and description block.
association. The role-bound runtime supplies the complete JSON Schema output contract The embedded prompt assets retain the shipped task instructions and
and repair count; the definition does not own an operator-facing repair default. output-schema association. An operator override controls which template
variables its messages reference: supplying a shared variable does not append
its text when the effective message omits that reference. The role-bound runtime
supplies the complete JSON Schema output contract and repair count; the
definition does not own an operator-facing repair default.
The correction-set schema requires a string replacement and accepts an empty The correction-set schema requires a string replacement and accepts an empty
string. Proposal guidance uses it for eligible span deletion and permits literal string. Proposal guidance uses it for eligible span deletion and permits literal
spacing edits; validator guidance evaluates the full before/after segment under spacing edits; validator guidance evaluates the full before/after segment under

10
docs/operations.md
View File

@@ -29,9 +29,13 @@ fatal errors. Stdout contains only transcript JSON; file output leaves it empty.
Existing report directories and parents for regular transcript files must be Existing report directories and parents for regular transcript files must be
writable. Audita prevents report writes from overwriting transcript, glossary, writable. Audita prevents report writes from overwriting transcript, glossary,
loaded configuration, or loaded profile sources, and rejects conflicting output loaded application configuration, host YAML discovered under `profiles_dir`,
destinations. The final transcript may intentionally replace its own input after or host message files referenced by effective PromptKit `content_file` fields,
successful processing; other input/configuration/profile sources remain protected. and rejects conflicting output destinations. Source paths come from the
runtime's captured configuration; unrelated files in the selected directory
remain eligible output destinations. The final transcript may intentionally
replace its own input after successful processing; other input/configuration/
profile sources remain protected.
Preparation never truncates an existing transcript. Device and pipe destinations Preparation never truncates an existing transcript. Device and pipe destinations
(such as `/dev/null` and named pipes) receive ordinary writes at publication time; (such as `/dev/null` and named pipes) receive ordinary writes at publication time;
they are not opened during preflight or replaced with regular files. A pipe write they are not opened during preflight or replaced with regular files. A pipe write

22
docs/oss/attribution.md
View File

@@ -142,7 +142,7 @@ License and notices: [Apache 2.0](licenses/Apache-2.0.txt).
## PromptKit ## PromptKit
Selected module: `gitea.maximumdirect.net/eric/promptkit` at `v0.11.2`. Selected module: `gitea.maximumdirect.net/eric/promptkit` at `v0.12.0`.
Copyright (C) 2026 Eric Rakestraw Copyright (C) 2026 Eric Rakestraw
@@ -150,37 +150,37 @@ SPDX-License-Identifier: LGPL-3.0-or-later
License and notices: [LGPL v3](licenses/LGPL-3.0.txt) | License and notices: [LGPL v3](licenses/LGPL-3.0.txt) |
[GPL v3](licenses/GPL-3.0.txt) | [GPL v3](licenses/GPL-3.0.txt) |
[COPYRIGHT](https://gitea.maximumdirect.net/eric/promptkit/src/commit/12d6d80024e80c552d122a076c22429bf6f2f6c2/COPYRIGHT). [COPYRIGHT](https://gitea.maximumdirect.net/eric/promptkit/src/commit/4b8169da286917fde7875425bb32b799b12c6af3/COPYRIGHT).
[Source](https://gitea.maximumdirect.net/eric/promptkit/src/commit/12d6d80024e80c552d122a076c22429bf6f2f6c2). [Source](https://gitea.maximumdirect.net/eric/promptkit/src/commit/4b8169da286917fde7875425bb32b799b12c6af3).
## Promptkit OpenRouter Catalog ## Promptkit OpenRouter Catalog
Selected module: `gitea.maximumdirect.net/eric/promptkit-backend-openrouter/v2` at `v2.20260926.0`. Selected module: `gitea.maximumdirect.net/eric/promptkit-backend-openrouter/v3` at `v3.20260930.0`.
Copyright (C) 2026 Eric Rakestraw Copyright (C) 2026 Eric Rakestraw
SPDX-License-Identifier: MIT SPDX-License-Identifier: MIT
License and notices: [MIT](licenses/MIT-Catalogs.txt) | License and notices: [MIT](licenses/MIT-Catalogs.txt) |
[COPYRIGHT](https://gitea.maximumdirect.net/eric/promptkit-backend-openrouter/src/commit/4f731d217e0bc39151c010f50fb5912706e7b6b3/COPYRIGHT) | [COPYRIGHT](https://gitea.maximumdirect.net/eric/promptkit-backend-openrouter/src/commit/f17d208cb7ad1fb043cda2266cd3011d11b40aa0/COPYRIGHT) |
[LICENSE](https://gitea.maximumdirect.net/eric/promptkit-backend-openrouter/src/commit/4f731d217e0bc39151c010f50fb5912706e7b6b3/LICENSE). [LICENSE](https://gitea.maximumdirect.net/eric/promptkit-backend-openrouter/src/commit/f17d208cb7ad1fb043cda2266cd3011d11b40aa0/LICENSE).
[Source](https://gitea.maximumdirect.net/eric/promptkit-backend-openrouter/src/commit/4f731d217e0bc39151c010f50fb5912706e7b6b3). [Source](https://gitea.maximumdirect.net/eric/promptkit-backend-openrouter/src/commit/f17d208cb7ad1fb043cda2266cd3011d11b40aa0).
## Promptkit Rakestrawhome Catalog ## Promptkit Rakestrawhome Catalog
Selected module: `gitea.maximumdirect.net/eric/promptkit-backend-rakestrawhome/v2` at `v2.20260926.0`. Selected module: `gitea.maximumdirect.net/eric/promptkit-backend-rakestrawhome/v3` at `v3.20260930.0`.
Copyright (C) 2026 Eric Rakestraw Copyright (C) 2026 Eric Rakestraw
SPDX-License-Identifier: MIT SPDX-License-Identifier: MIT
License and notices: [MIT](licenses/MIT-Catalogs.txt) | License and notices: [MIT](licenses/MIT-Catalogs.txt) |
[COPYRIGHT](https://gitea.maximumdirect.net/eric/promptkit-backend-rakestrawhome/src/commit/48918a3ad39fd99843cda94a86133b1b3881d596/COPYRIGHT) | [COPYRIGHT](https://gitea.maximumdirect.net/eric/promptkit-backend-rakestrawhome/src/commit/993443d88c451901f4c917fa00c204fd3b54d28d/COPYRIGHT) |
[LICENSE](https://gitea.maximumdirect.net/eric/promptkit-backend-rakestrawhome/src/commit/48918a3ad39fd99843cda94a86133b1b3881d596/LICENSE). [LICENSE](https://gitea.maximumdirect.net/eric/promptkit-backend-rakestrawhome/src/commit/993443d88c451901f4c917fa00c204fd3b54d28d/LICENSE).
[Source](https://gitea.maximumdirect.net/eric/promptkit-backend-rakestrawhome/src/commit/48918a3ad39fd99843cda94a86133b1b3881d596). [Source](https://gitea.maximumdirect.net/eric/promptkit-backend-rakestrawhome/src/commit/993443d88c451901f4c917fa00c204fd3b54d28d).
## yaml.v3 ## yaml.v3

8
docs/roadmap/implementation.md
View File

@@ -1,7 +1,7 @@
# Implementation Plan: PromptKit v0.12.0 Adoption # Implementation Plan: PromptKit v0.12.0 Adoption
**Last updated:** 2026-10-04 **Last updated:** 2026-10-04
**Status:** In progress; Stages 1–7 completed. **Status:** In progress; Stages 1–8 completed.
**Audience:** Luna coding agent. **Audience:** Luna coding agent.
## Scope and Execution ## Scope and Execution
@@ -461,6 +461,12 @@ focused wire, timeout, capacity, credential, and safe-view tests; `git diff
examples against the implemented loader, verify source revisions/notices, and examples against the implemented loader, verify source revisions/notices, and
run `git diff --check`. Commit the documentation and attribution unit. run `git diff --check`. Commit the documentation and attribution unit.
**Completion marker:** Stage 8 complete. Configuration, runtime, prompt,
integration, operations, contributor, and README guidance now reflects native
PromptKit layering, migration, presence, snapshot, and separate example-root
behavior. Attribution matches PromptKit v0.12.0 and both v3 catalog module
origins and notices. Local links/anchors and `git diff --check` passed.
## Stage 9 — Validate the Integrated End State and Close the Plan ## Stage 9 — Validate the Integrated End State and Close the Plan
**Primary files:** This plan and the feature roadmap; correct an implementation **Primary files:** This plan and the feature roadmap; correct an implementation