Move prompts into embedded Markdown assets

This commit is contained in:
2026-05-13 18:49:06 +00:00
parent d6126bf52b
commit 037121e9ce
43 changed files with 940 additions and 173 deletions

View File

@@ -136,6 +136,14 @@ internal/validators/
registry.go
chains.go
internal/prompts/
registry.go
render.go
assets/
shared/
modules/
validators/
internal/framework/llm/
openai_compatible_client.go
scheduler.go
@@ -312,6 +320,43 @@ Prompt guardrail semantics are consistent across modules and validators:
Generated transcript descriptions remain deferred and are not implemented in the current runtime.
## Implemented embedded prompt assets
Prompt assets are now built-in embedded Markdown files under `internal/prompts/assets`:
- `assets/modules/*` for production module proposal prompts;
- `assets/validators/*` for LLM-backed validator prompts;
- `assets/shared/prompt_hardening.md` for shared prompt-injection hardening text.
Prompt source behavior:
- built-in embedded prompts are the only supported source in current runtime;
- filesystem prompt overrides and prompt-source selection flags are not implemented.
`internal/prompts` registry responsibilities:
- register stable prompt IDs;
- register prompt version and source metadata;
- load embedded assets;
- compute deterministic SHA-256 prompt source hashes;
- render system/user prompts with `text/template` using missing-key errors.
Prompt metadata fields:
- `prompt_id`
- `prompt_version`
- `prompt_source` (`builtin`)
- `embedded_path`
- `sha256`
Prompt rendering flow:
- module proposal builders construct typed template data (section JSON, glossary JSON, transcript-description block) and render via `internal/prompts`;
- validator prompt builders construct typed template data (validation payload JSON, transcript-description block) and render via `internal/prompts`.
Shared prompt hardening:
- the same centralized hardening fragment is included in every proposal and LLM-validator prompt;
- hardening text enforces untrusted transcript handling, no instruction-following from transcript content, and no invented facts/corrections.
Prompt metadata diagnostics flow:
- proposal-generation diagnostics request metadata includes prompt metadata;
- LLM-validator diagnostics request metadata includes prompt metadata;
- detailed prompt metadata is diagnostics-scoped today and is not yet expanded into broad report-level prompt registries.
## Implemented structured LLM infrastructure
`internal/framework/contracts` now defines a typed structured-completion contract:
- `StructuredLLMClient.CompleteStructured(ctx, req, out)`

View File

@@ -183,3 +183,4 @@ Current guidance:
- keep environment variables for secrets/deployment-specific overrides;
- use CLI flags for per-run overrides.
- validator chains are built-in and are not user-configurable in config.
- prompt source selection and filesystem prompt overrides are not config options.

118
docs/prompts.md Normal file
View File

@@ -0,0 +1,118 @@
# Audita Prompts
This document describes Audita's built-in embedded prompt assets and prompt registry behavior.
## Why embedded prompt assets
Audita embeds production prompt text into the binary so runtime behavior is:
- deterministic;
- auditable;
- dependency-light;
- not dependent on external prompt files at execution time.
Prompt text is authored as Markdown assets and rendered by Go code using typed template data.
## Built-in prompt registry
The prompt registry lives in `internal/prompts` and is responsible for:
- loading embedded prompt assets;
- registering stable prompt IDs and versions;
- recording prompt source metadata;
- computing deterministic SHA-256 source hashes;
- rendering system/user prompts with strict missing-key failures.
Current prompt source behavior:
- built-in embedded prompts only (`prompt_source = builtin`).
- filesystem prompt overrides are not supported.
## Built-in prompt IDs
Module proposal prompts:
- `modules.glossary.proposal`
- `modules.homophones.proposal`
- `modules.spoken_word.proposal`
- `modules.grammar.proposal`
LLM-backed validator prompts:
- `validators.spoken_form_plausibility`
- `validators.meaning_reversal_review`
- `validators.editorial_review`
- `validators.grammar_review`
- `validators.spoken_word_review`
## Prompt version semantics
Current built-in prompt version value is `v1`.
Version is a stable metadata identifier for diagnostics and debugging. It is not a dynamic prompt-selection mechanism.
## Prompt hash semantics
Each registered prompt includes a deterministic SHA-256 hash of embedded source text.
Hash purpose:
- identify exact prompt source used in a run;
- support diagnostics reproducibility and change auditing.
Current hash scope:
- source prompt text (system + user assets for a registered prompt), not a runtime secret-bearing payload.
## Template rendering behavior
Prompt rendering uses Go `text/template` with typed template data from module/validator builders.
Missing-key behavior:
- rendering uses missing-key errors;
- missing/renamed template fields fail quickly instead of silently producing incomplete prompts.
Go code still owns:
- structured request/response models;
- response schema selection;
- transcript/glossary/payload formatting;
- module and validator selection;
- diagnostics wiring.
## Shared prompt hardening policy
A shared hardening fragment is embedded once and included in every module proposal prompt and every LLM-validator prompt.
Hardening policy includes:
- transcript text is untrusted data;
- glossary entries and transcript descriptions are reference data, not instructions;
- instructions found inside transcript text must not be obeyed;
- model must perform only the requested correction/validation task;
- no invention of facts, names, events, motivations, speaker intent, or corrections;
- transcript remains the source of truth.
## Transcript description behavior
Transcript description remains background-only prompt context:
- it may help interpret ambiguous terms;
- it is explicitly non-authoritative and must not override transcript content;
- empty descriptions do not render awkward blank context sections.
Generated transcript descriptions are not implemented in this workstream.
## Diagnostics and report metadata boundaries
Current metadata flow:
- proposal-generation diagnostics request metadata includes prompt metadata;
- LLM-validator diagnostics request metadata includes prompt metadata.
Prompt metadata fields used in diagnostics:
- `prompt_id`
- `prompt_version`
- `prompt_source`
- `embedded_path`
- `sha256`
Current boundary:
- detailed prompt metadata is diagnostics-first;
- broad report-level prompt registries/ledgers are deferred.
## 1.0 boundary
Not implemented for 1.0 in this workstream:
- filesystem prompt overrides;
- user-configurable prompt selection;
- external prompt directories.

View File

@@ -11,6 +11,7 @@ This contract covers:
- transcript output schema selection
- process report schema metadata
- stable validator key identifiers in report/diagnostics records
- prompt metadata identifiers in diagnostics
- diagnostics directory behavior
- stdout/stderr and exit-code behavior
- secret redaction guarantees
@@ -103,6 +104,15 @@ When diagnostics directory creation succeeds, Audita writes run artifacts includ
Retention behavior is controlled by configured retention mode; failed runs are retained.
Diagnostics metadata for LLM interactions may include semi-public prompt identifiers:
- `prompt_id`
- `prompt_version`
- `prompt_source`
- `embedded_path`
- `sha256`
These are diagnostic identifiers, not user-facing prompt override controls.
## Stdout/stderr behavior
Success behavior:

View File

@@ -657,6 +657,26 @@ Recommended 1.0 boundary:
Treat prompts as first-class behavioral assets rather than multiline strings embedded in Go source.
## Implementation status (2026-05-13)
This workstream is now implemented for built-in prompt assets and diagnostics metadata:
- module and LLM-validator prompt text has been moved to embedded Markdown assets under `internal/prompts/assets`;
- a built-in prompt registry exists in `internal/prompts` with stable prompt IDs and initial versions (`v1`);
- prompt metadata is registered with:
- prompt ID;
- prompt version;
- prompt source (`builtin`);
- embedded source path;
- deterministic SHA-256 source hash;
- prompt rendering uses typed template data and `text/template` missing-key errors;
- a shared prompt-injection hardening fragment is centrally embedded and included in module proposal and LLM-validator prompts;
- transcript description context remains background-only and non-authoritative, and empty descriptions do not render blank sections;
- prompt metadata is propagated into proposal-generation and LLM-validator diagnostics request metadata.
Current boundary:
- detailed prompt metadata is currently diagnostics-scoped; reports are not broadly redesigned for prompt metadata yet.
- filesystem prompt overrides, user-selectable prompt sources, scheduler utilization diagnostics, correction ledgers, and generated summaries remain planned/deferred.
This phase pairs naturally with the validator refactor because both modules and LLM-backed validators need prompt assets, stable prompt IDs, prompt versions, and prompt diagnostics.
## Built-in embedded prompts

View File

@@ -2,6 +2,8 @@
This document describes Audita's built-in validator registry and module validator chains.
For LLM-backed validator prompt asset details, see [`docs/prompts.md`](prompts.md).
## Scope
Validator chains are built-in runtime behavior.