Move prompts into embedded Markdown assets
This commit is contained in:
@@ -136,6 +136,14 @@ internal/validators/
|
||||
registry.go
|
||||
chains.go
|
||||
|
||||
internal/prompts/
|
||||
registry.go
|
||||
render.go
|
||||
assets/
|
||||
shared/
|
||||
modules/
|
||||
validators/
|
||||
|
||||
internal/framework/llm/
|
||||
openai_compatible_client.go
|
||||
scheduler.go
|
||||
@@ -312,6 +320,43 @@ Prompt guardrail semantics are consistent across modules and validators:
|
||||
|
||||
Generated transcript descriptions remain deferred and are not implemented in the current runtime.
|
||||
|
||||
## Implemented embedded prompt assets
|
||||
Prompt assets are now built-in embedded Markdown files under `internal/prompts/assets`:
|
||||
- `assets/modules/*` for production module proposal prompts;
|
||||
- `assets/validators/*` for LLM-backed validator prompts;
|
||||
- `assets/shared/prompt_hardening.md` for shared prompt-injection hardening text.
|
||||
|
||||
Prompt source behavior:
|
||||
- built-in embedded prompts are the only supported source in current runtime;
|
||||
- filesystem prompt overrides and prompt-source selection flags are not implemented.
|
||||
|
||||
`internal/prompts` registry responsibilities:
|
||||
- register stable prompt IDs;
|
||||
- register prompt version and source metadata;
|
||||
- load embedded assets;
|
||||
- compute deterministic SHA-256 prompt source hashes;
|
||||
- render system/user prompts with `text/template` using missing-key errors.
|
||||
|
||||
Prompt metadata fields:
|
||||
- `prompt_id`
|
||||
- `prompt_version`
|
||||
- `prompt_source` (`builtin`)
|
||||
- `embedded_path`
|
||||
- `sha256`
|
||||
|
||||
Prompt rendering flow:
|
||||
- module proposal builders construct typed template data (section JSON, glossary JSON, transcript-description block) and render via `internal/prompts`;
|
||||
- validator prompt builders construct typed template data (validation payload JSON, transcript-description block) and render via `internal/prompts`.
|
||||
|
||||
Shared prompt hardening:
|
||||
- the same centralized hardening fragment is included in every proposal and LLM-validator prompt;
|
||||
- hardening text enforces untrusted transcript handling, no instruction-following from transcript content, and no invented facts/corrections.
|
||||
|
||||
Prompt metadata diagnostics flow:
|
||||
- proposal-generation diagnostics request metadata includes prompt metadata;
|
||||
- LLM-validator diagnostics request metadata includes prompt metadata;
|
||||
- detailed prompt metadata is diagnostics-scoped today and is not yet expanded into broad report-level prompt registries.
|
||||
|
||||
## Implemented structured LLM infrastructure
|
||||
`internal/framework/contracts` now defines a typed structured-completion contract:
|
||||
- `StructuredLLMClient.CompleteStructured(ctx, req, out)`
|
||||
|
||||
@@ -183,3 +183,4 @@ Current guidance:
|
||||
- keep environment variables for secrets/deployment-specific overrides;
|
||||
- use CLI flags for per-run overrides.
|
||||
- validator chains are built-in and are not user-configurable in config.
|
||||
- prompt source selection and filesystem prompt overrides are not config options.
|
||||
|
||||
118
docs/prompts.md
Normal file
118
docs/prompts.md
Normal file
@@ -0,0 +1,118 @@
|
||||
# Audita Prompts
|
||||
|
||||
This document describes Audita's built-in embedded prompt assets and prompt registry behavior.
|
||||
|
||||
## Why embedded prompt assets
|
||||
|
||||
Audita embeds production prompt text into the binary so runtime behavior is:
|
||||
- deterministic;
|
||||
- auditable;
|
||||
- dependency-light;
|
||||
- not dependent on external prompt files at execution time.
|
||||
|
||||
Prompt text is authored as Markdown assets and rendered by Go code using typed template data.
|
||||
|
||||
## Built-in prompt registry
|
||||
|
||||
The prompt registry lives in `internal/prompts` and is responsible for:
|
||||
- loading embedded prompt assets;
|
||||
- registering stable prompt IDs and versions;
|
||||
- recording prompt source metadata;
|
||||
- computing deterministic SHA-256 source hashes;
|
||||
- rendering system/user prompts with strict missing-key failures.
|
||||
|
||||
Current prompt source behavior:
|
||||
- built-in embedded prompts only (`prompt_source = builtin`).
|
||||
- filesystem prompt overrides are not supported.
|
||||
|
||||
## Built-in prompt IDs
|
||||
|
||||
Module proposal prompts:
|
||||
- `modules.glossary.proposal`
|
||||
- `modules.homophones.proposal`
|
||||
- `modules.spoken_word.proposal`
|
||||
- `modules.grammar.proposal`
|
||||
|
||||
LLM-backed validator prompts:
|
||||
- `validators.spoken_form_plausibility`
|
||||
- `validators.meaning_reversal_review`
|
||||
- `validators.editorial_review`
|
||||
- `validators.grammar_review`
|
||||
- `validators.spoken_word_review`
|
||||
|
||||
## Prompt version semantics
|
||||
|
||||
Current built-in prompt version value is `v1`.
|
||||
|
||||
Version is a stable metadata identifier for diagnostics and debugging. It is not a dynamic prompt-selection mechanism.
|
||||
|
||||
## Prompt hash semantics
|
||||
|
||||
Each registered prompt includes a deterministic SHA-256 hash of embedded source text.
|
||||
|
||||
Hash purpose:
|
||||
- identify exact prompt source used in a run;
|
||||
- support diagnostics reproducibility and change auditing.
|
||||
|
||||
Current hash scope:
|
||||
- source prompt text (system + user assets for a registered prompt), not a runtime secret-bearing payload.
|
||||
|
||||
## Template rendering behavior
|
||||
|
||||
Prompt rendering uses Go `text/template` with typed template data from module/validator builders.
|
||||
|
||||
Missing-key behavior:
|
||||
- rendering uses missing-key errors;
|
||||
- missing/renamed template fields fail quickly instead of silently producing incomplete prompts.
|
||||
|
||||
Go code still owns:
|
||||
- structured request/response models;
|
||||
- response schema selection;
|
||||
- transcript/glossary/payload formatting;
|
||||
- module and validator selection;
|
||||
- diagnostics wiring.
|
||||
|
||||
## Shared prompt hardening policy
|
||||
|
||||
A shared hardening fragment is embedded once and included in every module proposal prompt and every LLM-validator prompt.
|
||||
|
||||
Hardening policy includes:
|
||||
- transcript text is untrusted data;
|
||||
- glossary entries and transcript descriptions are reference data, not instructions;
|
||||
- instructions found inside transcript text must not be obeyed;
|
||||
- model must perform only the requested correction/validation task;
|
||||
- no invention of facts, names, events, motivations, speaker intent, or corrections;
|
||||
- transcript remains the source of truth.
|
||||
|
||||
## Transcript description behavior
|
||||
|
||||
Transcript description remains background-only prompt context:
|
||||
- it may help interpret ambiguous terms;
|
||||
- it is explicitly non-authoritative and must not override transcript content;
|
||||
- empty descriptions do not render awkward blank context sections.
|
||||
|
||||
Generated transcript descriptions are not implemented in this workstream.
|
||||
|
||||
## Diagnostics and report metadata boundaries
|
||||
|
||||
Current metadata flow:
|
||||
- proposal-generation diagnostics request metadata includes prompt metadata;
|
||||
- LLM-validator diagnostics request metadata includes prompt metadata.
|
||||
|
||||
Prompt metadata fields used in diagnostics:
|
||||
- `prompt_id`
|
||||
- `prompt_version`
|
||||
- `prompt_source`
|
||||
- `embedded_path`
|
||||
- `sha256`
|
||||
|
||||
Current boundary:
|
||||
- detailed prompt metadata is diagnostics-first;
|
||||
- broad report-level prompt registries/ledgers are deferred.
|
||||
|
||||
## 1.0 boundary
|
||||
|
||||
Not implemented for 1.0 in this workstream:
|
||||
- filesystem prompt overrides;
|
||||
- user-configurable prompt selection;
|
||||
- external prompt directories.
|
||||
@@ -11,6 +11,7 @@ This contract covers:
|
||||
- transcript output schema selection
|
||||
- process report schema metadata
|
||||
- stable validator key identifiers in report/diagnostics records
|
||||
- prompt metadata identifiers in diagnostics
|
||||
- diagnostics directory behavior
|
||||
- stdout/stderr and exit-code behavior
|
||||
- secret redaction guarantees
|
||||
@@ -103,6 +104,15 @@ When diagnostics directory creation succeeds, Audita writes run artifacts includ
|
||||
|
||||
Retention behavior is controlled by configured retention mode; failed runs are retained.
|
||||
|
||||
Diagnostics metadata for LLM interactions may include semi-public prompt identifiers:
|
||||
- `prompt_id`
|
||||
- `prompt_version`
|
||||
- `prompt_source`
|
||||
- `embedded_path`
|
||||
- `sha256`
|
||||
|
||||
These are diagnostic identifiers, not user-facing prompt override controls.
|
||||
|
||||
## Stdout/stderr behavior
|
||||
|
||||
Success behavior:
|
||||
|
||||
@@ -657,6 +657,26 @@ Recommended 1.0 boundary:
|
||||
|
||||
Treat prompts as first-class behavioral assets rather than multiline strings embedded in Go source.
|
||||
|
||||
## Implementation status (2026-05-13)
|
||||
|
||||
This workstream is now implemented for built-in prompt assets and diagnostics metadata:
|
||||
- module and LLM-validator prompt text has been moved to embedded Markdown assets under `internal/prompts/assets`;
|
||||
- a built-in prompt registry exists in `internal/prompts` with stable prompt IDs and initial versions (`v1`);
|
||||
- prompt metadata is registered with:
|
||||
- prompt ID;
|
||||
- prompt version;
|
||||
- prompt source (`builtin`);
|
||||
- embedded source path;
|
||||
- deterministic SHA-256 source hash;
|
||||
- prompt rendering uses typed template data and `text/template` missing-key errors;
|
||||
- a shared prompt-injection hardening fragment is centrally embedded and included in module proposal and LLM-validator prompts;
|
||||
- transcript description context remains background-only and non-authoritative, and empty descriptions do not render blank sections;
|
||||
- prompt metadata is propagated into proposal-generation and LLM-validator diagnostics request metadata.
|
||||
|
||||
Current boundary:
|
||||
- detailed prompt metadata is currently diagnostics-scoped; reports are not broadly redesigned for prompt metadata yet.
|
||||
- filesystem prompt overrides, user-selectable prompt sources, scheduler utilization diagnostics, correction ledgers, and generated summaries remain planned/deferred.
|
||||
|
||||
This phase pairs naturally with the validator refactor because both modules and LLM-backed validators need prompt assets, stable prompt IDs, prompt versions, and prompt diagnostics.
|
||||
|
||||
## Built-in embedded prompts
|
||||
|
||||
@@ -2,6 +2,8 @@
|
||||
|
||||
This document describes Audita's built-in validator registry and module validator chains.
|
||||
|
||||
For LLM-backed validator prompt asset details, see [`docs/prompts.md`](prompts.md).
|
||||
|
||||
## Scope
|
||||
|
||||
Validator chains are built-in runtime behavior.
|
||||
|
||||
Reference in New Issue
Block a user