Align prompts and strengthen validation context

This commit is contained in:
2026-09-25 12:23:10 +00:00
parent 3eb73e70fc
commit 325c4ea7d3
23 changed files with 329 additions and 133 deletions

View File

@@ -1,6 +1,6 @@
# Audita Prompt Registry
`internal/prompts` embeds nine PromptKit YAML definitions under
`internal/prompts` embeds seven PromptKit YAML definitions under
`assets/prompts`. The IDs and selected version remain:
- `modules.glossary.proposal` (`v1`)
@@ -10,17 +10,15 @@
- `validators.spoken_form_plausibility` (`v1`)
- `validators.meaning_reversal_review` (`v1`)
- `validators.editorial_review` (`v1`)
- `validators.grammar_review` (`v1`)
- `validators.spoken_word_review` (`v1`)
PromptKit reads the embedded definitions and renders their system and user
messages from named string variables. The role-bound adapter will supply
messages from named string variables. The role-bound adapter supplies
`HardeningPolicy` from the single embedded `assets/shared/prompt_hardening.md`
fragment after caller variables, so callers cannot replace that policy. Proposal prompts receive the
section JSON, glossary JSON where used, and transcript description block.
Validator prompts receive the correction payload and description block.
The prompt assets retain the prior task instructions and output-schema
association. The role-bound runtime will supply the complete JSON Schema output contract
association. The role-bound runtime supplies the complete JSON Schema output contract
and repair count; the definition does not own an operator-facing repair default.
The correction-set schema requires a string replacement and accepts an empty
string. Proposal guidance uses it for eligible span deletion and permits literal
@@ -41,3 +39,13 @@ response-schema key, and logical-call attribution. Module and validator callers
pass these values through the role-bound adapter. See the
[LLM runtime](llm-runtime.md) for runtime assembly and
[configuration](../config.md) for operator-selected profiles.
Proposal instructions describe literal matching, each module's replacement policy,
source-based proposal identity, and confidence in whether an edit is warranted.
Validator instructions distinguish correction indices from segment IDs and confidence
in the verdict from the verdict itself. The grammar article exception and glossary
mishearing exception apply consistently across their instructions.
Only the three active validator definitions are retained. The unused internal
`grammar_review` and `spoken_word_review` definitions and type selections were
removed; built-in chains and operator configuration are unchanged.

View File

@@ -107,3 +107,34 @@ is covered by [report assembly](diagnostics-reporting.md).
- `internal/framework/validators/*_test.go`
- `internal/framework/processreport/correction_ledger_test.go`
- `internal/cli/run_test.go`
## Supplemental Validation Context
Each candidate includes the immediately preceding and following source segments
available within its fixed chunk, plus glossary entries whose names, aliases, or
recognized plurals appear in its original/resulting text or those neighbors.
Glossary matching reuses the protected vocabulary; entries retain glossary order.
This supplies evidence such as a troupe definition for a proposed `Jesters`
correction or an adjacent speaker turn explaining deliberate repetition. A correction
already clear from its own segment does not require such evidence. No audio,
out-of-chunk context, or complete transcript is implied.
Context is serialized inside the validation item and included in the existing
payload token estimate. If one enriched item exceeds the configured limit, optional
neighbors are removed from the end first, then glossary entries from the end,
until it fits or only required candidate data remains. Required before/after text
is never truncated. Existing oversized-candidate handling still applies if that
required data does not fit. Context is supplemental, not a promise that every
relevant definition or neighboring utterance always fits. This uses the existing
payload estimator, not a new guarantee about the provider's full context window.
## Protected-Name Repetition
Spoken-word cleanup may reduce repeated protected-name occurrences when every
original protected identity still appears in the resulting segment and surviving
spellings satisfy existing capitalization checks. Full-segment preview supports
both a replacement of the repeated phrase and deletion of a redundant span. This
exception only relaxes the deterministic count guard; editorial and meaning-reversal
validation still decide whether the repetition was accidental and the edit preserves
meaning. Intentional emphasis and substantive references remain protected by those
reviews. Other modules retain their existing protection rules.

View File

@@ -4,7 +4,7 @@
**Scope:** All four proposal prompts, three active LLM-validator prompts, two
inactive validator definitions, shared hardening text, response schemas, supplied
context, and downstream validation/application behavior.
**Status:** R16–R26 open. R1–R15 are resolved; their findings and resolution records
**Status:** R16–R26 addressed; see the resolution record below. R1–R15 are resolved; their findings and resolution records
remain in Git history through `b8089f4`.
## Architecture Assessment
@@ -180,8 +180,8 @@ a new literal-token rule or a guarantee of model accuracy.
**Category:** Optional maintenance cleanup.
**Evidence:** Registered [grammar_review](../../internal/prompts/assets/prompts/validators.grammar_review.yaml)
and [spoken_word_review](../../internal/prompts/assets/prompts/validators.spoken_word_review.yaml)
**Evidence:** Registered `grammar_review`
and `spoken_word_review`
definitions largely duplicate editorial review and have already diverged from its
updated deletion guidance. Neither is used by the [built-in chains](../../internal/validators/chains.go).
@@ -244,3 +244,30 @@ introduced.
framework solely to implement wording improvements.
- This document catalogs observations and recommendations only; no prompts,
application behavior, or release artifacts were changed by this update.
## Resolution Record
Implemented after the maintainer requested addressing all findings. The maintainer
explicitly selected allowing protected-name repetition cleanup with validation for
R21. The original evidence above describes the audited baseline, not current defects.
| Finding | Resolution and evidence |
| --- | --- |
| R16 | Grammar system and detailed rules consistently state the whole-word a/an exception. |
| R17 | Glossary guidance distinguishes correct protected spelling from a supported mishearing of another recognized term. |
| R18 | All proposal prompts explain literal case-sensitive matching and preserving disambiguating context; glossary explicitly describes replacing every matching occurrence. |
| R19 | Proposal prompts describe multiple source-based corrections per segment and the corrections envelope. Active validators require unchanged correction indices, including negative decisions, and the validations envelope. |
| R20 | Validators receive adjacent in-chunk source segments and mentioned glossary definitions. Optional context is reduced to fit the existing payload budget before required candidate data is considered oversized. Context selection and limits are documented in the validator internals. |
| R21 | Spoken-word protected-term checking permits reduced repetition only with surviving protected identities and existing capitalization checks. The user-approved exception leaves subsequent editorial and meaning validation in place; focused tests cover phrase replacement, literal deletion, last-reference removal, identity substitution, spelling degradation, and unchanged grammar protection. |
| R22 | Spoken-form guidance explicitly evaluates full before/after text for every correction. |
| R23 | Meaning-reversal guidance includes deletion of negation, with contextual judgment rather than a token-based prohibition. |
| R24 | Consumer inspection found only unused internal selections and their tests. Removed both inactive definitions, type selections, metadata entries, and associated cases; updated the seven-prompt inventory. No operator-selectable setting or built-in chain changed. |
| R25 | Proposal confidence describes whether the correction is warranted; validator confidence describes confidence in either verdict. Runtime thresholds are unchanged. |
| R26 | Hardening describes evidence-supported correction of source material, and grammar no longer promises a later pipeline stage. |
Validation: the complete `go test ./...` suite passed, including embedded prompt
preparation through PromptKit. New context tests verify supplied neighboring evidence,
relevant glossary summaries, chunk boundaries, serialized payloads, and budget effects.
The existing oversized-input regression confirms that a large neighbor does not
invalidate another otherwise usable candidate. Documentation links and whitespace
checks passed. No live model quality claims or release publication are implied.