diff --git a/README.md b/README.md index f06cb93..82a6dd9 100644 --- a/README.md +++ b/README.md @@ -40,6 +40,7 @@ CLI help: ```sh audita --help audita process --help +audita config --help ``` ## Test @@ -65,6 +66,16 @@ audita process transcript.json \ --report-json report.json ``` +Recommended config-based run: + +```sh +audita process transcript.json \ + --glossary glossary.yaml \ + --config audita.yml \ + --output corrected.json \ + --report-json report.json +``` + Explicit module override: ```sh @@ -116,8 +127,18 @@ For subprocess orchestration guidance, see [`docs/subprocess-operations.md`](doc Precedence: 1. defaults -2. environment (`AUDITA_*`) -3. CLI flags +2. config file (`--config`, `AUDITA_CONFIG`, or `/etc/audita/config.yml` when present) +3. environment (`AUDITA_*`) +4. CLI flags + +Config commands: + +```sh +audita config validate --config audita.yml +audita config print-effective --config audita.yml +``` + +For full config-file schema and examples, see [`docs/configuration.md`](docs/configuration.md). ### Modules diff --git a/docs/architecture.md b/docs/architecture.md index fb9421e..4997ef9 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -140,14 +140,20 @@ internal/framework/llm/ ``` ## Current CLI behavior -Primary command: +Primary commands: ```sh audita process --glossary [flags] +audita config validate --config +audita config print-effective [--config ] ``` Current runtime flow (`internal/cli/run.go`): -1. Load config from env. +1. Build runtime config from: + - defaults; + - file config source (`--config`, `AUDITA_CONFIG`, or `/etc/audita/config.yml` when present); + - environment overrides; + - CLI overrides. 2. Parse flags and apply CLI overrides. 3. Validate transcript positional argument and required `--glossary`. 4. Create per-run diagnostics directory. @@ -168,6 +174,15 @@ Current runtime flow (`internal/cli/run.go`): 15. Optionally write `--report-json`; always write run-dir `report.json`. 16. Apply work-dir retention. +Config command behavior (`internal/cli/run.go`): +- `audita config validate --config `: + - loads and validates a versioned YAML config file; + - does not require transcript or glossary inputs. +- `audita config print-effective [--config ]`: + - builds effective config from defaults + file config + env overrides; + - prints redacted JSON to stdout; + - does not require transcript or glossary inputs. + Parity fixture status: - representative Python-parity fixture coverage exists under `internal/cli/testdata/parity`; - parity tests use fake structured LLM responses for deterministic behavior, including default full-pipeline shape assertions; @@ -212,10 +227,34 @@ Optional: - `aliases`, `plural` ## Implemented config/env/flag behavior -Precedence: +Precedence for `audita process`: 1. defaults (`config.Default()`) -2. environment (`config.LoadFromEnv()`) -3. CLI flags (`ApplyCLIOverrides`) +2. config file (if resolved from `--config`, `AUDITA_CONFIG`, or default path) +3. environment overrides +4. CLI flags (`ApplyCLIOverrides`) + +File-config source behavior: +- explicit `--config `: + - required to exist, otherwise process fails clearly. +- `AUDITA_CONFIG` (when `--config` is not provided): + - required to exist, otherwise process fails clearly. +- default path `/etc/audita/config.yml` (when neither explicit source is provided): + - used only when present; + - silently ignored when missing. + +Versioned file-config behavior (`internal/core/config/file_config.go`): +- supported version: `version: 1`; +- missing version fails; +- unsupported version fails; +- strict unknown-field rejection is enabled. + +`api_key_env` behavior: +- file config can declare API key environment variable names for proposal/validation LLM settings; +- runtime resolves those names from the process environment during config application; +- no direct API-key value field is supported in file config. + +Redaction behavior: +- effective config artifacts and `audita config print-effective` both use the same redaction path (`Config.Redacted()`), so API keys are not emitted in plaintext. Implemented config surfaces include: - module list @@ -229,6 +268,7 @@ Implemented config surfaces include: Current caveat: - LLM/module-related settings are active for default and explicit module-run paths. +- compatibility environment variables and lower-level CLI tuning flags remain available while the preferred config-driven surface is adopted. Transcript description behavior: - `--transcript-description` is a process-flag input for optional user-supplied background context. diff --git a/docs/configuration.md b/docs/configuration.md new file mode 100644 index 0000000..bc99c93 --- /dev/null +++ b/docs/configuration.md @@ -0,0 +1,175 @@ +# Audita Configuration + +This document describes Audita's versioned YAML config support and related commands. + +## Purpose + +Audita's config file provides a stable place for pipeline defaults and runtime tuning that would otherwise require many environment variables or CLI flags. + +Use config files for baseline settings, then use environment variables and CLI flags for deployment and per-run overrides. + +## Supported version + +Current supported config version: + +- `version: 1` + +Rules: + +- missing `version` fails validation; +- unknown versions fail validation; +- unknown fields fail validation (strict decoding). + +## Config path resolution + +For `audita process`, config path resolution is: + +1. `--config ` if provided +2. `AUDITA_CONFIG` if set and `--config` is not provided +3. default `/etc/audita/config.yml` if present + +Missing-file behavior: + +- missing `--config` path: hard failure; +- missing `AUDITA_CONFIG` path: hard failure; +- missing `/etc/audita/config.yml`: non-fatal, run continues. + +## Precedence model + +Effective config precedence is: + +1. built-in defaults +2. file config +3. environment overrides +4. CLI overrides + +## Supported YAML fields + +```yaml +version: 1 + +pipeline: + modules: [glossary, homophones, glossary, spoken_word, grammar] + +llm: + proposal: + base_url: https://openrouter.ai/api/v1 + model: openrouter/google/gemma-4-31b-it + api_key_env: AUDITA_LLM_API_KEY + timeout: 120s + max_retries: 3 + + validation: + base_url: https://openrouter.ai/api/v1 + model: openrouter/google/gemma-4-31b-it + api_key_env: AUDITA_VALIDATION_LLM_API_KEY + timeout: 120s + max_retries: 3 + +concurrency: + total_llm: 2 + proposal_llm: 2 + validation_llm: 1 + +chunking: + target_sections: 8 + max_section_tokens: 8192 + min_section_tokens: 2048 + +normalization: + max_segment_gap: 4s + ellipsis_gap: 3.5s + max_segment_duration: 60s + max_segment_tokens: 2048 + +thresholds: + glossary: 0.8 + homophones: 0.8 + spoken_word: 0.8 + grammar: 0.8 + +context: + description: "optional transcript background context" + +diagnostics: + work_dir: /tmp/audita + retention: auto +``` + +Duration-like fields accept either: + +- numeric seconds (for example `120`, `3.5`), or +- duration strings (for example `120s`, `2m`). + +For LLM timeouts, duration strings must resolve to whole seconds. + +## Secret handling + +Use `api_key_env` for secrets: + +- `llm.proposal.api_key_env` +- `llm.validation.api_key_env` + +These fields must contain environment variable names, not secret values. + +At runtime, Audita resolves those names from the process environment. + +Redaction behavior: + +- run diagnostics `effective-config.json` is redacted; +- `audita config print-effective` output is redacted; +- API keys are never emitted in plaintext by those outputs. + +## Config commands + +Validate a config file: + +```sh +audita config validate --config ./audita.yml +``` + +Print redacted effective config: + +```sh +audita config print-effective --config ./audita.yml +``` + +`print-effective` loads defaults, then file config, then environment overrides. + +## Example: local OpenAI-compatible endpoint + +```yaml +version: 1 + +llm: + proposal: + base_url: http://localhost:8000/v1 + model: local/proposal-model + api_key_env: AUDITA_LLM_API_KEY + timeout: 90s + max_retries: 2 + + validation: + base_url: http://localhost:8000/v1 + model: local/validation-model + api_key_env: AUDITA_VALIDATION_LLM_API_KEY + timeout: 90s + max_retries: 2 + +pipeline: + modules: [glossary, homophones, glossary, spoken_word, grammar] + +diagnostics: + work_dir: /tmp/audita + retention: auto +``` + +## Compatibility notes + +Existing environment variables and lower-level CLI flags remain available for compatibility. + +Current guidance: + +- prefer file config for baseline behavior; +- keep environment variables for secrets/deployment-specific overrides; +- use CLI flags for per-run overrides. diff --git a/docs/roadmap.md b/docs/roadmap.md index fea25b0..016c91f 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -205,6 +205,25 @@ Update architecture documentation to explain: Introduce a versioned configuration file and clarify which settings are stable CLI flags, which settings belong in config, and which settings should remain environment-only. +## Implementation status (2026-05-13) + +This workstream is now implemented for runtime loading and basic command surface: +- versioned YAML file config with strict unknown-field rejection and `version: 1` validation; +- runtime config-source behavior for `audita process`: + - defaults; + - config file (`--config`, then `AUDITA_CONFIG`, then `/etc/audita/config.yml` if present); + - environment overrides; + - CLI overrides; +- explicit missing-file errors for `--config` and `AUDITA_CONFIG`, with non-fatal missing default-path behavior; +- `api_key_env` support in file config for proposal and validation LLM credentials; +- config command surface: + - `audita config validate --config ` + - `audita config print-effective [--config ]` +- redacted effective-config behavior preserved across diagnostics and config printing; +- compatibility environment variables and lower-level process flags remain available. + +This status update applies only to versioned config support and config commands. Output schema registries, broader public-contract work, validator refactors, prompt-asset registries, utilization diagnostics, and correction ledgers remain planned. + This phase should happen early because later workstreams need clean config locations for prompt registry settings, output schema selection, validator settings, diagnostics settings, and concurrency tuning. ## Configuration precedence diff --git a/docs/subprocess-operations.md b/docs/subprocess-operations.md index b17385b..f35f137 100644 --- a/docs/subprocess-operations.md +++ b/docs/subprocess-operations.md @@ -14,6 +14,7 @@ audita process \ ``` Recommended additions: +- `--config ` to select an explicit versioned config file. - `--work-dir ` to control diagnostics location. - `--work-dir-retention ` to control retained run directories. - `--total-llm-concurrency`, `--proposal-llm-concurrency`, and `--validation-llm-concurrency` when orchestration needs explicit LLM throughput controls. diff --git a/internal/cli/run.go b/internal/cli/run.go index 90c75f4..c442bfb 100644 --- a/internal/cli/run.go +++ b/internal/cli/run.go @@ -2,6 +2,7 @@ package cli import ( "context" + "encoding/json" "errors" "flag" "fmt" @@ -50,6 +51,8 @@ type processInvocation struct { OutputPath string ReportJSONPath string Config config.Config + ConfigPath string + ConfigSource string ExplicitModules bool } @@ -79,6 +82,8 @@ var processRunner = func(inv processInvocation, stdout io.Writer) (*normalizatio GlossaryPath: inv.GlossaryPath, OutputPath: inv.OutputPath, ReportJSONPath: inv.ReportJSONPath, + ConfigPath: inv.ConfigPath, + ConfigSource: inv.ConfigSource, TranscriptDescription: inv.Config.TranscriptDescription, Modules: append([]string(nil), inv.Config.Modules...), }); err != nil { @@ -340,6 +345,9 @@ func Run(args []string, stdout, stderr io.Writer) int { if args[0] == "process" { return runProcess(args[1:], stdout, stderr) } + if args[0] == "config" { + return runConfig(args[1:], stdout, stderr) + } fmt.Fprintf(stderr, "audita: unknown command %q\n\n", args[0]) writeRootUsage(stderr) @@ -349,8 +357,30 @@ func Run(args []string, stdout, stderr io.Writer) int { func runProcess(args []string, stdout, stderr io.Writer) int { startedAt := time.Now().UTC() - cfg, err := config.LoadFromEnv() + configPathOverride, configPathOverrideSet, err := findConfigPathOverride(args) if err != nil { + fmt.Fprintf(stderr, "audita process: invalid CLI configuration: %v\n", err) + return 2 + } + configPath, configSource, err := resolveConfigPath(configPathOverride, configPathOverrideSet, os.LookupEnv) + if err != nil { + fmt.Fprintf(stderr, "audita process: %v\n", err) + return 2 + } + + cfg := config.Default() + if configPath != "" { + fileCfg, fileErr := config.LoadFileConfig(configPath) + if fileErr != nil { + fmt.Fprintf(stderr, "audita process: invalid config file: %v\n", fileErr) + return 2 + } + if applyErr := cfg.ApplyFileConfig(fileCfg); applyErr != nil { + fmt.Fprintf(stderr, "audita process: invalid config file: %v\n", applyErr) + return 2 + } + } + if err := cfg.ApplyEnvOverrides(); err != nil { fmt.Fprintf(stderr, "audita process: invalid environment configuration: %v\n", err) return 2 } @@ -472,6 +502,8 @@ func runProcess(args []string, stdout, stderr io.Writer) int { OutputPath: *pFlags.outputPath, ReportJSONPath: *pFlags.reportJSONPath, Config: cfg, + ConfigPath: configPath, + ConfigSource: configSource, ExplicitModules: explicitModules, } @@ -541,6 +573,114 @@ func runProcess(args []string, stdout, stderr io.Writer) int { return 0 } +func runConfig(args []string, stdout, stderr io.Writer) int { + if len(args) == 0 || isHelpCommand(args) || hasHelpFlag(args) { + writeConfigUsage(stdout) + return 0 + } + + switch args[0] { + case "validate": + return runConfigValidate(args[1:], stdout, stderr) + case "print-effective": + return runConfigPrintEffective(args[1:], stdout, stderr) + default: + fmt.Fprintf(stderr, "audita config: unknown command %q\n\n", args[0]) + writeConfigUsage(stderr) + return 2 + } +} + +func runConfigValidate(args []string, stdout, stderr io.Writer) int { + fs := flag.NewFlagSet("config validate", flag.ContinueOnError) + fs.SetOutput(stderr) + configPath := fs.String("config", "", "Path to versioned YAML config file") + + if err := fs.Parse(args); err != nil { + if errors.Is(err, flag.ErrHelp) { + writeConfigValidateUsage(stdout) + return 0 + } + return 2 + } + if strings.TrimSpace(*configPath) == "" { + fmt.Fprintln(stderr, "audita config validate: --config is required") + return 2 + } + if len(fs.Args()) != 0 { + fmt.Fprintln(stderr, "audita config validate: unexpected positional arguments") + return 2 + } + + fileCfg, err := config.LoadFileConfig(strings.TrimSpace(*configPath)) + if err != nil { + fmt.Fprintf(stderr, "audita config validate: %v\n", err) + return 2 + } + cfg := config.Default() + if err := cfg.ApplyFileConfig(fileCfg); err != nil { + fmt.Fprintf(stderr, "audita config validate: %v\n", err) + return 2 + } + fmt.Fprintln(stdout, "config is valid") + return 0 +} + +func runConfigPrintEffective(args []string, stdout, stderr io.Writer) int { + fs := flag.NewFlagSet("config print-effective", flag.ContinueOnError) + fs.SetOutput(stderr) + configPath := fs.String("config", "", "Path to versioned YAML config file") + if err := fs.Parse(args); err != nil { + if errors.Is(err, flag.ErrHelp) { + writeConfigPrintEffectiveUsage(stdout) + return 0 + } + return 2 + } + if len(fs.Args()) != 0 { + fmt.Fprintln(stderr, "audita config print-effective: unexpected positional arguments") + return 2 + } + + configPathValue := strings.TrimSpace(*configPath) + configPathSet := configPathValue != "" + path, _, err := resolveConfigPath(configPathValue, configPathSet, os.LookupEnv) + if err != nil { + fmt.Fprintf(stderr, "audita config print-effective: %v\n", err) + return 2 + } + + cfg := config.Default() + if path != "" { + fileCfg, fileErr := config.LoadFileConfig(path) + if fileErr != nil { + fmt.Fprintf(stderr, "audita config print-effective: %v\n", fileErr) + return 2 + } + if applyErr := cfg.ApplyFileConfig(fileCfg); applyErr != nil { + fmt.Fprintf(stderr, "audita config print-effective: %v\n", applyErr) + return 2 + } + } + if err := cfg.ApplyEnvOverrides(); err != nil { + fmt.Fprintf(stderr, "audita config print-effective: %v\n", err) + return 2 + } + + redacted := cfg.Redacted() + out, err := json.MarshalIndent(redacted, "", " ") + if err != nil { + fmt.Fprintf(stderr, "audita config print-effective: %v\n", err) + return 1 + } + out = append(out, '\n') + if _, err := stdout.Write(out); err != nil { + fmt.Fprintf(stderr, "audita config print-effective: %v\n", err) + return 1 + } + return 0 +} + func extractErrorPhase(err error) (phase string, message string) { msg := err.Error() if strings.Contains(msg, ": ") { @@ -683,6 +823,7 @@ func mapValidatorRejected(in []runner.ValidatorRejectedChange) []reporting.Valid } type processFlags struct { + configPath *string glossaryPath *string outputPath *string reportJSONPath *string @@ -743,6 +884,7 @@ func newProcessFlagSet(cfg config.Config, stderr io.Writer) (*flag.FlagSet, proc } pFlags := processFlags{ + configPath: fs.String("config", "", "Path to versioned YAML config file"), glossaryPath: fs.String("glossary", "", "Path to glossary YAML file"), outputPath: fs.String("output", "", "Path to corrected transcript JSON output file"), reportJSONPath: fs.String("report-json", "", "Path to machine-readable report JSON output file"), @@ -781,6 +923,63 @@ func newProcessFlagSet(cfg config.Config, stderr io.Writer) (*flag.FlagSet, proc return fs, pFlags } +func findConfigPathOverride(args []string) (path string, set bool, err error) { + for i := 0; i < len(args); i++ { + arg := strings.TrimSpace(args[i]) + if arg == "" { + continue + } + if arg == "--config" { + if i+1 >= len(args) { + return "", false, fmt.Errorf("--config requires a path") + } + return strings.TrimSpace(args[i+1]), true, nil + } + if strings.HasPrefix(arg, "--config=") { + return strings.TrimSpace(strings.TrimPrefix(arg, "--config=")), true, nil + } + } + return "", false, nil +} + +func resolveConfigPath(cliConfigPath string, cliConfigPathSet bool, lookup func(string) (string, bool)) (path string, source string, err error) { + if cliConfigPathSet { + path = strings.TrimSpace(cliConfigPath) + if path == "" { + return "", "", fmt.Errorf("--config requires a non-empty path") + } + if _, statErr := os.Stat(path); statErr != nil { + if os.IsNotExist(statErr) { + return "", "", fmt.Errorf("config file not found: %s", path) + } + return "", "", fmt.Errorf("cannot access config file %s: %w", path, statErr) + } + return path, "flag", nil + } + + if raw, ok := lookup("AUDITA_CONFIG"); ok { + path = strings.TrimSpace(raw) + if path == "" { + return "", "", fmt.Errorf("AUDITA_CONFIG must not be empty") + } + if _, statErr := os.Stat(path); statErr != nil { + if os.IsNotExist(statErr) { + return "", "", fmt.Errorf("config file not found: %s", path) + } + return "", "", fmt.Errorf("cannot access config file %s: %w", path, statErr) + } + return path, "env", nil + } + + defaultPath := config.DefaultConfigPath + if _, statErr := os.Stat(defaultPath); statErr == nil { + return defaultPath, "default", nil + } else if !os.IsNotExist(statErr) { + return "", "", fmt.Errorf("cannot access config file %s: %w", defaultPath, statErr) + } + return "", "", nil +} + func isHelpCommand(args []string) bool { if len(args) == 0 { return false @@ -817,11 +1016,37 @@ func writeRootUsage(w io.Writer) { fmt.Fprintln(w) fmt.Fprintln(w, "Commands:") fmt.Fprintln(w, " process Process a transcript JSON file") + fmt.Fprintln(w, " config Validate and inspect config") fmt.Fprintln(w) fmt.Fprintln(w, "Example:") fmt.Fprintln(w, " audita process transcript.json --glossary glossary.yaml --output corrected.json") } +func writeConfigUsage(w io.Writer) { + fmt.Fprintln(w, "Validate and inspect Audita config.") + fmt.Fprintln(w) + fmt.Fprintln(w, "Usage:") + fmt.Fprintln(w, " audita config [flags]") + fmt.Fprintln(w) + fmt.Fprintln(w, "Commands:") + fmt.Fprintln(w, " validate Validate a versioned YAML config file") + fmt.Fprintln(w, " print-effective Print redacted effective config JSON (defaults + config file + env)") +} + +func writeConfigValidateUsage(w io.Writer) { + fmt.Fprintln(w, "Validate a versioned YAML config file.") + fmt.Fprintln(w) + fmt.Fprintln(w, "Usage:") + fmt.Fprintln(w, " audita config validate --config ") +} + +func writeConfigPrintEffectiveUsage(w io.Writer) { + fmt.Fprintln(w, "Print redacted effective config JSON.") + fmt.Fprintln(w) + fmt.Fprintln(w, "Usage:") + fmt.Fprintln(w, " audita config print-effective [--config ]") +} + func writeProcessUsage(w io.Writer, fs *flag.FlagSet) { fmt.Fprintln(w, "Process a transcript JSON file.") fmt.Fprintln(w) diff --git a/internal/cli/run_test.go b/internal/cli/run_test.go index 03573ac..944f55c 100644 --- a/internal/cli/run_test.go +++ b/internal/cli/run_test.go @@ -54,6 +54,7 @@ func TestRunProcessHelpListsExpectedFlags(t *testing.T) { } for _, expectedFlag := range []string{ + "--config", "--glossary", "--output", "--report-json", @@ -97,6 +98,394 @@ func TestRunProcessHelpListsExpectedFlags(t *testing.T) { } } +func TestResolveConfigPathDefaultIgnoredWhenMissing(t *testing.T) { + lookup := func(string) (string, bool) { return "", false } + path, source, err := resolveConfigPath("", false, lookup) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + if path != "" || source != "" { + t.Fatalf("expected no config path/source, got path=%q source=%q", path, source) + } +} + +func TestRunConfigValidateSuccess(t *testing.T) { + var stdout bytes.Buffer + var stderr bytes.Buffer + cfgPath := writeFile(t, "config.yml", "version: 1\n") + + exitCode := Run([]string{"config", "validate", "--config", cfgPath}, &stdout, &stderr) + if exitCode != 0 { + t.Fatalf("expected success, got %d stderr=%q", exitCode, stderr.String()) + } + if !strings.Contains(stdout.String(), "config is valid") { + t.Fatalf("expected success message, got %q", stdout.String()) + } + if stderr.Len() != 0 { + t.Fatalf("expected empty stderr on success, got %q", stderr.String()) + } +} + +func TestRunConfigValidateInvalidVersion(t *testing.T) { + var stdout bytes.Buffer + var stderr bytes.Buffer + cfgPath := writeFile(t, "config.yml", "version: 999\n") + + exitCode := Run([]string{"config", "validate", "--config", cfgPath}, &stdout, &stderr) + if exitCode == 0 { + t.Fatalf("expected failure for unsupported config version") + } + if stdout.Len() != 0 { + t.Fatalf("expected empty stdout on failure, got %q", stdout.String()) + } + if !strings.Contains(stderr.String(), "unsupported config version") { + t.Fatalf("expected version error, got %q", stderr.String()) + } +} + +func TestRunConfigValidateUnknownField(t *testing.T) { + var stdout bytes.Buffer + var stderr bytes.Buffer + cfgPath := writeFile(t, "config.yml", "version: 1\nunknown_field: true\n") + + exitCode := Run([]string{"config", "validate", "--config", cfgPath}, &stdout, &stderr) + if exitCode == 0 { + t.Fatalf("expected failure for unknown field") + } + if stdout.Len() != 0 { + t.Fatalf("expected empty stdout on failure, got %q", stdout.String()) + } + if !strings.Contains(stderr.String(), "field unknown_field not found") { + t.Fatalf("expected unknown-field error, got %q", stderr.String()) + } +} + +func TestRunConfigPrintEffectiveOutputsRedactedJSON(t *testing.T) { + var stdout bytes.Buffer + var stderr bytes.Buffer + + secret := "super-secret-api-key" + t.Setenv("AUDITA_LLM_API_KEY", secret) + cfgPath := writeFile(t, "config.yml", "version: 1\n") + + exitCode := Run([]string{"config", "print-effective", "--config", cfgPath}, &stdout, &stderr) + if exitCode != 0 { + t.Fatalf("expected success, got %d stderr=%q", exitCode, stderr.String()) + } + if stderr.Len() != 0 { + t.Fatalf("expected empty stderr on success, got %q", stderr.String()) + } + if strings.Contains(stdout.String(), secret) { + t.Fatalf("print-effective leaked secret") + } + + var out map[string]any + if err := json.Unmarshal(stdout.Bytes(), &out); err != nil { + t.Fatalf("expected valid JSON output, got error: %v output=%q", err, stdout.String()) + } + primary, ok := out["PrimaryLLM"].(map[string]any) + if !ok { + t.Fatalf("expected PrimaryLLM object, got %#v", out["PrimaryLLM"]) + } + if got := primary["APIKey"]; got != "[REDACTED]" { + t.Fatalf("expected redacted API key, got %#v", got) + } +} + +func TestRunConfigCommandDoesNotRequireTranscriptOrGlossary(t *testing.T) { + var stdout bytes.Buffer + var stderr bytes.Buffer + cfgPath := writeFile(t, "config.yml", "version: 1\n") + + exitCode := Run([]string{"config", "validate", "--config", cfgPath}, &stdout, &stderr) + if exitCode != 0 { + t.Fatalf("expected success without transcript/glossary args, got %d stderr=%q", exitCode, stderr.String()) + } +} + +func TestRunProcessConfigFlagMissingFileFails(t *testing.T) { + var stdout bytes.Buffer + var stderr bytes.Buffer + + exitCode := Run([]string{ + "process", + fixturePath("tiny_transcript.json"), + "--glossary", + fixturePath("tiny_glossary.yaml"), + "--config", + filepath.Join(t.TempDir(), "missing.yaml"), + }, &stdout, &stderr) + if exitCode == 0 { + t.Fatalf("expected failure for missing --config file") + } + if stdout.Len() != 0 { + t.Fatalf("expected empty stdout on failure, got %q", stdout.String()) + } + if !strings.Contains(stderr.String(), "config file not found") { + t.Fatalf("expected missing config file error, got %q", stderr.String()) + } +} + +func TestRunProcessAUDITAConfigMissingFileFails(t *testing.T) { + var stdout bytes.Buffer + var stderr bytes.Buffer + t.Setenv("AUDITA_CONFIG", filepath.Join(t.TempDir(), "missing-env.yaml")) + + exitCode := Run([]string{ + "process", + fixturePath("tiny_transcript.json"), + "--glossary", + fixturePath("tiny_glossary.yaml"), + }, &stdout, &stderr) + if exitCode == 0 { + t.Fatalf("expected failure for missing AUDITA_CONFIG file") + } + if stdout.Len() != 0 { + t.Fatalf("expected empty stdout on failure, got %q", stdout.String()) + } + if !strings.Contains(stderr.String(), "config file not found") { + t.Fatalf("expected missing config file error, got %q", stderr.String()) + } +} + +func TestRunProcessConfigFileAffectsRuntimeAndInvocationMetadata(t *testing.T) { + var stdout bytes.Buffer + var stderr bytes.Buffer + + workDir := t.TempDir() + reportPath := filepath.Join(t.TempDir(), "report.json") + outputPath := filepath.Join(t.TempDir(), "out.json") + cfgPath := writeFile(t, "config.yml", ` +version: 1 +pipeline: + modules: [grammar] +context: + description: "config file transcript context" +diagnostics: + work_dir: `+workDir+` + retention: always +`) + + exitCode := Run([]string{ + "process", + fixturePath("tiny_transcript.json"), + "--glossary", + fixturePath("tiny_glossary.yaml"), + "--config", + cfgPath, + "--output", + outputPath, + "--report-json", + reportPath, + }, &stdout, &stderr) + if exitCode != 0 { + t.Fatalf("expected success, got %d stderr=%q", exitCode, stderr.String()) + } + if stdout.Len() != 0 { + t.Fatalf("expected empty stdout with --output, got %q", stdout.String()) + } + + runPath := onlyRunDir(t, workDir) + configBytes := readFile(t, filepath.Join(runPath, "effective-config.json")) + var effectiveConfig struct { + Modules []string `json:"Modules"` + TranscriptDescription string `json:"TranscriptDescription"` + } + if err := json.Unmarshal(configBytes, &effectiveConfig); err != nil { + t.Fatalf("failed to parse effective config metadata: %v", err) + } + if strings.Join(effectiveConfig.Modules, ",") != "grammar" { + t.Fatalf("expected grammar module from config file, got %#v", effectiveConfig.Modules) + } + if effectiveConfig.TranscriptDescription != "config file transcript context" { + t.Fatalf("unexpected transcript description from config file: %q", effectiveConfig.TranscriptDescription) + } + + invocationBytes := readFile(t, filepath.Join(runPath, "invocation.json")) + var invocation struct { + ConfigPath string `json:"config_path"` + ConfigSource string `json:"config_source"` + } + if err := json.Unmarshal(invocationBytes, &invocation); err != nil { + t.Fatalf("failed to parse invocation metadata: %v", err) + } + if invocation.ConfigPath != cfgPath { + t.Fatalf("unexpected invocation config_path: %q", invocation.ConfigPath) + } + if invocation.ConfigSource != "flag" { + t.Fatalf("unexpected invocation config_source: %q", invocation.ConfigSource) + } +} + +func TestRunProcessEnvOverridesConfigFile(t *testing.T) { + processModuleFactory = fakeModuleFactory{modules: map[string]contracts.TranscriptModule{ + "m": fakeModule{ + key: "m", + policy: proposals.ReplacementPolicyRequireUnique, + validators: []contracts.Validator{ + fakeValidator{name: "capture-config", validateF: func(req contracts.ValidationRequest) (validators.Result, error) { + if req.Config == nil { + t.Fatal("expected config in validation request") + } + if req.Config.PrimaryLLM.Model != "env-model" { + t.Fatalf("expected env model override, got %q", req.Config.PrimaryLLM.Model) + } + return validators.Result{ValidatorName: "capture-config", Decisions: nil}, nil + }}, + }, + proposeF: func(req contracts.ProposalRequest) ([]proposals.CorrectionProposal, error) { return nil, nil }, + }, + }} + t.Cleanup(func() { processModuleFactory = nil }) + + t.Setenv("AUDITA_MODEL", "env-model") + cfgPath := writeFile(t, "config.yml", ` +version: 1 +pipeline: + modules: [m] +llm: + proposal: + model: file-model +`) + + var stdout, stderr bytes.Buffer + exitCode := Run([]string{ + "process", + fixturePath("tiny_transcript.json"), + "--glossary", fixturePath("tiny_glossary.yaml"), + "--config", cfgPath, + "--modules", "m", + }, &stdout, &stderr) + if exitCode != 0 { + t.Fatalf("expected success, got %d stderr=%q", exitCode, stderr.String()) + } +} + +func TestRunProcessCLIOverridesEnvAndConfigFile(t *testing.T) { + processModuleFactory = fakeModuleFactory{modules: map[string]contracts.TranscriptModule{ + "m": fakeModule{ + key: "m", + policy: proposals.ReplacementPolicyRequireUnique, + validators: []contracts.Validator{ + fakeValidator{name: "capture-config", validateF: func(req contracts.ValidationRequest) (validators.Result, error) { + if req.Config == nil { + t.Fatal("expected config in validation request") + } + if req.Config.PrimaryLLM.Model != "cli-model" { + t.Fatalf("expected CLI model override, got %q", req.Config.PrimaryLLM.Model) + } + return validators.Result{ValidatorName: "capture-config", Decisions: nil}, nil + }}, + }, + proposeF: func(req contracts.ProposalRequest) ([]proposals.CorrectionProposal, error) { return nil, nil }, + }, + }} + t.Cleanup(func() { processModuleFactory = nil }) + + t.Setenv("AUDITA_MODEL", "env-model") + cfgPath := writeFile(t, "config.yml", ` +version: 1 +pipeline: + modules: [m] +llm: + proposal: + model: file-model +`) + + var stdout, stderr bytes.Buffer + exitCode := Run([]string{ + "process", + fixturePath("tiny_transcript.json"), + "--glossary", fixturePath("tiny_glossary.yaml"), + "--config", cfgPath, + "--modules", "m", + "--model", "cli-model", + }, &stdout, &stderr) + if exitCode != 0 { + t.Fatalf("expected success, got %d stderr=%q", exitCode, stderr.String()) + } +} + +func TestRunProcessConfigAPIKeyEnvResolvesAndRedactsEffectiveConfig(t *testing.T) { + secret := "secret-from-config-env" + t.Setenv("PROPOSAL_KEY_FROM_CONFIG", secret) + + workDir := t.TempDir() + outputPath := filepath.Join(t.TempDir(), "out.json") + cfgPath := writeFile(t, "config.yml", ` +version: 1 +pipeline: + modules: [grammar] +llm: + proposal: + api_key_env: PROPOSAL_KEY_FROM_CONFIG +diagnostics: + work_dir: `+workDir+` + retention: always +`) + + var stdout, stderr bytes.Buffer + exitCode := Run([]string{ + "process", + fixturePath("tiny_transcript.json"), + "--glossary", fixturePath("tiny_glossary.yaml"), + "--config", cfgPath, + "--output", outputPath, + }, &stdout, &stderr) + if exitCode != 0 { + t.Fatalf("expected success, got %d stderr=%q", exitCode, stderr.String()) + } + + runPath := onlyRunDir(t, workDir) + configBytes := readFile(t, filepath.Join(runPath, "effective-config.json")) + if strings.Contains(string(configBytes), secret) { + t.Fatalf("effective config artifact leaked API key from api_key_env") + } +} + +func TestRunProcessTranscriptDescriptionCLIOverridesConfigFileContextDescription(t *testing.T) { + processModuleFactory = fakeModuleFactory{modules: map[string]contracts.TranscriptModule{ + "m": fakeModule{ + key: "m", + policy: proposals.ReplacementPolicyRequireUnique, + validators: []contracts.Validator{ + fakeValidator{name: "capture-config", validateF: func(req contracts.ValidationRequest) (validators.Result, error) { + if req.Config == nil { + t.Fatal("expected config in validation request") + } + if req.Config.TranscriptDescription != "cli transcript description" { + t.Fatalf("expected CLI transcript description to override config file, got %q", req.Config.TranscriptDescription) + } + return validators.Result{ValidatorName: "capture-config", Decisions: nil}, nil + }}, + }, + proposeF: func(req contracts.ProposalRequest) ([]proposals.CorrectionProposal, error) { return nil, nil }, + }, + }} + t.Cleanup(func() { processModuleFactory = nil }) + + cfgPath := writeFile(t, "config.yml", ` +version: 1 +pipeline: + modules: [m] +context: + description: "file transcript description" +`) + + var stdout, stderr bytes.Buffer + exitCode := Run([]string{ + "process", + fixturePath("tiny_transcript.json"), + "--glossary", fixturePath("tiny_glossary.yaml"), + "--config", cfgPath, + "--modules", "m", + "--transcript-description", "cli transcript description", + }, &stdout, &stderr) + if exitCode != 0 { + t.Fatalf("expected success, got %d stderr=%q", exitCode, stderr.String()) + } +} + func TestRunProcessMissingTranscriptPath(t *testing.T) { var stdout bytes.Buffer var stderr bytes.Buffer diff --git a/internal/core/config/env.go b/internal/core/config/env.go index 33d19a8..f9ff257 100644 --- a/internal/core/config/env.go +++ b/internal/core/config/env.go @@ -6,17 +6,38 @@ import ( "strconv" ) +const DefaultConfigPath = "/etc/audita/config.yml" + func LoadFromEnv() (Config, error) { - return loadFromLookup(os.LookupEnv) + cfg := Default() + if err := cfg.applyEnvOverrides(os.LookupEnv); err != nil { + return Config{}, err + } + return cfg, nil } func loadFromLookup(lookup func(string) (string, bool)) (Config, error) { cfg := Default() + if err := cfg.applyEnvOverrides(lookup); err != nil { + return Config{}, err + } + return cfg, nil +} +func (c *Config) ApplyEnvOverrides() error { + return c.applyEnvOverrides(os.LookupEnv) +} + +func (c *Config) applyEnvOverrides(lookup func(string) (string, bool)) error { + if c == nil { + return fmt.Errorf("config must not be nil") + } + + cfg := c if raw, ok := lookup("AUDITA_MODULES"); ok { modules, err := ParseModulesCSV(raw) if err != nil { - return Config{}, fmt.Errorf("AUDITA_MODULES: %w", err) + return fmt.Errorf("AUDITA_MODULES: %w", err) } cfg.Modules = modules } @@ -48,7 +69,7 @@ func loadFromLookup(lookup func(string) (string, bool)) (Config, error) { if raw, ok := lookup("AUDITA_LLM_TIMEOUT_SECONDS"); ok { value, err := parseInt(raw) if err != nil { - return Config{}, fmt.Errorf("AUDITA_LLM_TIMEOUT_SECONDS: %w", err) + return fmt.Errorf("AUDITA_LLM_TIMEOUT_SECONDS: %w", err) } cfg.PrimaryLLM.TimeoutSeconds = value } @@ -56,7 +77,7 @@ func loadFromLookup(lookup func(string) (string, bool)) (Config, error) { if raw, ok := lookup("AUDITA_VALIDATION_LLM_TIMEOUT_SECONDS"); ok { value, err := parseInt(raw) if err != nil { - return Config{}, fmt.Errorf("AUDITA_VALIDATION_LLM_TIMEOUT_SECONDS: %w", err) + return fmt.Errorf("AUDITA_VALIDATION_LLM_TIMEOUT_SECONDS: %w", err) } cfg.ValidationLLM.TimeoutSeconds = &value } @@ -64,7 +85,7 @@ func loadFromLookup(lookup func(string) (string, bool)) (Config, error) { if raw, ok := lookup("AUDITA_MAX_RETRIES"); ok { value, err := parseInt(raw) if err != nil { - return Config{}, fmt.Errorf("AUDITA_MAX_RETRIES: %w", err) + return fmt.Errorf("AUDITA_MAX_RETRIES: %w", err) } cfg.PrimaryLLM.MaxRetries = value } @@ -72,7 +93,7 @@ func loadFromLookup(lookup func(string) (string, bool)) (Config, error) { if raw, ok := lookup("AUDITA_TOTAL_LLM_CONCURRENCY"); ok { value, err := parseInt(raw) if err != nil { - return Config{}, fmt.Errorf("AUDITA_TOTAL_LLM_CONCURRENCY: %w", err) + return fmt.Errorf("AUDITA_TOTAL_LLM_CONCURRENCY: %w", err) } cfg.TotalLLMConcurrency = value totalConcurrencySet = true @@ -80,7 +101,7 @@ func loadFromLookup(lookup func(string) (string, bool)) (Config, error) { if raw, ok := lookup("AUDITA_LLM_CONCURRENCY"); ok { value, err := parseInt(raw) if err != nil { - return Config{}, fmt.Errorf("AUDITA_LLM_CONCURRENCY: %w", err) + return fmt.Errorf("AUDITA_LLM_CONCURRENCY: %w", err) } if !totalConcurrencySet { cfg.TotalLLMConcurrency = value @@ -92,7 +113,7 @@ func loadFromLookup(lookup func(string) (string, bool)) (Config, error) { if raw, ok := lookup("AUDITA_PROPOSAL_LLM_CONCURRENCY"); ok { value, err := parseInt(raw) if err != nil { - return Config{}, fmt.Errorf("AUDITA_PROPOSAL_LLM_CONCURRENCY: %w", err) + return fmt.Errorf("AUDITA_PROPOSAL_LLM_CONCURRENCY: %w", err) } cfg.ProposalLLMConcurrency = value proposalConcurrencySet = true @@ -104,14 +125,14 @@ func loadFromLookup(lookup func(string) (string, bool)) (Config, error) { if raw, ok := lookup("AUDITA_VALIDATION_MAX_RETRIES"); ok { value, err := parseInt(raw) if err != nil { - return Config{}, fmt.Errorf("AUDITA_VALIDATION_MAX_RETRIES: %w", err) + return fmt.Errorf("AUDITA_VALIDATION_MAX_RETRIES: %w", err) } cfg.ValidationLLM.MaxRetries = &value } if raw, ok := lookup("AUDITA_VALIDATION_LLM_CONCURRENCY"); ok { value, err := parseInt(raw) if err != nil { - return Config{}, fmt.Errorf("AUDITA_VALIDATION_LLM_CONCURRENCY: %w", err) + return fmt.Errorf("AUDITA_VALIDATION_LLM_CONCURRENCY: %w", err) } cfg.ValidationLLMConcurrency = &value } @@ -119,7 +140,7 @@ func loadFromLookup(lookup func(string) (string, bool)) (Config, error) { if raw, ok := lookup("AUDITA_VALIDATION_MAX_PROMPT_TOKENS"); ok { value, err := parseInt(raw) if err != nil { - return Config{}, fmt.Errorf("AUDITA_VALIDATION_MAX_PROMPT_TOKENS: %w", err) + return fmt.Errorf("AUDITA_VALIDATION_MAX_PROMPT_TOKENS: %w", err) } cfg.ValidationMaxPromptTokens = value } @@ -127,7 +148,7 @@ func loadFromLookup(lookup func(string) (string, bool)) (Config, error) { if raw, ok := lookup("AUDITA_MAX_SECTION_TOKENS"); ok { value, err := parseInt(raw) if err != nil { - return Config{}, fmt.Errorf("AUDITA_MAX_SECTION_TOKENS: %w", err) + return fmt.Errorf("AUDITA_MAX_SECTION_TOKENS: %w", err) } cfg.MaxSectionTokens = value } @@ -135,7 +156,7 @@ func loadFromLookup(lookup func(string) (string, bool)) (Config, error) { if raw, ok := lookup("AUDITA_MIN_SECTION_TOKENS"); ok { value, err := parseInt(raw) if err != nil { - return Config{}, fmt.Errorf("AUDITA_MIN_SECTION_TOKENS: %w", err) + return fmt.Errorf("AUDITA_MIN_SECTION_TOKENS: %w", err) } cfg.MinSectionTokens = value } @@ -143,7 +164,7 @@ func loadFromLookup(lookup func(string) (string, bool)) (Config, error) { if raw, ok := lookup("AUDITA_TARGET_SECTIONS"); ok { value, err := parseInt(raw) if err != nil { - return Config{}, fmt.Errorf("AUDITA_TARGET_SECTIONS: %w", err) + return fmt.Errorf("AUDITA_TARGET_SECTIONS: %w", err) } cfg.TargetSections = &value } @@ -151,28 +172,28 @@ func loadFromLookup(lookup func(string) (string, bool)) (Config, error) { if raw, ok := lookup("AUDITA_GLOSSARY_CONFIDENCE_THRESHOLD"); ok { value, err := parseFloat(raw) if err != nil { - return Config{}, fmt.Errorf("AUDITA_GLOSSARY_CONFIDENCE_THRESHOLD: %w", err) + return fmt.Errorf("AUDITA_GLOSSARY_CONFIDENCE_THRESHOLD: %w", err) } cfg.Thresholds.Glossary = value } if raw, ok := lookup("AUDITA_GRAMMAR_CONFIDENCE_THRESHOLD"); ok { value, err := parseFloat(raw) if err != nil { - return Config{}, fmt.Errorf("AUDITA_GRAMMAR_CONFIDENCE_THRESHOLD: %w", err) + return fmt.Errorf("AUDITA_GRAMMAR_CONFIDENCE_THRESHOLD: %w", err) } cfg.Thresholds.Grammar = value } if raw, ok := lookup("AUDITA_HOMOPHONES_CONFIDENCE_THRESHOLD"); ok { value, err := parseFloat(raw) if err != nil { - return Config{}, fmt.Errorf("AUDITA_HOMOPHONES_CONFIDENCE_THRESHOLD: %w", err) + return fmt.Errorf("AUDITA_HOMOPHONES_CONFIDENCE_THRESHOLD: %w", err) } cfg.Thresholds.Homophones = value } if raw, ok := lookup("AUDITA_SPOKEN_WORD_CONFIDENCE_THRESHOLD"); ok { value, err := parseFloat(raw) if err != nil { - return Config{}, fmt.Errorf("AUDITA_SPOKEN_WORD_CONFIDENCE_THRESHOLD: %w", err) + return fmt.Errorf("AUDITA_SPOKEN_WORD_CONFIDENCE_THRESHOLD: %w", err) } cfg.Thresholds.SpokenWord = value } @@ -180,28 +201,28 @@ func loadFromLookup(lookup func(string) (string, bool)) (Config, error) { if raw, ok := lookup("AUDITA_NORMALIZE_MAX_SEGMENT_GAP"); ok { value, err := parseFloat(raw) if err != nil { - return Config{}, fmt.Errorf("AUDITA_NORMALIZE_MAX_SEGMENT_GAP: %w", err) + return fmt.Errorf("AUDITA_NORMALIZE_MAX_SEGMENT_GAP: %w", err) } cfg.Normalization.MaxSegmentGap = value } if raw, ok := lookup("AUDITA_NORMALIZE_ELLIPSIS_GAP"); ok { value, err := parseFloat(raw) if err != nil { - return Config{}, fmt.Errorf("AUDITA_NORMALIZE_ELLIPSIS_GAP: %w", err) + return fmt.Errorf("AUDITA_NORMALIZE_ELLIPSIS_GAP: %w", err) } cfg.Normalization.EllipsisGap = value } if raw, ok := lookup("AUDITA_NORMALIZE_MAX_SEGMENT_DURATION"); ok { value, err := parseFloat(raw) if err != nil { - return Config{}, fmt.Errorf("AUDITA_NORMALIZE_MAX_SEGMENT_DURATION: %w", err) + return fmt.Errorf("AUDITA_NORMALIZE_MAX_SEGMENT_DURATION: %w", err) } cfg.Normalization.MaxSegmentDuration = value } if raw, ok := lookup("AUDITA_NORMALIZE_MAX_SEGMENT_TOKENS"); ok { value, err := parseInt(raw) if err != nil { - return Config{}, fmt.Errorf("AUDITA_NORMALIZE_MAX_SEGMENT_TOKENS: %w", err) + return fmt.Errorf("AUDITA_NORMALIZE_MAX_SEGMENT_TOKENS: %w", err) } cfg.Normalization.MaxSegmentTokens = value } @@ -216,10 +237,10 @@ func loadFromLookup(lookup func(string) (string, bool)) (Config, error) { cfg.syncLegacyConcurrencyAliases() if err := cfg.Validate(); err != nil { - return Config{}, err + return err } - return cfg, nil + return nil } func parseInt(raw string) (int, error) { diff --git a/internal/core/config/file_config.go b/internal/core/config/file_config.go new file mode 100644 index 0000000..5d37ce7 --- /dev/null +++ b/internal/core/config/file_config.go @@ -0,0 +1,334 @@ +package config + +import ( + "fmt" + "os" + "path/filepath" + "regexp" + "strings" + "time" + + "gopkg.in/yaml.v3" +) + +const SupportedFileConfigVersion = 1 + +var envVarNamePattern = regexp.MustCompile(`^[A-Za-z_][A-Za-z0-9_]*$`) + +type FileConfig struct { + Version int `yaml:"version"` + Pipeline *FileConfigPipeline `yaml:"pipeline,omitempty"` + LLM *FileConfigLLM `yaml:"llm,omitempty"` + Concurrency *FileConfigConcurrency `yaml:"concurrency,omitempty"` + Chunking *FileConfigChunking `yaml:"chunking,omitempty"` + Normalization *FileConfigNormalization `yaml:"normalization,omitempty"` + Thresholds *FileConfigThresholds `yaml:"thresholds,omitempty"` + Context *FileConfigContext `yaml:"context,omitempty"` + Diagnostics *FileConfigDiagnostics `yaml:"diagnostics,omitempty"` +} + +type FileConfigPipeline struct { + Modules []string `yaml:"modules,omitempty"` +} + +type FileConfigLLM struct { + Proposal *FileConfigLLMTarget `yaml:"proposal,omitempty"` + Validation *FileConfigLLMTarget `yaml:"validation,omitempty"` +} + +type FileConfigLLMTarget struct { + BaseURL *string `yaml:"base_url,omitempty"` + Model *string `yaml:"model,omitempty"` + APIKeyEnv *string `yaml:"api_key_env,omitempty"` + Timeout *fileConfigDurationOrInt `yaml:"timeout,omitempty"` + MaxRetries *int `yaml:"max_retries,omitempty"` +} + +type FileConfigConcurrency struct { + TotalLLM *int `yaml:"total_llm,omitempty"` + ProposalLLM *int `yaml:"proposal_llm,omitempty"` + ValidationLLM *int `yaml:"validation_llm,omitempty"` +} + +type FileConfigChunking struct { + TargetSections *int `yaml:"target_sections,omitempty"` + MaxSectionTokens *int `yaml:"max_section_tokens,omitempty"` + MinSectionTokens *int `yaml:"min_section_tokens,omitempty"` +} + +type FileConfigNormalization struct { + MaxSegmentGap *fileConfigDurationOrFloat `yaml:"max_segment_gap,omitempty"` + EllipsisGap *fileConfigDurationOrFloat `yaml:"ellipsis_gap,omitempty"` + MaxSegmentDuration *fileConfigDurationOrFloat `yaml:"max_segment_duration,omitempty"` + MaxSegmentTokens *int `yaml:"max_segment_tokens,omitempty"` +} + +type FileConfigThresholds struct { + Glossary *float64 `yaml:"glossary,omitempty"` + Homophones *float64 `yaml:"homophones,omitempty"` + SpokenWord *float64 `yaml:"spoken_word,omitempty"` + Grammar *float64 `yaml:"grammar,omitempty"` +} + +type FileConfigContext struct { + Description *string `yaml:"description,omitempty"` +} + +type FileConfigDiagnostics struct { + WorkDir *string `yaml:"work_dir,omitempty"` + Retention *string `yaml:"retention,omitempty"` +} + +type fileConfigDurationOrInt struct { + seconds int +} + +func (v *fileConfigDurationOrInt) UnmarshalYAML(node *yaml.Node) error { + switch node.Kind { + case yaml.ScalarNode: + if node.Tag == "!!int" { + var n int + if err := node.Decode(&n); err != nil { + return fmt.Errorf("must be an integer seconds value or duration string") + } + v.seconds = n + return nil + } + + var s string + if err := node.Decode(&s); err != nil { + return fmt.Errorf("must be an integer seconds value or duration string") + } + d, err := time.ParseDuration(strings.TrimSpace(s)) + if err != nil { + return fmt.Errorf("invalid duration %q", s) + } + if d <= 0 { + v.seconds = int(d / time.Second) + return nil + } + if d%time.Second != 0 { + return fmt.Errorf("duration %q must resolve to whole seconds", s) + } + v.seconds = int(d / time.Second) + return nil + default: + return fmt.Errorf("must be an integer seconds value or duration string") + } +} + +func (v fileConfigDurationOrInt) Seconds() int { return v.seconds } + +type fileConfigDurationOrFloat struct { + seconds float64 +} + +func (v *fileConfigDurationOrFloat) UnmarshalYAML(node *yaml.Node) error { + switch node.Kind { + case yaml.ScalarNode: + if node.Tag == "!!int" || node.Tag == "!!float" { + var f float64 + if err := node.Decode(&f); err != nil { + return fmt.Errorf("must be a numeric seconds value or duration string") + } + v.seconds = f + return nil + } + var s string + if err := node.Decode(&s); err != nil { + return fmt.Errorf("must be a numeric seconds value or duration string") + } + d, err := time.ParseDuration(strings.TrimSpace(s)) + if err != nil { + return fmt.Errorf("invalid duration %q", s) + } + v.seconds = d.Seconds() + return nil + default: + return fmt.Errorf("must be a numeric seconds value or duration string") + } +} + +func (v fileConfigDurationOrFloat) Seconds() float64 { return v.seconds } + +func LoadFileConfig(path string) (FileConfig, error) { + b, err := os.ReadFile(path) + if err != nil { + return FileConfig{}, fmt.Errorf("read config file %q: %w", path, err) + } + cfg, err := ParseFileConfigYAML(b) + if err != nil { + return FileConfig{}, fmt.Errorf("parse config file %q: %w", path, err) + } + return cfg, nil +} + +func ParseFileConfigYAML(data []byte) (FileConfig, error) { + var fileCfg FileConfig + dec := yaml.NewDecoder(strings.NewReader(string(data))) + dec.KnownFields(true) + if err := dec.Decode(&fileCfg); err != nil { + return FileConfig{}, fmt.Errorf("decode yaml: %w", err) + } + if fileCfg.Version == 0 { + return FileConfig{}, fmt.Errorf("config version is required") + } + if fileCfg.Version != SupportedFileConfigVersion { + return FileConfig{}, fmt.Errorf("unsupported config version %d", fileCfg.Version) + } + return fileCfg, nil +} + +func (c *Config) ApplyFileConfig(fileCfg FileConfig) error { + return c.applyFileConfigWithLookup(fileCfg, os.LookupEnv) +} + +func (c *Config) applyFileConfigWithLookup(fileCfg FileConfig, lookup func(string) (string, bool)) error { + if c == nil { + return fmt.Errorf("config must not be nil") + } + if fileCfg.Version != SupportedFileConfigVersion { + return fmt.Errorf("unsupported config version %d", fileCfg.Version) + } + + if fileCfg.Pipeline != nil && len(fileCfg.Pipeline.Modules) > 0 { + c.Modules = append([]string(nil), fileCfg.Pipeline.Modules...) + } + + if fileCfg.LLM != nil { + if fileCfg.LLM.Proposal != nil { + if fileCfg.LLM.Proposal.BaseURL != nil { + c.PrimaryLLM.BaseURL = *fileCfg.LLM.Proposal.BaseURL + } + if fileCfg.LLM.Proposal.Model != nil { + c.PrimaryLLM.Model = *fileCfg.LLM.Proposal.Model + } + if fileCfg.LLM.Proposal.Timeout != nil { + c.PrimaryLLM.TimeoutSeconds = fileCfg.LLM.Proposal.Timeout.Seconds() + } + if fileCfg.LLM.Proposal.MaxRetries != nil { + c.PrimaryLLM.MaxRetries = *fileCfg.LLM.Proposal.MaxRetries + } + if fileCfg.LLM.Proposal.APIKeyEnv != nil { + apiKey, err := resolveAPIKeyEnv(*fileCfg.LLM.Proposal.APIKeyEnv, lookup) + if err != nil { + return fmt.Errorf("llm.proposal.api_key_env: %w", err) + } + c.PrimaryLLM.APIKey = apiKey + } + } + if fileCfg.LLM.Validation != nil { + if fileCfg.LLM.Validation.BaseURL != nil { + c.ValidationLLM.BaseURL = *fileCfg.LLM.Validation.BaseURL + } + if fileCfg.LLM.Validation.Model != nil { + c.ValidationLLM.Model = *fileCfg.LLM.Validation.Model + } + if fileCfg.LLM.Validation.Timeout != nil { + v := fileCfg.LLM.Validation.Timeout.Seconds() + c.ValidationLLM.TimeoutSeconds = &v + } + if fileCfg.LLM.Validation.MaxRetries != nil { + v := *fileCfg.LLM.Validation.MaxRetries + c.ValidationLLM.MaxRetries = &v + } + if fileCfg.LLM.Validation.APIKeyEnv != nil { + apiKey, err := resolveAPIKeyEnv(*fileCfg.LLM.Validation.APIKeyEnv, lookup) + if err != nil { + return fmt.Errorf("llm.validation.api_key_env: %w", err) + } + c.ValidationLLM.APIKey = apiKey + } + } + } + + if fileCfg.Concurrency != nil { + if fileCfg.Concurrency.TotalLLM != nil { + c.TotalLLMConcurrency = *fileCfg.Concurrency.TotalLLM + } + if fileCfg.Concurrency.ProposalLLM != nil { + c.ProposalLLMConcurrency = *fileCfg.Concurrency.ProposalLLM + } + if fileCfg.Concurrency.ValidationLLM != nil { + v := *fileCfg.Concurrency.ValidationLLM + c.ValidationLLMConcurrency = &v + } + } + + if fileCfg.Chunking != nil { + if fileCfg.Chunking.TargetSections != nil { + v := *fileCfg.Chunking.TargetSections + c.TargetSections = &v + } + if fileCfg.Chunking.MaxSectionTokens != nil { + c.MaxSectionTokens = *fileCfg.Chunking.MaxSectionTokens + } + if fileCfg.Chunking.MinSectionTokens != nil { + c.MinSectionTokens = *fileCfg.Chunking.MinSectionTokens + } + } + + if fileCfg.Normalization != nil { + if fileCfg.Normalization.MaxSegmentGap != nil { + c.Normalization.MaxSegmentGap = fileCfg.Normalization.MaxSegmentGap.Seconds() + } + if fileCfg.Normalization.EllipsisGap != nil { + c.Normalization.EllipsisGap = fileCfg.Normalization.EllipsisGap.Seconds() + } + if fileCfg.Normalization.MaxSegmentDuration != nil { + c.Normalization.MaxSegmentDuration = fileCfg.Normalization.MaxSegmentDuration.Seconds() + } + if fileCfg.Normalization.MaxSegmentTokens != nil { + c.Normalization.MaxSegmentTokens = *fileCfg.Normalization.MaxSegmentTokens + } + } + + if fileCfg.Thresholds != nil { + if fileCfg.Thresholds.Glossary != nil { + c.Thresholds.Glossary = *fileCfg.Thresholds.Glossary + } + if fileCfg.Thresholds.Homophones != nil { + c.Thresholds.Homophones = *fileCfg.Thresholds.Homophones + } + if fileCfg.Thresholds.SpokenWord != nil { + c.Thresholds.SpokenWord = *fileCfg.Thresholds.SpokenWord + } + if fileCfg.Thresholds.Grammar != nil { + c.Thresholds.Grammar = *fileCfg.Thresholds.Grammar + } + } + + if fileCfg.Context != nil && fileCfg.Context.Description != nil { + c.TranscriptDescription = strings.TrimSpace(*fileCfg.Context.Description) + } + + if fileCfg.Diagnostics != nil { + if fileCfg.Diagnostics.WorkDir != nil { + c.WorkDir = *fileCfg.Diagnostics.WorkDir + } + if fileCfg.Diagnostics.Retention != nil { + c.WorkDirRetention = WorkDirRetention(*fileCfg.Diagnostics.Retention) + } + } + + c.syncLegacyConcurrencyAliases() + if err := c.Validate(); err != nil { + return err + } + return nil +} + +func resolveAPIKeyEnv(envName string, lookup func(string) (string, bool)) (string, error) { + name := strings.TrimSpace(envName) + if name == "" { + return "", fmt.Errorf("must not be empty") + } + if !envVarNamePattern.MatchString(name) { + return "", fmt.Errorf("must be an environment variable name") + } + if strings.Contains(name, string(filepath.Separator)) { + return "", fmt.Errorf("must be an environment variable name") + } + v, _ := lookup(name) + return v, nil +} diff --git a/internal/core/config/file_config_test.go b/internal/core/config/file_config_test.go new file mode 100644 index 0000000..2edd559 --- /dev/null +++ b/internal/core/config/file_config_test.go @@ -0,0 +1,280 @@ +package config + +import ( + "os" + "strings" + "testing" +) + +func TestParseFileConfigYAMLValid(t *testing.T) { + raw := ` +version: 1 +pipeline: + modules: [glossary, homophones, grammar] +llm: + proposal: + base_url: https://example.test/v1 + model: provider/model-a + api_key_env: AUDITA_PROPOSAL_KEY + timeout: 2m + max_retries: 4 + validation: + base_url: https://example.test/validation + model: provider/model-b + api_key_env: AUDITA_VALIDATION_KEY + timeout: 45 + max_retries: 3 +concurrency: + total_llm: 8 + proposal_llm: 4 + validation_llm: 2 +chunking: + target_sections: 6 + max_section_tokens: 9000 + min_section_tokens: 3000 +normalization: + max_segment_gap: 1.5s + ellipsis_gap: 2 + max_segment_duration: 45s + max_segment_tokens: 1500 +thresholds: + glossary: 0.9 + homophones: 0.7 + spoken_word: 0.8 + grammar: 0.75 +context: + description: " crowd scene with many proper nouns " +diagnostics: + work_dir: /tmp/audita-config + retention: always +` + cfg, err := ParseFileConfigYAML([]byte(raw)) + if err != nil { + t.Fatalf("ParseFileConfigYAML error: %v", err) + } + if cfg.Version != 1 { + t.Fatalf("expected version 1, got %d", cfg.Version) + } + if cfg.Pipeline == nil || len(cfg.Pipeline.Modules) != 3 { + t.Fatalf("unexpected pipeline modules: %#v", cfg.Pipeline) + } + if cfg.LLM == nil || cfg.LLM.Proposal == nil || cfg.LLM.Validation == nil { + t.Fatalf("expected llm proposal+validation blocks") + } + if cfg.LLM.Proposal.Timeout == nil || cfg.LLM.Proposal.Timeout.Seconds() != 120 { + t.Fatalf("expected proposal timeout 120s, got %#v", cfg.LLM.Proposal.Timeout) + } + if cfg.LLM.Validation.Timeout == nil || cfg.LLM.Validation.Timeout.Seconds() != 45 { + t.Fatalf("expected validation timeout 45s, got %#v", cfg.LLM.Validation.Timeout) + } + if cfg.Normalization == nil || cfg.Normalization.MaxSegmentGap == nil || cfg.Normalization.MaxSegmentGap.Seconds() != 1.5 { + t.Fatalf("expected parsed duration for normalization max_segment_gap") + } +} + +func TestParseFileConfigYAMLRejectsUnknownField(t *testing.T) { + raw := ` +version: 1 +pipeline: + modules: [grammar] +output: + schema: v1 +` + _, err := ParseFileConfigYAML([]byte(raw)) + if err == nil { + t.Fatalf("expected unknown field error") + } + if !strings.Contains(err.Error(), "field output not found") { + t.Fatalf("unexpected error: %v", err) + } +} + +func TestParseFileConfigYAMLRejectsMissingVersion(t *testing.T) { + raw := `pipeline: {modules: [grammar]}` + _, err := ParseFileConfigYAML([]byte(raw)) + if err == nil { + t.Fatalf("expected missing version error") + } + if !strings.Contains(err.Error(), "config version is required") { + t.Fatalf("unexpected error: %v", err) + } +} + +func TestParseFileConfigYAMLRejectsUnsupportedVersion(t *testing.T) { + raw := `version: 2` + _, err := ParseFileConfigYAML([]byte(raw)) + if err == nil { + t.Fatalf("expected unsupported version error") + } + if !strings.Contains(err.Error(), "unsupported config version 2") { + t.Fatalf("unexpected error: %v", err) + } +} + +func TestApplyFileConfigParsesAndMergesFields(t *testing.T) { + raw := ` +version: 1 +pipeline: + modules: [spoken_word, grammar] +llm: + proposal: + model: provider/new-proposal + api_key_env: PROPOSAL_KEY_NAME + timeout: 90s + max_retries: 5 + validation: + model: provider/new-validation + api_key_env: VALIDATION_KEY_NAME + timeout: 150 + max_retries: 6 +concurrency: + total_llm: 7 + proposal_llm: 3 + validation_llm: 2 +chunking: + target_sections: 9 +thresholds: + glossary: 0.91 + homophones: 0.61 + spoken_word: 0.71 + grammar: 0.81 +diagnostics: + retention: never +` + fileCfg, err := ParseFileConfigYAML([]byte(raw)) + if err != nil { + t.Fatalf("ParseFileConfigYAML error: %v", err) + } + + cfg := Default() + lookup := func(name string) (string, bool) { + switch name { + case "PROPOSAL_KEY_NAME": + return "proposal-secret", true + case "VALIDATION_KEY_NAME": + return "validation-secret", true + default: + return "", false + } + } + + if err := cfg.applyFileConfigWithLookup(fileCfg, lookup); err != nil { + t.Fatalf("applyFileConfigWithLookup error: %v", err) + } + if strings.Join(cfg.Modules, ",") != "spoken_word,grammar" { + t.Fatalf("unexpected modules: %#v", cfg.Modules) + } + if cfg.PrimaryLLM.Model != "provider/new-proposal" { + t.Fatalf("unexpected proposal model: %q", cfg.PrimaryLLM.Model) + } + if cfg.PrimaryLLM.APIKey != "proposal-secret" { + t.Fatalf("expected proposal key from api_key_env lookup, got %q", cfg.PrimaryLLM.APIKey) + } + if cfg.PrimaryLLM.TimeoutSeconds != 90 { + t.Fatalf("unexpected proposal timeout: %d", cfg.PrimaryLLM.TimeoutSeconds) + } + if cfg.ValidationLLM.Model != "provider/new-validation" { + t.Fatalf("unexpected validation model: %q", cfg.ValidationLLM.Model) + } + if cfg.ValidationLLM.APIKey != "validation-secret" { + t.Fatalf("expected validation key from api_key_env lookup, got %q", cfg.ValidationLLM.APIKey) + } + if cfg.ValidationLLM.TimeoutSeconds == nil || *cfg.ValidationLLM.TimeoutSeconds != 150 { + t.Fatalf("unexpected validation timeout: %#v", cfg.ValidationLLM.TimeoutSeconds) + } + if cfg.TotalLLMConcurrency != 7 || cfg.ProposalLLMConcurrency != 3 { + t.Fatalf("unexpected llm concurrency values: total=%d proposal=%d", cfg.TotalLLMConcurrency, cfg.ProposalLLMConcurrency) + } + if cfg.ValidationLLMConcurrency == nil || *cfg.ValidationLLMConcurrency != 2 { + t.Fatalf("unexpected validation llm concurrency: %#v", cfg.ValidationLLMConcurrency) + } + if cfg.TargetSections == nil || *cfg.TargetSections != 9 { + t.Fatalf("unexpected target sections: %#v", cfg.TargetSections) + } + if cfg.WorkDirRetention != WorkDirRetentionNever { + t.Fatalf("unexpected retention: %q", cfg.WorkDirRetention) + } + if cfg.PrimaryLLM.Concurrency != 7 { + t.Fatalf("expected legacy alias to sync, got %d", cfg.PrimaryLLM.Concurrency) + } + if cfg.ValidationLLM.Concurrency == nil || *cfg.ValidationLLM.Concurrency != 2 { + t.Fatalf("expected validation alias to sync, got %#v", cfg.ValidationLLM.Concurrency) + } +} + +func TestApplyFileConfigContextDescriptionTrim(t *testing.T) { + raw := ` +version: 1 +context: + description: " scene context " +` + fileCfg, err := ParseFileConfigYAML([]byte(raw)) + if err != nil { + t.Fatalf("ParseFileConfigYAML error: %v", err) + } + cfg := Default() + if err := cfg.applyFileConfigWithLookup(fileCfg, mapLookup(map[string]string{})); err != nil { + t.Fatalf("applyFileConfigWithLookup error: %v", err) + } + if cfg.TranscriptDescription != "scene context" { + t.Fatalf("unexpected transcript description: %q", cfg.TranscriptDescription) + } +} + +func TestApplyFileConfigRejectsInvalidAPIKeyEnvName(t *testing.T) { + raw := ` +version: 1 +llm: + proposal: + api_key_env: "not a var name" +` + fileCfg, err := ParseFileConfigYAML([]byte(raw)) + if err != nil { + t.Fatalf("ParseFileConfigYAML error: %v", err) + } + cfg := Default() + err = cfg.applyFileConfigWithLookup(fileCfg, mapLookup(map[string]string{})) + if err == nil { + t.Fatalf("expected api_key_env validation error") + } + if !strings.Contains(err.Error(), "environment variable name") { + t.Fatalf("unexpected error: %v", err) + } +} + +func TestParseFileConfigDurationParsingErrors(t *testing.T) { + raw := ` +version: 1 +llm: + proposal: + timeout: "1.5s" +` + _, err := ParseFileConfigYAML([]byte(raw)) + if err == nil { + t.Fatalf("expected duration parse error") + } + if !strings.Contains(err.Error(), "whole seconds") { + t.Fatalf("unexpected error: %v", err) + } +} + +func TestLoadFileConfigReadsFromPath(t *testing.T) { + p := writeTempFileConfig(t, "version: 1\n") + cfg, err := LoadFileConfig(p) + if err != nil { + t.Fatalf("LoadFileConfig error: %v", err) + } + if cfg.Version != 1 { + t.Fatalf("expected version 1, got %d", cfg.Version) + } +} + +func writeTempFileConfig(t *testing.T, contents string) string { + t.Helper() + dir := t.TempDir() + path := dir + "/config.yaml" + if err := os.WriteFile(path, []byte(contents), 0o600); err != nil { + t.Fatalf("write config file: %v", err) + } + return path +} diff --git a/internal/core/diagnostics/run_dir.go b/internal/core/diagnostics/run_dir.go index 10355fe..50a00ca 100644 --- a/internal/core/diagnostics/run_dir.go +++ b/internal/core/diagnostics/run_dir.go @@ -53,6 +53,8 @@ type InvocationMetadata struct { GlossaryPath string `json:"glossary_path"` OutputPath string `json:"output_path,omitempty"` ReportJSONPath string `json:"report_json_path,omitempty"` + ConfigPath string `json:"config_path,omitempty"` + ConfigSource string `json:"config_source,omitempty"` TranscriptDescription string `json:"transcript_description,omitempty"` Modules []string `json:"modules"` RunID string `json:"run_id"`