Add configuration path explanations

This commit is contained in:
2026-09-26 18:17:57 +00:00
parent 3d7c9aa324
commit 41304e4eea
7 changed files with 131 additions and 6 deletions

View File

@@ -28,6 +28,7 @@ go run ./cmd/scriptorium render \
- `scriptorium config validate`: validate application configuration without building an engine. - `scriptorium config validate`: validate application configuration without building an engine.
- `scriptorium config show`: show the resolved application configuration. - `scriptorium config show`: show the resolved application configuration.
- `scriptorium config sources`: show the ordered configuration sources. - `scriptorium config sources`: show the ordered configuration sources.
- `scriptorium config explain`: explain one effective or historical configuration path.
All commands accept `--config <path>` and `--config-overlay <name>` and reject All commands accept `--config <path>` and `--config-overlay <name>` and reject
positional arguments. `--config-overlay` selects one application configuration positional arguments. `--config-overlay` selects one application configuration
@@ -226,6 +227,7 @@ controls are shown as zero values when Promptkit leaves them unspecified.
scriptorium config validate [flags] scriptorium config validate [flags]
scriptorium config show [flags] scriptorium config show [flags]
scriptorium config sources [flags] scriptorium config sources [flags]
scriptorium config explain --path JSON_POINTER [flags]
``` ```
Each accepts `--config`, `--config-overlay`, `--prompt-dir`, `--profile-dir`, Each accepts `--config`, `--config-overlay`, `--prompt-dir`, `--profile-dir`,
@@ -244,10 +246,21 @@ record has `id`, `label`, and `layer`, plus `path` when available; explicit CLI
overrides appear as synthetic sources. Defaults-only input without overrides overrides appear as synthetic sources. Defaults-only input without overrides
produces `[]` for `sources`. produces `[]` for `sources`.
`explain` requires an explicitly supplied `--path` in JSON Pointer syntax.
The empty string selects the root; `/server/addr` selects a field; `~1` and
`~0` escape `/` and `~` inside mapping keys. Numeric tokens are interpreted
according to the containing mapping or sequence. Dotted paths, file paths,
and URL fragments are not accepted. ConfigKit writes a human-readable account
of the effective value and its chronological default, base, overlay, CLI,
and normalization history, including removed historical values. This is not a
JSON schema or a byte-for-byte stable prose format. Unknown and malformed paths
exit 1 without successful explanation output.
Diagnostics and warnings go to stderr, never into successful stdout. Parse, Diagnostics and warnings go to stderr, never into successful stdout. Parse,
load, output, and diagnostic-write failures exit 1. Ordinary output writes may load, output, and diagnostic-write failures exit 1. Ordinary output writes may
leave partial data on failure. The inspection output can contain paths, leave partial data on failure. The inspection output can contain paths,
identifiers, `api_key_env` reference names, and ordinary `extra_params` values; identifiers, `api_key_env` reference names, and ordinary `extra_params` values,
including displaced historical values in explanations;
protect it accordingly. Raw API-key values are not read or emitted by these protect it accordingly. Raw API-key values are not read or emitted by these
commands. commands.

View File

@@ -95,7 +95,8 @@ The size fields must be zero or greater. The [HTTP API](api.md) defines how
each limit is enforced and reported. `server.artifact_root` configures an HTTP each limit is enforced and reported. `server.artifact_root` configures an HTTP
deployment boundary; see [operations](operations.md) for deployment handling. deployment boundary; see [operations](operations.md) for deployment handling.
The `config validate`, `config show`, and `config sources` commands inspect this The `config validate`, `config show`, `config sources`, and `config explain`
commands inspect this
application configuration without creating a Promptkit engine. `show` is a application configuration without creating a Promptkit engine. `show` is a
resolved inspection snapshot, not necessarily a reloadable file. Application resolved inspection snapshot, not necessarily a reloadable file. Application
validation does not check framework sources, backend connectivity, or credential validation does not check framework sources, backend connectivity, or credential
@@ -103,6 +104,12 @@ availability. `api_key_env` is an environment-variable *name* and remains
visible in inspection; its environment value is never read by ConfigKit. visible in inspection; its environment value is never read by ConfigKit.
Ordinary backend `extra_params` values also remain visible, so keep secrets out Ordinary backend `extra_params` values also remain visible, so keep secrets out
of those fields and protect inspection output as potentially sensitive. of those fields and protect inspection output as potentially sensitive.
`config explain --path /server/addr` uses JSON Pointer syntax to show the
selected field's current value and history from defaults, files, overlays,
explicit CLI overrides, and normalization. `--path ''` selects the root.
History includes replaced and removed values; `show` presents the final
resolved snapshot. Neither renderer looks up credential bytes, and neither
claims to identify arbitrary secrets in unclassified parameter values.
## Custom Backends ## Custom Backends

View File

@@ -99,6 +99,10 @@ Before changing production inputs, profiles, or schemas:
policy. policy.
Inspection output can include configuration paths, names, and ordinary backend Inspection output can include configuration paths, names, and ordinary backend
parameters. `config explain --path` uses JSON Pointer paths and can show
displaced historical values as well as the final value. Protect both the
selected paths and the output, especially when backend parameters contain
sensitive identifiers; ConfigKit does not infer arbitrary secrets in those
parameters. Apply parameters. Apply
access and retention controls; the application stores only API-key environment access and retention controls; the application stores only API-key environment
reference names, not the resolved credential bytes. reference names, not the resolved credential bytes.

View File

@@ -390,7 +390,11 @@ selected API-key boundary without masking or changing backend parameter data.
## Stage 6: Add Per-Path Explanation And Verify Credential Boundaries ## Stage 6: Add Per-Path Explanation And Verify Credential Boundaries
**Status: Not started.** **Status: Complete.** `config explain` uses required JSON Pointer paths and
ConfigKit's snapshot renderer, including historical and normalized values.
Tests cover root, escaped/numeric keys, unknown paths, writer failures, and
synthetic credential-reference boundaries. Full test, race, vet, build, and
diff checks passed.
Primary files: configuration command implementation/tests and CLI/config docs. Primary files: configuration command implementation/tests and CLI/config docs.

View File

@@ -23,16 +23,19 @@ func configCommand(args []string, stdout, stderr io.Writer) int {
return ExitRuntimeError return ExitRuntimeError
} }
mode := args[0] mode := args[0]
if mode != "validate" && mode != "show" && mode != "sources" { if mode != "validate" && mode != "show" && mode != "sources" && mode != "explain" {
fmt.Fprintln(stderr, "config parse error: unknown configuration mode") fmt.Fprintln(stderr, "config parse error: unknown configuration mode")
return ExitRuntimeError return ExitRuntimeError
} }
fs := flag.NewFlagSet("config "+mode, flag.ContinueOnError) fs := flag.NewFlagSet("config "+mode, flag.ContinueOnError)
fs.SetOutput(io.Discard) fs.SetOutput(io.Discard)
var path, overlay string var path, overlay, pointer string
registerConfigSelectionFlags(fs, &path, &overlay) registerConfigSelectionFlags(fs, &path, &overlay)
registerConfigurationOverrides(fs) registerConfigurationOverrides(fs)
if mode == "explain" {
fs.StringVar(&pointer, "path", "", "configuration path as a JSON Pointer")
}
if err := fs.Parse(args[1:]); err != nil { if err := fs.Parse(args[1:]); err != nil {
writeApplicationError(stderr, "config parse", err) writeApplicationError(stderr, "config parse", err)
return ExitRuntimeError return ExitRuntimeError
@@ -41,6 +44,19 @@ func configCommand(args []string, stdout, stderr io.Writer) int {
fmt.Fprintln(stderr, "config parse error: positional arguments are not accepted") fmt.Fprintln(stderr, "config parse error: positional arguments are not accepted")
return ExitRuntimeError return ExitRuntimeError
} }
var explanationPath configkit.Path
if mode == "explain" {
if !flagWasSet(fs, "path") {
fmt.Fprintln(stderr, "config parse error: --path is required")
return ExitRuntimeError
}
var err error
explanationPath, err = configkit.ParsePath(pointer)
if err != nil {
fmt.Fprintln(stderr, "config parse error: --path must be a JSON Pointer")
return ExitRuntimeError
}
}
result, err := appconfig.Load(appconfig.LoadOptions{ result, err := appconfig.Load(appconfig.LoadOptions{
Path: path, Explicit: flagWasSet(fs, "config"), Overlay: overlay, Path: path, Explicit: flagWasSet(fs, "config"), Overlay: overlay,
@@ -66,6 +82,8 @@ func configCommand(args []string, stdout, stderr io.Writer) int {
}) })
} }
err = json.NewEncoder(stdout).Encode(sources) err = json.NewEncoder(stdout).Encode(sources)
case "explain":
err = result.WriteExplanation(stdout, explanationPath)
} }
if err != nil { if err != nil {
fmt.Fprintf(stderr, "config output error: %v\n", err) fmt.Fprintf(stderr, "config output error: %v\n", err)

View File

@@ -126,3 +126,82 @@ func TestConfigCommandFailuresDoNotProduceSuccessfulOutput(t *testing.T) {
t.Fatalf("source writer error returned %d", code) t.Fatalf("source writer error returned %d", code)
} }
} }
func TestConfigExplainUsesResolvedHistoryAndKeepsParametersVisible(t *testing.T) {
dir := t.TempDir()
if err := os.MkdirAll(filepath.Join(dir, "overlays"), 0o755); err != nil {
t.Fatal(err)
}
base := filepath.Join(dir, "config.yml")
for path, body := range map[string]string{
base: `prompt_dir: " ./prompts/../prompts "
server:
addr: :8080
backends:
"team/blue~1":
endpoint: http://127.0.0.1:1/v1
api_key_env: SCRIPTORIUM_EXPLAIN_KEY
extra_params:
"0": base-parameter
`,
filepath.Join(dir, "overlays", "local.yml"): `server:
addr: :9090
backends:
"team/blue~1":
extra_params:
"0": overlay-parameter
`,
} {
if err := os.WriteFile(path, []byte(body), 0o600); err != nil {
t.Fatal(err)
}
}
const keyValue = "synthetic-key-bytes-must-not-appear"
t.Setenv("SCRIPTORIUM_EXPLAIN_KEY", keyValue)
args := []string{"--config", base, "--config-overlay", "local", "--addr", ":9090"}
for _, tc := range []struct {
pointer string
want []string
}{
{pointer: "/server/addr", want: []string{":8080", ":9090", "--addr"}},
{pointer: "/prompt_dir", want: []string{"normalized application path", "prompts"}},
{pointer: "/backends/team~1blue~01/extra_params/0", want: []string{"base-parameter", "overlay-parameter"}},
{pointer: "/backends/team~1blue~01/api_key_env", want: []string{"SCRIPTORIUM_EXPLAIN_KEY"}},
{pointer: "", want: []string{"base-parameter", "overlay-parameter", "SCRIPTORIUM_EXPLAIN_KEY"}},
} {
code, stdout, stderr := runCLICommand(t, configCommand, append([]string{"explain", "--path", tc.pointer}, args...))
if code != ExitOK || stderr != "" || strings.Contains(stdout, keyValue) {
t.Fatalf("explain %q failed or disclosed key: code=%d out=%q err=%q", tc.pointer, code, stdout, stderr)
}
for _, want := range tc.want {
if !strings.Contains(stdout, want) {
t.Fatalf("explain %q lacks %q: %q", tc.pointer, want, stdout)
}
}
}
if err := os.Unsetenv("SCRIPTORIUM_EXPLAIN_KEY"); err != nil {
t.Fatal(err)
}
code, stdout, stderr := runCLICommand(t, configCommand, append([]string{"explain", "--path", "/backends/team~1blue~01/api_key_env"}, args...))
if code != ExitOK || stderr != "" || !strings.Contains(stdout, "SCRIPTORIUM_EXPLAIN_KEY") || strings.Contains(stdout, keyValue) {
t.Fatalf("absent credential changed reference explanation: code=%d out=%q err=%q", code, stdout, stderr)
}
}
func TestConfigExplainRejectsUnknownOrMalformedPaths(t *testing.T) {
path := writeAppConfigFile(t, "{}\n")
for _, args := range [][]string{
{"explain", "--config", path},
{"explain", "--config", path, "--path", "not-a-pointer"},
{"explain", "--config", path, "--path", "/missing"},
} {
code, stdout, stderr := runCLICommand(t, configCommand, args)
if code != ExitRuntimeError || stdout != "" || stderr == "" {
t.Fatalf("invalid explanation path accepted: args=%q code=%d out=%q err=%q", args, code, stdout, stderr)
}
}
var stderr bytes.Buffer
if code := configCommand([]string{"explain", "--config", path, "--path", ""}, errorOutputWriter{}, &stderr); code != ExitRuntimeError || !strings.Contains(stderr.String(), "output error") {
t.Fatalf("explanation writer failure not reported: code=%d err=%q", code, stderr.String())
}
}

View File

@@ -760,5 +760,5 @@ func printUsage(w io.Writer) {
fmt.Fprintf(w, " serve: scriptorium serve [--config PATH] [--config-overlay NAME] [--addr %s] [--prompt-dir DIR] [--profile-dir DIR] [--schema-dir DIR] [--artifact-root DIR] [--max-request-bytes N] [--max-artifact-bytes N] [--max-response-bytes N]\n", defaults.HTTPAddrDefault) fmt.Fprintf(w, " serve: scriptorium serve [--config PATH] [--config-overlay NAME] [--addr %s] [--prompt-dir DIR] [--profile-dir DIR] [--schema-dir DIR] [--artifact-root DIR] [--max-request-bytes N] [--max-artifact-bytes N] [--max-response-bytes N]\n", defaults.HTTPAddrDefault)
fmt.Fprintln(w, " inspect prompt: scriptorium inspect prompt --prompt ID [--prompt-version VERSION] [--config PATH] [--config-overlay NAME] [--prompt-dir DIR] [--format text|json] [--out path]") fmt.Fprintln(w, " inspect prompt: scriptorium inspect prompt --prompt ID [--prompt-version VERSION] [--config PATH] [--config-overlay NAME] [--prompt-dir DIR] [--format text|json] [--out path]")
fmt.Fprintln(w, " inspect profile: scriptorium inspect profile --profile ID [--config PATH] [--config-overlay NAME] [--profile-dir DIR] [--format text|json] [--out path]") fmt.Fprintln(w, " inspect profile: scriptorium inspect profile --profile ID [--config PATH] [--config-overlay NAME] [--profile-dir DIR] [--format text|json] [--out path]")
fmt.Fprintln(w, " config: scriptorium config <validate|show|sources> [--config PATH] [--config-overlay NAME] [application overrides]") fmt.Fprintln(w, " config: scriptorium config <validate|show|sources|explain> [--config PATH] [--config-overlay NAME] [application overrides] [--path /pointer for explain]")
} }