Compare commits
3 Commits
d53a64650b
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
| 40cbf42438 | |||
| 44a318694b | |||
| 9389f69eb0 |
@@ -374,6 +374,7 @@ Rules:
|
||||
| `pipeline.render.include_segment_ids` | bool | No | `true` |
|
||||
| `pipeline.render.include_metadata` | bool | No | `false` |
|
||||
| `pipeline.scriptorium.binary` | string | No | `scriptorium` |
|
||||
| `pipeline.scriptorium.credentials[]` | list[secret reference] | No | empty; exact environment-variable names forwarded to Scriptorium |
|
||||
| `pipeline.scriptorium.config_path` | string | No | empty |
|
||||
| `pipeline.scriptorium.timeout` | duration | No | `10m` |
|
||||
| `pipeline.scriptorium.render_debug` | bool | No | `false` |
|
||||
@@ -544,6 +545,25 @@ contract-incompatible lane fails extraction. See the
|
||||
current ten-lane D&D mapping and the [Notarius contract](./integrations/notarius.md)
|
||||
for compatibility ownership.
|
||||
|
||||
### Scriptorium Credentials
|
||||
|
||||
Set `scriptorium.credentials: [OPENROUTER_API_KEY]` in the pipeline file to
|
||||
forward that provider key to Scriptorium. List additional provider names only
|
||||
when the selected downstream configuration needs them. The same list applies
|
||||
to trim bounds generation, analysis (including character families), and their
|
||||
render-debug calls. Scriptorium retains provider/profile selection; Narratio
|
||||
does not pass a model-request `--api-key-env` override.
|
||||
|
||||
As with Audita and Notarius, names are ConfigKit secret references resolved at
|
||||
pipeline load time using the existing environment/secrets bootstrap precedence,
|
||||
even when no Scriptorium stage executes. Omitted, null, or empty lists forward
|
||||
nothing. An overlay replaces the entire list; `[]` clears it. Captured values
|
||||
are forwarded under their exact names and registered for diagnostic redaction.
|
||||
Inspection and generated metadata contain names only. Credential changes do
|
||||
not invalidate completed trim or analysis work; rotating values also leaves
|
||||
the effective configuration digest unchanged. Other environment variables
|
||||
retain the shared minimal subprocess inheritance rules.
|
||||
|
||||
### Scriptorium Artifact Entries
|
||||
|
||||
For each `pipeline.scriptorium.artifacts.<name>`:
|
||||
|
||||
@@ -21,7 +21,7 @@ Both request types carry:
|
||||
- output path;
|
||||
- timeout;
|
||||
- generated config + stdout/stderr log paths;
|
||||
- optional API-key env var name;
|
||||
- optional list of already-resolved credential references;
|
||||
- optional working directory.
|
||||
|
||||
## Result Contract
|
||||
@@ -39,7 +39,16 @@ Request validation fails for:
|
||||
- non-positive timeout;
|
||||
- empty input/var names;
|
||||
- empty input path values;
|
||||
- missing required credential env var when `APIKeyEnv` is set.
|
||||
- unresolved or empty selected credential references.
|
||||
|
||||
Both modes use the shared credential forwarding used by Audita and Notarius:
|
||||
selected names and captured values augment the minimal child environment, and
|
||||
all selected values are registered for captured-stream redaction. The adapter
|
||||
does not reread the parent environment or override Scriptorium's provider
|
||||
selection with `--api-key-env`. Generated invocation records and result metadata
|
||||
contain `credential_env_names`, never credential values. This does not scan or
|
||||
redact files written directly by Scriptorium. Selection semantics belong in
|
||||
[Scriptorium credentials](../config.md#scriptorium-credentials).
|
||||
|
||||
Run behavior:
|
||||
- subprocess errors propagate with context;
|
||||
|
||||
@@ -69,13 +69,13 @@ It also serializes an effective resume request as `--resume` and a nonempty
|
||||
validated chunk mode as `--chunk_cache <mode>`. The extract stage, not the
|
||||
adapter, resolves force precedence before constructing the request.
|
||||
Its request may also carry already-resolved ConfigKit secret references.
|
||||
Audita's request carries the same kind of list. Both adapters use the shared
|
||||
Audita and Scriptorium requests carry the same kind of list. All three adapters use the shared
|
||||
subprocess helper to convert their selected references into exact-name child
|
||||
environment overrides and sensitive-name registration. That helper rejects
|
||||
unresolved or empty programmatic references, but neither adapter rereads the
|
||||
unresolved or empty programmatic references, but no adapter rereads the
|
||||
parent environment or inspects downstream configuration to discover names.
|
||||
Empty lists add no credential overrides. See the public [Audita](../integrations/audita.md)
|
||||
and [Notarius](../integrations/notarius.md) contracts for selection and
|
||||
[Notarius](../integrations/notarius.md), and [Scriptorium](../integrations/scriptorium.md) contracts for selection and
|
||||
downstream-file boundaries.
|
||||
|
||||
Application-owned object-store construction loads configured filesystem
|
||||
|
||||
@@ -11,6 +11,7 @@ See the [release procedure](../release.md) for creating a candidate and tag.
|
||||
When asynchronous publication succeeds, the corresponding Gitea release is the
|
||||
canonical source for downloadable binaries and checksums.
|
||||
|
||||
- [v3.1.0](v3.1.0.md)
|
||||
- [v3.0.0](v3.0.0.md)
|
||||
- [v2.2.1](v2.2.1.md)
|
||||
- [v2.2.0](v2.2.0.md)
|
||||
|
||||
40
docs/releases/v3.1.0.md
Normal file
40
docs/releases/v3.1.0.md
Normal file
@@ -0,0 +1,40 @@
|
||||
# Narratio v3.1.0
|
||||
|
||||
This minor release adds explicit provider-credential forwarding to Scriptorium
|
||||
for trim bounds generation and artifact analysis.
|
||||
|
||||
## Compatibility and upgrade
|
||||
|
||||
Existing pipelines remain valid. An omitted, null, or empty
|
||||
`pipeline.scriptorium.credentials` list continues to forward no provider keys.
|
||||
When Scriptorium needs a credential, provision its environment variable through
|
||||
Narratio's existing environment or secrets-directory mechanism and list the
|
||||
exact name in the pipeline, for example:
|
||||
|
||||
```yaml
|
||||
scriptorium:
|
||||
credentials: [OPENROUTER_API_KEY]
|
||||
```
|
||||
|
||||
Selected references must resolve to nonempty values when Narratio loads the
|
||||
pipeline, even if the current command will not run Scriptorium. A Narratio
|
||||
overlay replaces the complete list, and `[]` clears it. After correcting a
|
||||
failed trim or analysis invocation, retry normally; successful work need not be
|
||||
forced solely because a credential was added or rotated. See
|
||||
[Scriptorium credentials](../config.md#scriptorium-credentials) and the
|
||||
[401 troubleshooting guide](../troubleshooting.md#scriptorium-returns-http-401-during-trim-or-analyze).
|
||||
|
||||
## Changes
|
||||
|
||||
- Forward selected, already-resolved credential names and values to Scriptorium
|
||||
for trim, analysis, character-family generation, and render-debug calls.
|
||||
Scriptorium still owns provider and profile selection; Narratio does not add
|
||||
a model-request `--api-key-env` override.
|
||||
- Register selected values for captured subprocess-output and failure-diagnostic
|
||||
redaction. Inspection and invocation metadata contain credential names, never
|
||||
their values. This protection does not scan files Scriptorium writes directly.
|
||||
- Reuse the credential-forwarding boundary shared with Audita and Notarius.
|
||||
Retire the completed Scriptorium credential-forwarding bug report from the
|
||||
source roadmap; its history remains in Git.
|
||||
|
||||
[COPYRIGHT](https://gitea.maximumdirect.net/eric/narratio/src/tag/v3.1.0/COPYRIGHT) | [LICENSE](https://gitea.maximumdirect.net/eric/narratio/src/tag/v3.1.0/LICENSE) | [ATTRIBUTION](https://gitea.maximumdirect.net/eric/narratio/src/tag/v3.1.0/docs/oss/attribution.md)
|
||||
@@ -739,6 +739,20 @@ manifest-owned.
|
||||
Relevant references: [Operations: Stage Execution and Continuation Behavior](./operations.md#stage-execution-and-continuation-behavior)
|
||||
and [Manifest Internals](./internal/manifest.md#transcribe-owned-speaker-state).
|
||||
|
||||
## Scriptorium returns HTTP 401 during trim or analyze
|
||||
|
||||
Check that Narratio's pipeline selects the credential names required by
|
||||
Scriptorium, for example `scriptorium.credentials: [OPENROUTER_API_KEY]`.
|
||||
Provision the values through the existing environment or secrets directory;
|
||||
exporting a key without selecting it does not forward it to the child process.
|
||||
See [Scriptorium credentials](config.md#scriptorium-credentials).
|
||||
|
||||
Once corrected, retry the failed run normally; forcing previously successful
|
||||
transcription or analysis is unnecessary. If authentication still fails, check
|
||||
the selected Scriptorium backend, credential validity, and provider access
|
||||
without printing keys. A 401 can indicate an invalid key as well as a missing
|
||||
one.
|
||||
|
||||
## Scriptorium private input changed without a rerun
|
||||
|
||||
Symptom:
|
||||
|
||||
@@ -217,6 +217,9 @@ notarius:
|
||||
|
||||
scriptorium:
|
||||
binary: scriptorium
|
||||
# Select provider names provisioned through Narratio's environment/secrets.
|
||||
# Used for trim, analysis, and render-debug calls; no provider override.
|
||||
# credentials: [OPENROUTER_API_KEY, RAKESTRAWHOME_API_KEY]
|
||||
config_path: /usr/local/etc/scriptorium/config.yml
|
||||
timeout: 10m
|
||||
render_debug: false
|
||||
|
||||
@@ -68,6 +68,7 @@ normalize:
|
||||
|
||||
scriptorium:
|
||||
binary: scriptorium
|
||||
# credentials: [OPENROUTER_API_KEY] # Enable when this provider is configured.
|
||||
config_path: /usr/local/etc/scriptorium/config.yml
|
||||
timeout: 10m
|
||||
render_debug: false
|
||||
|
||||
@@ -3,6 +3,7 @@ package scriptorium
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"slices"
|
||||
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/fileops"
|
||||
)
|
||||
@@ -65,6 +66,7 @@ func (f *FakeRunner) RunArtifact(ctx context.Context, req RunArtifactRequest) (A
|
||||
if err := ctx.Err(); err != nil {
|
||||
return ArtifactResult{}, err
|
||||
}
|
||||
req.Credentials = slices.Clone(req.Credentials)
|
||||
f.RunRequests = append(f.RunRequests, req)
|
||||
if f.RunErr != nil {
|
||||
return ArtifactResult{}, f.RunErr
|
||||
@@ -105,6 +107,7 @@ func (f *FakeRunner) RenderArtifact(ctx context.Context, req RenderArtifactReque
|
||||
if err := ctx.Err(); err != nil {
|
||||
return ArtifactResult{}, err
|
||||
}
|
||||
req.Credentials = slices.Clone(req.Credentials)
|
||||
f.RenderRequests = append(f.RenderRequests, req)
|
||||
if f.RenderErr != nil {
|
||||
return ArtifactResult{}, f.RenderErr
|
||||
|
||||
@@ -4,6 +4,8 @@ package scriptorium
|
||||
import (
|
||||
"context"
|
||||
"time"
|
||||
|
||||
configkit "gitea.maximumdirect.net/eric/ConfigKit"
|
||||
)
|
||||
|
||||
const (
|
||||
@@ -32,7 +34,7 @@ type RunArtifactRequest struct {
|
||||
StderrLogPath string
|
||||
GeneratedConfigPath string
|
||||
Timeout time.Duration
|
||||
APIKeyEnv string
|
||||
Credentials []configkit.Secret
|
||||
WorkingDir string
|
||||
}
|
||||
|
||||
@@ -49,7 +51,7 @@ type RenderArtifactRequest struct {
|
||||
StderrLogPath string
|
||||
GeneratedConfigPath string
|
||||
Timeout time.Duration
|
||||
APIKeyEnv string
|
||||
Credentials []configkit.Secret
|
||||
WorkingDir string
|
||||
}
|
||||
|
||||
|
||||
@@ -28,7 +28,12 @@ func (r *SubprocessRunner) RunArtifact(ctx context.Context, req RunArtifactReque
|
||||
if r == nil {
|
||||
return ArtifactResult{}, fmt.Errorf("scriptorium subprocess runner is nil")
|
||||
}
|
||||
credentialPresent, err := validateCommonRunRequest(req.Binary, req.ConfigPath, req.PromptID, req.OutputPath, req.Timeout, req.APIKeyEnv, req.InputPaths, req.Vars)
|
||||
err := validateCommonRunRequest(req.Binary, req.ConfigPath, req.PromptID, req.OutputPath, req.Timeout, req.InputPaths, req.Vars)
|
||||
if err != nil {
|
||||
return ArtifactResult{}, err
|
||||
}
|
||||
|
||||
envOverrides, sensitiveNames, err := subprocess.CredentialEnvironment(req.Credentials)
|
||||
if err != nil {
|
||||
return ArtifactResult{}, err
|
||||
}
|
||||
@@ -46,8 +51,7 @@ func (r *SubprocessRunner) RunArtifact(ctx context.Context, req RunArtifactReque
|
||||
InputPaths: cloneStringMap(req.InputPaths),
|
||||
Vars: cloneStringMap(req.Vars),
|
||||
OutputPath: req.OutputPath,
|
||||
APIKeyEnv: strings.TrimSpace(req.APIKeyEnv),
|
||||
APIKeyEnvPresent: credentialPresent,
|
||||
CredentialNames: sensitiveNames,
|
||||
WorkingDir: req.WorkingDir,
|
||||
RenderFormat: "",
|
||||
RenderPromptStore: false,
|
||||
@@ -56,7 +60,6 @@ func (r *SubprocessRunner) RunArtifact(ctx context.Context, req RunArtifactReque
|
||||
}
|
||||
}
|
||||
|
||||
envOverrides, sensitiveNames := credentialEnvironment(req.APIKeyEnv)
|
||||
runRes, runErr := subprocess.Run(ctx, subprocess.RunRequest{
|
||||
Executable: req.Binary,
|
||||
Args: args,
|
||||
@@ -80,12 +83,11 @@ func (r *SubprocessRunner) RunArtifact(ctx context.Context, req RunArtifactReque
|
||||
PromptID: req.PromptID,
|
||||
ProfileID: req.ProfileID,
|
||||
Metadata: map[string]any{
|
||||
"adapter": "scriptorium_subprocess",
|
||||
"config_path": strings.TrimSpace(req.ConfigPath),
|
||||
"inputs_count": len(req.InputPaths),
|
||||
"vars_count": len(req.Vars),
|
||||
"api_key_env": strings.TrimSpace(req.APIKeyEnv),
|
||||
"api_key_env_present": credentialPresent,
|
||||
"adapter": "scriptorium_subprocess",
|
||||
"config_path": strings.TrimSpace(req.ConfigPath),
|
||||
"inputs_count": len(req.InputPaths),
|
||||
"vars_count": len(req.Vars),
|
||||
"credential_env_names": sensitiveNames,
|
||||
},
|
||||
}
|
||||
|
||||
@@ -114,7 +116,12 @@ func (r *SubprocessRunner) RenderArtifact(ctx context.Context, req RenderArtifac
|
||||
if r == nil {
|
||||
return ArtifactResult{}, fmt.Errorf("scriptorium subprocess runner is nil")
|
||||
}
|
||||
credentialPresent, err := validateCommonRunRequest(req.Binary, req.ConfigPath, req.PromptID, req.OutputPath, req.Timeout, req.APIKeyEnv, req.InputPaths, req.Vars)
|
||||
err := validateCommonRunRequest(req.Binary, req.ConfigPath, req.PromptID, req.OutputPath, req.Timeout, req.InputPaths, req.Vars)
|
||||
if err != nil {
|
||||
return ArtifactResult{}, err
|
||||
}
|
||||
|
||||
envOverrides, sensitiveNames, err := subprocess.CredentialEnvironment(req.Credentials)
|
||||
if err != nil {
|
||||
return ArtifactResult{}, err
|
||||
}
|
||||
@@ -132,8 +139,7 @@ func (r *SubprocessRunner) RenderArtifact(ctx context.Context, req RenderArtifac
|
||||
InputPaths: cloneStringMap(req.InputPaths),
|
||||
Vars: cloneStringMap(req.Vars),
|
||||
OutputPath: req.OutputPath,
|
||||
APIKeyEnv: strings.TrimSpace(req.APIKeyEnv),
|
||||
APIKeyEnvPresent: credentialPresent,
|
||||
CredentialNames: sensitiveNames,
|
||||
WorkingDir: req.WorkingDir,
|
||||
RenderFormat: "json",
|
||||
RenderPromptStore: false,
|
||||
@@ -142,7 +148,6 @@ func (r *SubprocessRunner) RenderArtifact(ctx context.Context, req RenderArtifac
|
||||
}
|
||||
}
|
||||
|
||||
envOverrides, sensitiveNames := credentialEnvironment(req.APIKeyEnv)
|
||||
runRes, runErr := subprocess.Run(ctx, subprocess.RunRequest{
|
||||
Executable: req.Binary,
|
||||
Args: args,
|
||||
@@ -166,13 +171,12 @@ func (r *SubprocessRunner) RenderArtifact(ctx context.Context, req RenderArtifac
|
||||
PromptID: req.PromptID,
|
||||
ProfileID: req.ProfileID,
|
||||
Metadata: map[string]any{
|
||||
"adapter": "scriptorium_subprocess",
|
||||
"config_path": strings.TrimSpace(req.ConfigPath),
|
||||
"inputs_count": len(req.InputPaths),
|
||||
"vars_count": len(req.Vars),
|
||||
"api_key_env": strings.TrimSpace(req.APIKeyEnv),
|
||||
"api_key_env_present": credentialPresent,
|
||||
"format": "json",
|
||||
"adapter": "scriptorium_subprocess",
|
||||
"config_path": strings.TrimSpace(req.ConfigPath),
|
||||
"inputs_count": len(req.InputPaths),
|
||||
"vars_count": len(req.Vars),
|
||||
"credential_env_names": sensitiveNames,
|
||||
"format": "json",
|
||||
},
|
||||
}
|
||||
|
||||
@@ -191,57 +195,39 @@ func validateCommonRunRequest(
|
||||
promptID string,
|
||||
outputPath string,
|
||||
timeout time.Duration,
|
||||
apiKeyEnv string,
|
||||
inputPaths map[string]string,
|
||||
vars map[string]string,
|
||||
) (bool, error) {
|
||||
) error {
|
||||
if strings.TrimSpace(binary) == "" {
|
||||
return false, fmt.Errorf("scriptorium binary is required")
|
||||
return fmt.Errorf("scriptorium binary is required")
|
||||
}
|
||||
if strings.TrimSpace(configPath) == "" && configPath != "" {
|
||||
return false, fmt.Errorf("scriptorium config path must be non-empty when provided")
|
||||
return fmt.Errorf("scriptorium config path must be non-empty when provided")
|
||||
}
|
||||
if strings.TrimSpace(promptID) == "" {
|
||||
return false, fmt.Errorf("scriptorium prompt id is required")
|
||||
return fmt.Errorf("scriptorium prompt id is required")
|
||||
}
|
||||
if strings.TrimSpace(outputPath) == "" {
|
||||
return false, fmt.Errorf("scriptorium output path is required")
|
||||
return fmt.Errorf("scriptorium output path is required")
|
||||
}
|
||||
if timeout <= 0 {
|
||||
return false, fmt.Errorf("scriptorium timeout must be > 0")
|
||||
return fmt.Errorf("scriptorium timeout must be > 0")
|
||||
}
|
||||
for name, path := range inputPaths {
|
||||
if strings.TrimSpace(name) == "" {
|
||||
return false, fmt.Errorf("scriptorium input names must be non-empty")
|
||||
return fmt.Errorf("scriptorium input names must be non-empty")
|
||||
}
|
||||
if strings.TrimSpace(path) == "" {
|
||||
return false, fmt.Errorf("scriptorium input %q path is required", name)
|
||||
return fmt.Errorf("scriptorium input %q path is required", name)
|
||||
}
|
||||
}
|
||||
for name := range vars {
|
||||
if strings.TrimSpace(name) == "" {
|
||||
return false, fmt.Errorf("scriptorium variable names must be non-empty")
|
||||
return fmt.Errorf("scriptorium variable names must be non-empty")
|
||||
}
|
||||
}
|
||||
|
||||
trimmedEnv := strings.TrimSpace(apiKeyEnv)
|
||||
if trimmedEnv == "" {
|
||||
return false, nil
|
||||
}
|
||||
value, ok := os.LookupEnv(trimmedEnv)
|
||||
if !ok || strings.TrimSpace(value) == "" {
|
||||
return false, fmt.Errorf("scriptorium required credential environment variable %s is not set", trimmedEnv)
|
||||
}
|
||||
return true, nil
|
||||
}
|
||||
|
||||
func credentialEnvironment(apiKeyEnv string) (map[string]string, []string) {
|
||||
name := strings.TrimSpace(apiKeyEnv)
|
||||
if name == "" {
|
||||
return nil, nil
|
||||
}
|
||||
value, _ := os.LookupEnv(name)
|
||||
return map[string]string{name: value}, []string{name}
|
||||
return nil
|
||||
}
|
||||
|
||||
func buildRunArgs(req RunArtifactRequest) []string {
|
||||
@@ -258,9 +244,6 @@ func buildRunArgs(req RunArtifactRequest) []string {
|
||||
for _, kv := range sortedKeyValues(req.Vars) {
|
||||
args = append(args, "--var", kv)
|
||||
}
|
||||
if envName := strings.TrimSpace(req.APIKeyEnv); envName != "" {
|
||||
args = append(args, "--api-key-env", envName)
|
||||
}
|
||||
args = append(args, "--timeout", req.Timeout.String())
|
||||
args = append(args, "--out", req.OutputPath)
|
||||
return args
|
||||
@@ -280,9 +263,6 @@ func buildRenderArgs(req RenderArtifactRequest) []string {
|
||||
for _, kv := range sortedKeyValues(req.Vars) {
|
||||
args = append(args, "--var", kv)
|
||||
}
|
||||
if envName := strings.TrimSpace(req.APIKeyEnv); envName != "" {
|
||||
args = append(args, "--api-key-env", envName)
|
||||
}
|
||||
args = append(args, "--timeout", req.Timeout.String())
|
||||
args = append(args, "--format", "json")
|
||||
args = append(args, "--out", req.OutputPath)
|
||||
@@ -316,8 +296,7 @@ type invocationPayload struct {
|
||||
InputPaths map[string]string
|
||||
Vars map[string]string
|
||||
OutputPath string
|
||||
APIKeyEnv string
|
||||
APIKeyEnvPresent bool
|
||||
CredentialNames []string
|
||||
WorkingDir string
|
||||
RenderFormat string
|
||||
RenderPromptStore bool
|
||||
@@ -336,8 +315,7 @@ func writeInvocationConfig(path string, payload invocationPayload) error {
|
||||
"input_paths": payload.InputPaths,
|
||||
"vars": payload.Vars,
|
||||
"output_path": payload.OutputPath,
|
||||
"api_key_env": payload.APIKeyEnv,
|
||||
"api_key_env_present": payload.APIKeyEnvPresent,
|
||||
"credential_env_names": payload.CredentialNames,
|
||||
"working_dir": payload.WorkingDir,
|
||||
"render_format": payload.RenderFormat,
|
||||
"render_prompt_logged": payload.RenderPromptStore,
|
||||
|
||||
@@ -9,6 +9,8 @@ import (
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
configkit "gitea.maximumdirect.net/eric/ConfigKit"
|
||||
)
|
||||
|
||||
func TestSubprocessRunnerRunSuccessBuildsDeterministicArgsAndCapturesLogs(t *testing.T) {
|
||||
@@ -16,6 +18,13 @@ func TestSubprocessRunnerRunSuccessBuildsDeterministicArgsAndCapturesLogs(t *tes
|
||||
t.Setenv("GO_WANT_SCRIPTORIUM_HELPER", "1")
|
||||
t.Setenv("SCRIPTORIUM_HELPER_MODE", "run_success")
|
||||
t.Setenv("OPENAI_KEY_SOURCE", "super-secret")
|
||||
loaded, err := configkit.Load[struct {
|
||||
Credentials []configkit.Secret `yaml:"credentials"`
|
||||
}](configkit.Bytes("test.yml", []byte("credentials: [OPENAI_KEY_SOURCE]\n")))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Setenv("OPENAI_KEY_SOURCE", "changed-after-load")
|
||||
recordPath := filepath.Join(t.TempDir(), "record.json")
|
||||
t.Setenv("SCRIPTORIUM_HELPER_RECORD_PATH", recordPath)
|
||||
|
||||
@@ -34,7 +43,7 @@ func TestSubprocessRunnerRunSuccessBuildsDeterministicArgsAndCapturesLogs(t *tes
|
||||
StderrLogPath: filepath.Join(dir, "logs", "scriptorium.run.stderr.log"),
|
||||
GeneratedConfigPath: filepath.Join(dir, "config", "scriptorium.run.generated.yml"),
|
||||
Timeout: mustParseScriptoriumDuration(t, "2s"),
|
||||
APIKeyEnv: "OPENAI_KEY_SOURCE",
|
||||
Credentials: loaded.Config.Credentials,
|
||||
}
|
||||
writeScriptoriumFile(t, req.InputPaths["transcript"], `{"segments":[]}`)
|
||||
writeScriptoriumFile(t, req.InputPaths["other"], "notes\n")
|
||||
@@ -70,7 +79,6 @@ func TestSubprocessRunnerRunSuccessBuildsDeterministicArgsAndCapturesLogs(t *tes
|
||||
"--input", "transcript=" + req.InputPaths["transcript"],
|
||||
"--var", "campaign_name=Icewind Dale",
|
||||
"--var", "session_id=2026-05-03",
|
||||
"--api-key-env", "OPENAI_KEY_SOURCE",
|
||||
"--timeout", "2s",
|
||||
"--out", req.OutputPath,
|
||||
}
|
||||
@@ -94,13 +102,31 @@ func TestSubprocessRunnerRunExitCodeOneFails(t *testing.T) {
|
||||
t.Setenv("GO_WANT_SCRIPTORIUM_HELPER", "1")
|
||||
t.Setenv("SCRIPTORIUM_HELPER_MODE", "fail_exit1")
|
||||
t.Setenv("SCRIPTORIUM_HELPER_RECORD_PATH", filepath.Join(t.TempDir(), "record.json"))
|
||||
t.Setenv("CUSTOM_PROVIDER_KEY", "synthetic-failure-secret")
|
||||
loaded, loadErr := configkit.Load[struct {
|
||||
Credentials []configkit.Secret `yaml:"credentials"`
|
||||
}](configkit.Bytes("test.yml", []byte("credentials: [CUSTOM_PROVIDER_KEY]\n")))
|
||||
if loadErr != nil {
|
||||
t.Fatal(loadErr)
|
||||
}
|
||||
|
||||
runner := NewSubprocessRunner()
|
||||
req := runReqForTest(t, writeScriptoriumHelperWrapper(t))
|
||||
req.Credentials = loaded.Config.Credentials
|
||||
res, err := runner.RunArtifact(context.Background(), req)
|
||||
if err == nil {
|
||||
t.Fatal("RunArtifact() error = nil, want non-nil")
|
||||
}
|
||||
if strings.Contains(err.Error(), "synthetic-failure-secret") {
|
||||
t.Fatal("failure leaked credential")
|
||||
}
|
||||
data, readErr := os.ReadFile(req.StderrLogPath)
|
||||
if readErr != nil {
|
||||
t.Fatal(readErr)
|
||||
}
|
||||
if strings.Contains(string(data), "synthetic-failure-secret") || !strings.Contains(string(data), "<redacted>") {
|
||||
t.Fatal("failure log did not redact selected credential")
|
||||
}
|
||||
if res.ExitCode != 1 {
|
||||
t.Fatalf("ExitCode = %d, want 1", res.ExitCode)
|
||||
}
|
||||
@@ -245,6 +271,7 @@ func TestScriptoriumSubprocessHelper(t *testing.T) {
|
||||
os.Exit(0)
|
||||
case "fail_exit1":
|
||||
_, _ = os.Stderr.WriteString("scriptorium helper failure\n")
|
||||
_, _ = os.Stderr.WriteString(os.Getenv("CUSTOM_PROVIDER_KEY"))
|
||||
os.Exit(1)
|
||||
case "fail_exit2_with_output":
|
||||
writeScriptoriumHelperFile(outputPath, "validation failed artifact\n")
|
||||
|
||||
@@ -52,6 +52,18 @@ seriatim:
|
||||
audita:
|
||||
binary: `+fixture+`
|
||||
credentials: [`+sharedProviderName+`, `+auditaOnlyName+`]
|
||||
scriptorium:
|
||||
binary: `+fixture+`
|
||||
credentials: [`+sharedProviderName+`]
|
||||
render_debug: true
|
||||
artifacts:
|
||||
recap:
|
||||
enabled: true
|
||||
prompt_id: recap
|
||||
output_path: artifacts/recap.md
|
||||
inputs:
|
||||
transcript:
|
||||
source: narratio.transcript.final_trimmed
|
||||
notarius:
|
||||
enabled: true
|
||||
binary: `+fixture+`
|
||||
@@ -132,6 +144,12 @@ inputs:
|
||||
t.Fatalf("extract did not execute: %q", stdout.String())
|
||||
}
|
||||
redactedLogs := 0
|
||||
stdout.Reset()
|
||||
stderr.Reset()
|
||||
if code := Execute(append([]string{"analyze", "2026-05-03"}, args...), &stdout, &stderr); code != 0 {
|
||||
t.Fatalf("analyze exit = %d; stderr=%q", code, stderr.String())
|
||||
}
|
||||
assertNoProviderValues(t, stdout.Bytes(), stderr.Bytes())
|
||||
if err := filepath.WalkDir(workRoot, func(path string, entry os.DirEntry, walkErr error) error {
|
||||
if walkErr != nil || entry.IsDir() {
|
||||
return walkErr
|
||||
@@ -147,8 +165,8 @@ inputs:
|
||||
}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if redactedLogs < 2 {
|
||||
t.Fatalf("redacted subprocess logs = %d, want Audita and Notarius", redactedLogs)
|
||||
if redactedLogs < 4 {
|
||||
t.Fatalf("redacted subprocess logs = %d, want Audita, Notarius and Scriptorium run/render", redactedLogs)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -193,6 +211,24 @@ func TestProviderCredentialProcessFixture(t *testing.T) {
|
||||
_, _ = os.Stderr.WriteString("shared credential not forwarded\n")
|
||||
os.Exit(2)
|
||||
}
|
||||
if appSeriatimFlagValue(args, "--prompt") != "" {
|
||||
if _, present := os.LookupEnv(auditaOnlyName); present {
|
||||
os.Exit(2)
|
||||
}
|
||||
if appSeriatimFlagValue(args, "--api-key-env") != "" {
|
||||
os.Exit(2)
|
||||
}
|
||||
output := appSeriatimFlagValue(args, "--out")
|
||||
if err := os.MkdirAll(filepath.Dir(output), 0o755); err != nil {
|
||||
os.Exit(2)
|
||||
}
|
||||
if err := os.WriteFile(output, []byte("{}\n"), 0o644); err != nil {
|
||||
os.Exit(2)
|
||||
}
|
||||
_, _ = os.Stdout.WriteString(sharedProviderKey + "\n")
|
||||
_, _ = os.Stderr.WriteString(sharedProviderKey + "\n")
|
||||
os.Exit(0)
|
||||
}
|
||||
switch args[0] {
|
||||
case "process":
|
||||
if os.Getenv(auditaOnlyName) != auditaOnlyKey {
|
||||
|
||||
@@ -220,6 +220,7 @@ type RenderConfig struct {
|
||||
|
||||
// ScriptoriumConfig configures Scriptorium-backed artifact generation.
|
||||
type ScriptoriumConfig struct {
|
||||
Credentials []configkit.Secret `yaml:"credentials,omitempty" json:"credentials,omitempty"`
|
||||
Binary string `yaml:"binary" json:"binary"`
|
||||
ConfigPath string `yaml:"config_path" json:"config_path"`
|
||||
Timeout string `yaml:"timeout" json:"timeout"`
|
||||
|
||||
@@ -73,6 +73,24 @@ func (cfg NotariusConfig) MarshalYAML() (any, error) {
|
||||
}, nil
|
||||
}
|
||||
|
||||
// MarshalYAML exposes only public Scriptorium credential reference names.
|
||||
func (cfg ScriptoriumConfig) MarshalYAML() (any, error) {
|
||||
type safeScriptorium struct {
|
||||
Binary string `yaml:"binary"`
|
||||
ConfigPath string `yaml:"config_path"`
|
||||
Timeout string `yaml:"timeout"`
|
||||
RenderDebug bool `yaml:"render_debug"`
|
||||
Credentials []string `yaml:"credentials,omitempty"`
|
||||
Artifacts map[string]ScriptoriumArtifactConfig `yaml:"artifacts"`
|
||||
ArtifactFamilies map[string]ScriptoriumArtifactFamilyConfig `yaml:"artifact_families"`
|
||||
}
|
||||
return safeScriptorium{
|
||||
Binary: cfg.Binary, ConfigPath: cfg.ConfigPath, Timeout: cfg.Timeout,
|
||||
RenderDebug: cfg.RenderDebug, Credentials: secretReferenceNames(cfg.Credentials),
|
||||
Artifacts: cfg.Artifacts, ArtifactFamilies: cfg.ArtifactFamilies,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func secretReferenceNames(secrets []configkit.Secret) []string {
|
||||
if secrets == nil {
|
||||
return nil
|
||||
|
||||
@@ -30,6 +30,7 @@ type effectivePipelineProjection struct {
|
||||
}
|
||||
|
||||
type effectiveScriptoriumConfig struct {
|
||||
Credentials []string `yaml:"credentials,omitempty" json:"credentials,omitempty"`
|
||||
Binary string `yaml:"binary" json:"binary"`
|
||||
ConfigPath string `yaml:"config_path" json:"config_path"`
|
||||
Timeout string `yaml:"timeout" json:"timeout"`
|
||||
@@ -52,7 +53,8 @@ func projectEffectivePipeline(cfg *PipelineConfig, logicalNotarius bool) effecti
|
||||
}
|
||||
if cfg.Scriptorium != nil {
|
||||
projection.Scriptorium = &effectiveScriptoriumConfig{
|
||||
Binary: cfg.Scriptorium.Binary, ConfigPath: cfg.Scriptorium.ConfigPath,
|
||||
Credentials: secretReferenceNames(cfg.Scriptorium.Credentials),
|
||||
Binary: cfg.Scriptorium.Binary, ConfigPath: cfg.Scriptorium.ConfigPath,
|
||||
Timeout: cfg.Scriptorium.Timeout, RenderDebug: cfg.Scriptorium.RenderDebug,
|
||||
Artifacts: cfg.Scriptorium.Artifacts,
|
||||
}
|
||||
|
||||
65
internal/config/scriptorium_credentials_test.go
Normal file
65
internal/config/scriptorium_credentials_test.go
Normal file
@@ -0,0 +1,65 @@
|
||||
package config
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"gopkg.in/yaml.v3"
|
||||
)
|
||||
|
||||
func TestScriptoriumCredentialsComposeAndSerializeSafely(t *testing.T) {
|
||||
t.Setenv("OPENROUTER_API_KEY", "synthetic-first-secret")
|
||||
t.Setenv("CUSTOM_PROVIDER_KEY", "synthetic-second-secret")
|
||||
dir := t.TempDir()
|
||||
root := writePipelineFixture(t, dir, "pipeline.yml", "whisperx:\n transcribe_url: https://example.test/transcribe\nscriptorium:\n credentials: [OPENROUTER_API_KEY]\n")
|
||||
base, err := LoadPipeline(root)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, marshal := range []func() ([]byte, error){
|
||||
func() ([]byte, error) { return yaml.Marshal(base) },
|
||||
func() ([]byte, error) { return MarshalEffectivePipeline(base) },
|
||||
} {
|
||||
data, err := marshal()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !strings.Contains(string(data), "OPENROUTER_API_KEY") || strings.Contains(string(data), "synthetic-first-secret") {
|
||||
t.Fatal("unsafe or missing credential projection")
|
||||
}
|
||||
}
|
||||
t.Setenv("OPENROUTER_API_KEY", "rotated-secret")
|
||||
rotated, err := LoadPipeline(root)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if EffectivePipelineDigest(base) != EffectivePipelineDigest(rotated) {
|
||||
t.Fatal("rotation changed effective digest")
|
||||
}
|
||||
for _, tc := range []struct{ name, value, want string }{
|
||||
{"replace", "[CUSTOM_PROVIDER_KEY]", "CUSTOM_PROVIDER_KEY"},
|
||||
{"clear", "[]", ""}, {"null", "null", ""},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
writePipelineFixture(t, dir, "overlays/"+tc.name+".yml", "scriptorium:\n credentials: "+tc.value+"\n")
|
||||
loaded, err := LoadPipelineWithOptions(root, PipelineLoadOptions{Overlay: &tc.name})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
creds := loaded.Scriptorium.Credentials
|
||||
if tc.want == "" {
|
||||
if len(creds) != 0 {
|
||||
t.Fatal("credentials not cleared")
|
||||
}
|
||||
return
|
||||
}
|
||||
if len(creds) != 1 || creds[0].EnvName() != tc.want || creds[0].Value() != "synthetic-second-secret" {
|
||||
t.Fatal("overlay did not replace resolved credentials")
|
||||
}
|
||||
})
|
||||
}
|
||||
t.Setenv("OPENROUTER_API_KEY", "")
|
||||
if _, err := LoadPipeline(root); err == nil {
|
||||
t.Fatal("empty selected credential accepted")
|
||||
}
|
||||
}
|
||||
@@ -197,6 +197,7 @@ func invokeAnalyzeRender(
|
||||
req := scriptorium.RenderArtifactRequest{
|
||||
Binary: execution.Env.Config.Pipeline.Scriptorium.Binary,
|
||||
ConfigPath: execution.Env.Config.Pipeline.Scriptorium.ConfigPath,
|
||||
Credentials: execution.Env.Config.Pipeline.Scriptorium.Credentials,
|
||||
PromptID: prepared.promptID,
|
||||
ProfileID: prepared.profileID,
|
||||
InputPaths: prepared.inputPaths,
|
||||
@@ -254,6 +255,7 @@ func invokeAnalyzeArtifact(
|
||||
req := scriptorium.RunArtifactRequest{
|
||||
Binary: execution.Env.Config.Pipeline.Scriptorium.Binary,
|
||||
ConfigPath: execution.Env.Config.Pipeline.Scriptorium.ConfigPath,
|
||||
Credentials: execution.Env.Config.Pipeline.Scriptorium.Credentials,
|
||||
PromptID: prepared.promptID,
|
||||
ProfileID: prepared.profileID,
|
||||
InputPaths: prepared.inputPaths,
|
||||
|
||||
@@ -100,6 +100,7 @@ func TestTrimSemanticConfigSensitivity(t *testing.T) {
|
||||
env.Config.Pipeline.Trim.Bounds.Timeout = "1h"
|
||||
env.Config.Pipeline.Trim.Seriatim.Report = boolPointer(false)
|
||||
env.Config.Pipeline.Scriptorium.Binary = "/opt/scriptorium"
|
||||
env.Config.Pipeline.Scriptorium.Credentials = []configkit.Secret{configkit.SecretEnv("OTHER_PROVIDER")}
|
||||
env.Config.Pipeline.Scriptorium.Timeout = "2h"
|
||||
env.Config.Pipeline.Seriatim.Binary = "/opt/seriatim"
|
||||
}},
|
||||
|
||||
@@ -171,6 +171,7 @@ func (trimStage) Run(ctx context.Context, env *Env, m *manifest.Manifest) (*Stag
|
||||
renderReq := scriptorium.RenderArtifactRequest{
|
||||
Binary: env.Config.Pipeline.Scriptorium.Binary,
|
||||
ConfigPath: env.Config.Pipeline.Scriptorium.ConfigPath,
|
||||
Credentials: env.Config.Pipeline.Scriptorium.Credentials,
|
||||
PromptID: boundsCfg.PromptID,
|
||||
ProfileID: boundsCfg.ProfileID,
|
||||
InputPaths: inputPaths,
|
||||
@@ -216,6 +217,7 @@ func (trimStage) Run(ctx context.Context, env *Env, m *manifest.Manifest) (*Stag
|
||||
boundsReq := scriptorium.RunArtifactRequest{
|
||||
Binary: env.Config.Pipeline.Scriptorium.Binary,
|
||||
ConfigPath: env.Config.Pipeline.Scriptorium.ConfigPath,
|
||||
Credentials: env.Config.Pipeline.Scriptorium.Credentials,
|
||||
PromptID: boundsCfg.PromptID,
|
||||
ProfileID: boundsCfg.ProfileID,
|
||||
InputPaths: inputPaths,
|
||||
|
||||
@@ -9,6 +9,7 @@ import (
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
configkit "gitea.maximumdirect.net/eric/ConfigKit"
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/adapters/scriptorium"
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/adapters/seriatim"
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
|
||||
@@ -117,6 +118,7 @@ func TestTrimStageRecordsLogAndGeneratedConfigRefs(t *testing.T) {
|
||||
|
||||
func TestTrimStageRenderDebugDiagnosticsAreNotStageOutputs(t *testing.T) {
|
||||
env, m, scr, _ := setupTrimEnv(t)
|
||||
env.Config.Pipeline.Scriptorium.Credentials = []configkit.Secret{configkit.SecretEnv("TRIM_PROVIDER_KEY")}
|
||||
paths := sessionPathsForEnv(env, m.SessionID)
|
||||
writeFile(t, filepath.Join(paths.TranscriptsDir, "final.json"), `{"segments":[{"id":1},{"id":2}]}`)
|
||||
setCurrentBuiltInEvidence(t, m, "normalize", artifacts.TranscriptOutputKindFinal, artifacts.ArtifactTranscriptFinal, filepath.Join(paths.TranscriptsDir, "final.json"))
|
||||
@@ -133,6 +135,11 @@ func TestTrimStageRenderDebugDiagnosticsAreNotStageOutputs(t *testing.T) {
|
||||
if len(scr.RenderRequests) != 1 {
|
||||
t.Fatalf("scriptorium render requests = %d, want 1", len(scr.RenderRequests))
|
||||
}
|
||||
for _, credentials := range [][]configkit.Secret{scr.RenderRequests[0].Credentials, scr.RunRequests[0].Credentials} {
|
||||
if len(credentials) != 1 || credentials[0].EnvName() != "TRIM_PROVIDER_KEY" {
|
||||
t.Fatal("trim did not forward Scriptorium credential references")
|
||||
}
|
||||
}
|
||||
if scr.RenderRequests[0].Vars["session_id"] != "narratio-session-2026-05-03" {
|
||||
t.Fatalf("render session_id var = %q, want sticky narratio session id", scr.RenderRequests[0].Vars["session_id"])
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user