// Package promptdebug writes explicitly requested prompt diagnostics outside // ordinary application state. package promptdebug import ( "encoding/json" "fmt" "net/url" "os" "path/filepath" "strings" "time" "gitea.maximumdirect.net/eric/weatherreporter/internal/promptexec" "gitea.maximumdirect.net/eric/weatherreporter/internal/report" ) const ( promptPreparationDebugSchemaVersion = "weatherreporter.prompt_preparation_debug.v1" promptExecutionDebugSchemaVersion = "weatherreporter.prompt_execution_debug.v1" debugDirectoryMode = 0o700 debugFileMode = 0o600 ) // PromptDebugWriter stores explicitly requested content-rich diagnostics outside // the managed report workspace. A writer created without a root is disabled. type PromptDebugWriter struct { root string } // PromptDebugRef identifies one debug capture directory. type PromptDebugRef struct { ReportID report.ID ValidDate string RunID string } // PromptDebugMessage is one rendered prompt message retained only in the // explicitly enabled debug store. type PromptDebugMessage struct { Role string `json:"role"` Content string `json:"content"` } // PromptDebugOutput records the declared structured-output contract. type PromptDebugOutput struct { Format string `json:"format"` ValidationMode string `json:"validationMode"` SchemaPath string `json:"schemaPath"` } // PromptDebugPreparation is the explicit, content-safe mapping of preparation // provenance. It intentionally has no fields for credentials or dependencies. type PromptDebugPreparation struct { PromptID string `json:"promptId"` PromptVersion string `json:"promptVersion"` PromptHash string `json:"promptHash"` RenderedPromptHash string `json:"renderedPromptHash"` InputHashes map[string]string `json:"inputHashes,omitempty"` ProfileID string `json:"profileId"` BackendID string `json:"backendId"` ModelName string `json:"modelName"` Output PromptDebugOutput `json:"output"` StartedAt time.Time `json:"startedAt"` EndedAt time.Time `json:"endedAt"` Duration time.Duration `json:"duration"` DataPackagePath string `json:"dataPackagePath"` } // PromptPreparationDebugArtifact is the on-disk preparation debug record. type PromptPreparationDebugArtifact struct { SchemaVersion string `json:"schemaVersion"` ReportID report.ID `json:"reportId"` ValidDate string `json:"validDate"` RunID string `json:"runId"` Preparation PromptDebugPreparation `json:"preparation"` RenderedMessages []PromptDebugMessage `json:"renderedMessages,omitempty"` StructuredSchema json.RawMessage `json:"structuredSchema,omitempty"` Endpoint string `json:"endpoint,omitempty"` Parameters json.RawMessage `json:"parameters,omitempty"` } // PromptDebugUsage is the explicit mapping of provider token accounting. type PromptDebugUsage struct { PromptTokens int `json:"promptTokens"` CompletionTokens int `json:"completionTokens"` TotalTokens int `json:"totalTokens"` CachedTokens int `json:"cachedTokens"` CacheWriteTokens int `json:"cacheWriteTokens"` } // PromptDebugValidation is the completed validation detail retained in the // explicitly enabled debug store. type PromptDebugValidation struct { Status string `json:"status"` Mode string `json:"mode"` SchemaPath string `json:"schemaPath"` Diagnostics []string `json:"diagnostics,omitempty"` } // PromptDebugExecution is the explicit mapping of execution provenance. type PromptDebugExecution struct { RunID string `json:"runId"` PromptID string `json:"promptId"` PromptVersion string `json:"promptVersion"` PromptHash string `json:"promptHash"` RenderedPromptHash string `json:"renderedPromptHash"` InputHashes map[string]string `json:"inputHashes,omitempty"` ProfileID string `json:"profileId"` BackendID string `json:"backendId"` ModelName string `json:"modelName"` GeneratedHash string `json:"generatedHash,omitempty"` Usage PromptDebugUsage `json:"usage"` StartedAt time.Time `json:"startedAt"` EndedAt time.Time `json:"endedAt"` Duration time.Duration `json:"duration"` DataPackagePath string `json:"dataPackagePath"` } // PromptExecutionDebugArtifact is the on-disk execution debug record. type PromptExecutionDebugArtifact struct { SchemaVersion string `json:"schemaVersion"` ReportID report.ID `json:"reportId"` ValidDate string `json:"validDate"` RunID string `json:"runId"` Execution PromptDebugExecution `json:"execution"` Validation PromptDebugValidation `json:"validation"` RawOutput string `json:"rawOutput,omitempty"` DebugValidationDetails []string `json:"debugValidationDetails,omitempty"` } // NewPromptDebugWriter initializes an explicitly rooted writer. An empty root // creates a disabled writer without touching the filesystem. func NewPromptDebugWriter(root string) (*PromptDebugWriter, error) { if strings.TrimSpace(root) == "" { return &PromptDebugWriter{}, nil } if !filepath.IsAbs(root) { return nil, fmt.Errorf("prompt debug root must be absolute") } cleaned := filepath.Clean(root) if cleaned == string(filepath.Separator) { return nil, fmt.Errorf("prompt debug root must not be the filesystem root") } if err := ensureSecureDirectory(cleaned); err != nil { return nil, fmt.Errorf("initialize prompt debug root %q: %w", cleaned, err) } return &PromptDebugWriter{root: cleaned}, nil } func (w *PromptDebugWriter) Enabled() bool { return w != nil && w.root != "" } // WritePreparation stores the explicitly captured preparation details and // returns the per-run debug directory. Disabled writers do no filesystem work. func (w *PromptDebugWriter) WritePreparation(ref PromptDebugRef, preparation promptexec.Preparation, debug *promptexec.PreparationDebug) (string, error) { if !w.Enabled() { return "", nil } directory, err := w.runDirectory(ref) if err != nil { return "", err } artifact := PromptPreparationDebugArtifact{ SchemaVersion: promptPreparationDebugSchemaVersion, ReportID: ref.ReportID, ValidDate: ref.ValidDate, RunID: ref.RunID, Preparation: promptDebugPreparation(preparation), } if debug != nil { artifact.RenderedMessages = promptDebugMessages(debug.RenderedMessages) artifact.StructuredSchema = copyRawJSON(debug.StructuredSchema) artifact.Endpoint = safePromptDebugEndpoint(debug.Endpoint) parameters, err := redactPromptDebugParameters(debug.ParametersJSON) if err != nil { return "", err } artifact.Parameters = parameters } if err := writeSecureJSON(filepath.Join(directory, "preparation.json"), artifact); err != nil { return "", err } return directory, nil } // WriteExecution stores the explicitly captured execution details and returns // the per-run debug directory. Disabled writers do no filesystem work. func (w *PromptDebugWriter) WriteExecution(ref PromptDebugRef, execution promptexec.Execution) (string, error) { if !w.Enabled() { return "", nil } directory, err := w.runDirectory(ref) if err != nil { return "", err } artifact := PromptExecutionDebugArtifact{ SchemaVersion: promptExecutionDebugSchemaVersion, ReportID: ref.ReportID, ValidDate: ref.ValidDate, RunID: ref.RunID, Execution: promptDebugExecution(execution), Validation: promptDebugValidation(execution.Validation), RawOutput: string(execution.RawOutput), } if execution.Debug != nil { if len(execution.Debug.RawOutput) > 0 { artifact.RawOutput = string(execution.Debug.RawOutput) } artifact.DebugValidationDetails = append([]string(nil), execution.Debug.ValidationDiagnostics...) } if err := writeSecureJSON(filepath.Join(directory, "execution.json"), artifact); err != nil { return "", err } return directory, nil } func (w *PromptDebugWriter) runDirectory(ref PromptDebugRef) (string, error) { if err := validatePromptDebugRef(ref); err != nil { return "", err } if err := ensureSecureDirectory(w.root); err != nil { return "", fmt.Errorf("validate prompt debug root %q: %w", w.root, err) } directory := filepath.Join(w.root, string(ref.ReportID), ref.ValidDate, ref.RunID) if !isWithinDirectory(w.root, directory) { return "", fmt.Errorf("prompt debug path escapes root") } if err := ensureSecureDirectory(directory); err != nil { return "", fmt.Errorf("create prompt debug directory %q: %w", directory, err) } return directory, nil } func validatePromptDebugRef(ref PromptDebugRef) error { if err := validatePromptDebugSegment("report id", string(ref.ReportID)); err != nil { return err } if err := validatePromptDebugSegment("valid date", ref.ValidDate); err != nil { return err } if parsed, err := time.Parse("2006-01-02", ref.ValidDate); err != nil || parsed.Format("2006-01-02") != ref.ValidDate { return fmt.Errorf("valid date must use YYYY-MM-DD") } return validatePromptDebugSegment("run id", ref.RunID) } func validatePromptDebugSegment(name string, value string) error { if strings.TrimSpace(value) == "" { return fmt.Errorf("prompt debug %s is required", name) } if filepath.IsAbs(value) || strings.ContainsAny(value, `/\\`) || value == "." || value == ".." { return fmt.Errorf("prompt debug %s must be a safe path segment", name) } return nil } func ensureSecureDirectory(path string) error { if !filepath.IsAbs(path) { return fmt.Errorf("directory must be absolute") } cleaned := filepath.Clean(path) volume := filepath.VolumeName(cleaned) current := volume + string(filepath.Separator) for _, component := range strings.Split(strings.TrimPrefix(cleaned, current), string(filepath.Separator)) { if component == "" { continue } current = filepath.Join(current, component) info, err := os.Lstat(current) if os.IsNotExist(err) { if err := os.Mkdir(current, debugDirectoryMode); err != nil { return err } if err := os.Chmod(current, debugDirectoryMode); err != nil { return err } continue } if err != nil { return err } if info.Mode()&os.ModeSymlink != 0 { return fmt.Errorf("directory component %q must not be a symlink", current) } if !info.IsDir() { return fmt.Errorf("directory component %q is not a directory", current) } } if err := os.Chmod(cleaned, debugDirectoryMode); err != nil { return err } return nil } func isWithinDirectory(root string, path string) bool { relative, err := filepath.Rel(root, path) return err == nil && relative != ".." && !strings.HasPrefix(relative, ".."+string(filepath.Separator)) && !filepath.IsAbs(relative) } func writeSecureJSON(path string, value any) error { data, err := json.MarshalIndent(value, "", " ") if err != nil { return fmt.Errorf("marshal %q: %w", path, err) } if info, err := os.Lstat(path); err == nil { if info.Mode()&os.ModeSymlink != 0 || !info.Mode().IsRegular() { return fmt.Errorf("prompt debug file %q is not a regular file", path) } } else if !os.IsNotExist(err) { return err } temporary, err := os.CreateTemp(filepath.Dir(path), "."+filepath.Base(path)+".*.tmp") if err != nil { return err } temporaryPath := temporary.Name() defer os.Remove(temporaryPath) if err := temporary.Chmod(debugFileMode); err != nil { temporary.Close() return err } if _, err := temporary.Write(data); err != nil { temporary.Close() return err } if err := temporary.Close(); err != nil { return err } if err := os.Rename(temporaryPath, path); err != nil { return err } return nil } func promptDebugPreparation(value promptexec.Preparation) PromptDebugPreparation { return PromptDebugPreparation{ PromptID: value.PromptID, PromptVersion: value.PromptVersion, PromptHash: value.PromptHash, RenderedPromptHash: value.RenderedPromptHash, InputHashes: copyPromptDebugMap(value.InputHashes), ProfileID: value.ProfileID, BackendID: value.BackendID, ModelName: value.ModelName, Output: PromptDebugOutput{Format: value.Output.Format, ValidationMode: value.Output.ValidationMode, SchemaPath: value.Output.SchemaPath}, StartedAt: value.StartedAt, EndedAt: value.EndedAt, Duration: value.Duration, DataPackagePath: value.DataPackagePath, } } func promptDebugExecution(value promptexec.Execution) PromptDebugExecution { return PromptDebugExecution{ RunID: value.RunID, PromptID: value.PromptID, PromptVersion: value.PromptVersion, PromptHash: value.PromptHash, RenderedPromptHash: value.RenderedPromptHash, InputHashes: copyPromptDebugMap(value.InputHashes), ProfileID: value.ProfileID, BackendID: value.BackendID, ModelName: value.ModelName, GeneratedHash: value.GeneratedHash, Usage: PromptDebugUsage{PromptTokens: value.Usage.PromptTokens, CompletionTokens: value.Usage.CompletionTokens, TotalTokens: value.Usage.TotalTokens, CachedTokens: value.Usage.CachedTokens, CacheWriteTokens: value.Usage.CacheWriteTokens}, StartedAt: value.StartedAt, EndedAt: value.EndedAt, Duration: value.Duration, DataPackagePath: value.DataPackagePath, } } func promptDebugValidation(value promptexec.Validation) PromptDebugValidation { return PromptDebugValidation{Status: string(value.Status), Mode: value.Mode, SchemaPath: value.SchemaPath, Diagnostics: append([]string(nil), value.Diagnostics...)} } func promptDebugMessages(values []promptexec.RenderedMessage) []PromptDebugMessage { if len(values) == 0 { return nil } result := make([]PromptDebugMessage, len(values)) for index, value := range values { result[index] = PromptDebugMessage{Role: value.Role, Content: value.Content} } return result } func copyPromptDebugMap(values map[string]string) map[string]string { if values == nil { return nil } result := make(map[string]string, len(values)) for key, value := range values { result[key] = value } return result } func copyRawJSON(value []byte) json.RawMessage { if len(value) == 0 { return nil } return append(json.RawMessage(nil), value...) } func safePromptDebugEndpoint(value string) string { parsed, err := url.Parse(value) if err != nil { return "" } parsed.User = nil parameters := parsed.Query() for key := range parameters { if isPromptDebugSecretKey(key) { parameters[key] = []string{"[redacted]"} } } parsed.RawQuery = parameters.Encode() parsed.Fragment = "" return parsed.String() } func redactPromptDebugParameters(value []byte) (json.RawMessage, error) { if len(value) == 0 { return nil, nil } var decoded any if err := json.Unmarshal(value, &decoded); err != nil { return nil, fmt.Errorf("decode prompt debug parameters: %w", err) } redactPromptDebugValue(decoded) encoded, err := json.Marshal(decoded) if err != nil { return nil, fmt.Errorf("encode prompt debug parameters: %w", err) } return encoded, nil } func redactPromptDebugValue(value any) { switch typed := value.(type) { case map[string]any: for key, item := range typed { if isPromptDebugSecretKey(key) { typed[key] = "[redacted]" continue } redactPromptDebugValue(item) } case []any: for _, item := range typed { redactPromptDebugValue(item) } } } func isPromptDebugSecretKey(key string) bool { normalized := strings.NewReplacer("_", "", "-", "", " ", "").Replace(strings.ToLower(key)) return strings.Contains(normalized, "credential") || strings.Contains(normalized, "secret") || strings.Contains(normalized, "password") || strings.Contains(normalized, "token") || strings.Contains(normalized, "apikey") || strings.Contains(normalized, "authorization") }