eric/narratio
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 27 Wiki Activity

Remove invocation analyze projection

Browse Source
This commit is contained in:
Eric Rakestraw
2026-09-03 04:11:33 +00:00
parent 7997f81f47
commit 2d20900cbf
10 changed files with 53 additions and 97 deletions
Download Patch File Download Diff File Expand all files Collapse all files

20
docs/internal/manifest.md
View File

@@ -79,18 +79,14 @@ do not replace the concrete configured key or infer family membership from a
name, so older records without them remain valid.
The session-stage collection is the reconciled authority across invocations.
The corresponding collection on an invocation's `analyze` stage record is an
audit of only the artifacts evaluated or attempted by that run. These records
remain analyze-owned data inside the fixed stage; they are not dynamic stages
or generic subtasks.
These records remain analyze-owned data inside the fixed stage; they are not
dynamic stages or generic subtasks.
The stage result contract has one analyze-specific projection boundary. On
success, the runner validates and deep-copies the complete reconciled session
collection and the invocation subset. Aggregate session outputs are rebuilt in
configured-key order from current session records only; invocation outputs are
limited to current records produced by that invocation's run ID. Ordinary
stage outputs cannot accompany this projection, so there is one source of
artifact authority.
collection. Aggregate session outputs are rebuilt in configured-key order from
current session records only. Ordinary stage outputs cannot accompany this
projection, so there is one source of artifact authority.
Successful incremental execution replaces only evaluated artifact records and
preserves valid unrelated current records. Rebuilt outputs are compared by
@@ -111,14 +107,14 @@ the successful projection then restores the aggregate status.
Analyze may return a projection together with an error. That restricted result
cannot carry ordinary outputs, skip state, aggregate logs, generated configs,
or metadata. The runner persists only the validated per-artifact collections,
then marks the aggregate analyze and run state failed and invalidates delivery
then marks the aggregate analyze state failed and invalidates delivery
dependents conservatively. Unrelated current records survive because the
session projection is complete. A malformed projection is not applied, and a
failed session projection save restores the prior per-artifact authority before
terminal failure persistence.
The incremental executor constructs this restricted projection at each
scheduled artifact boundary. The active record is failed without output,
The incremental executor constructs this restricted projection over its
scheduled artifact work. The active record is failed without output,
current transitive dependents are stale, unrelated current records survive, and
only earlier validated and materialized completions remain current in the
reconciled session collection. A terminal session-save failure is joined with

25
docs/internal/stage-analyze.md
View File

@@ -32,7 +32,7 @@ Supported source families:
- when `pipeline.scriptorium` is absent or no configured artifact is
executable, completes successfully with no outputs and records explanatory
metadata. This is not an explicit self-skip: both manifests record success,
metadata. This is not an explicit self-skip: the session manifest records success,
satisfy publish's prerequisite, and an ordinary later run reuses the result
while the effective set remains empty. Enabling or selecting an artifact
later makes missing versioned evidence non-resumable and schedules it without
@@ -118,11 +118,9 @@ Supported source families:
remain current. If that semantic identity changes, unselected transitive
dependents become stale without being executed; dependents included in the
invocation are evaluated in dependency order instead.
- reports all evaluated targets and prerequisites in invocation state. The
runner reconstructs aggregate session outputs from every current session
record and invocation outputs from only records produced by the current run.
Unrelated stale records do not make an otherwise successful partial
invocation fail.
- returns one complete reconciled session collection. The runner reconstructs
aggregate session outputs from every current session record; unrelated stale
records do not make an otherwise successful partial invocation fail.
- resolves previous-session sources from local `previous/` cache only.
- runs optional render-debug, then artifact execution.
- validates non-empty output files and materializes canonical outputs.
@@ -140,16 +138,17 @@ Supported source families:
- a scheduled artifact failure returns the restricted analyze-state projection
with the active artifact marked `failed`, a bounded error, and no output
authority. Current transitive dependents become stale without execution.
- earlier artifacts from the invocation remain current only after their
run-local output passed validation and canonical materialization. They remain
in invocation history; unattempted later artifacts do not appear there.
- earlier artifacts remain current only after their run-local output passed
validation and canonical materialization; unattempted later artifacts retain
their reconciled session state.
- unrelated current records survive a partial failure. Old canonical bytes for
the failed artifact and newly materialized bytes whose projection cannot be
persisted are incidental, not current evidence.
- the runner persists a valid partial projection before it marks aggregate
analyze failed and invalidates publish and notify through the application
dependency relation. Projection-persistence errors retain the last durable
per-artifact authority and are joined with the original failure context.
- the runner applies one valid partial projection at the existing end-of-stage
save boundary before it marks aggregate analyze failed and invalidates publish
and notify through the application dependency relation. Projection-persistence
errors retain the last durable per-artifact authority and are joined with the
original failure context. Per-artifact checkpoint persistence is future work.
## Invariants

2
docs/roadmap/implementation.md
View File

@@ -235,7 +235,7 @@ preserving the existing session-ledger lifecycle and immediate command result.
## Stage 3 — Remove Invocation-Only Analyze Projection
**Status: Pending**
**Status: Completed**
### Goal

19
internal/app/analyze_projection.go
View File

@@ -2,7 +2,6 @@ package app
import (
"fmt"
"reflect"
"sort"
"gitea.maximumdirect.net/eric/narratio/internal/manifest"
@@ -10,8 +9,7 @@ import (
)
type validatedAnalyzeProjection struct {
session map[string]manifest.AnalyzeArtifactRecord
invocation map[string]manifest.AnalyzeArtifactRecord
session map[string]manifest.AnalyzeArtifactRecord
}
type analyzeStateSnapshot struct {
@@ -70,23 +68,10 @@ func validateFailedAnalyzeProjection(stageName string, result *stage.StageResult
func validateAndCloneAnalyzeProjection(projection *stage.AnalyzeStateProjection) (*validatedAnalyzeProjection, error) {
session := manifest.CloneAnalyzeArtifactCollection(projection.Session)
invocation := manifest.CloneAnalyzeArtifactCollection(projection.Invocation)
if err := manifest.ValidateAnalyzeArtifactCollection(manifest.AnalyzeStateContractVersion, session); err != nil {
return nil, fmt.Errorf("validate reconciled session analyze state: %w", err)
}
if err := manifest.ValidateAnalyzeArtifactCollection(manifest.AnalyzeStateContractVersion, invocation); err != nil {
return nil, fmt.Errorf("validate invocation analyze state: %w", err)
}
for key, invocationRecord := range invocation {
sessionRecord, ok := session[key]
if !ok {
return nil, fmt.Errorf("invocation analyze artifact %q is absent from reconciled session state", key)
}
if !reflect.DeepEqual(invocationRecord, sessionRecord) {
return nil, fmt.Errorf("invocation analyze artifact %q contradicts reconciled session state", key)
}
}
return &validatedAnalyzeProjection{session: session, invocation: invocation}, nil
return &validatedAnalyzeProjection{session: session}, nil
}
func applyAnalyzeProjection(

15
internal/app/analyze_projection_test.go
View File

@@ -41,10 +41,6 @@ func TestExecuteStagesProjectsReconciledSessionAnalyzeState(t *testing.T) {
Logs: []string{"aggregate-analyze.log"},
AnalyzeState: &stage.AnalyzeStateProjection{
Session: session,
Invocation: map[string]manifest.AnalyzeArtifactRecord{
"player_handout": oldRecord,
"session_recap": newRecord,
},
},
}, nil
}}
@@ -95,10 +91,6 @@ func TestExecuteStagesPersistsRestrictedAnalyzeStateOnPartialError(t *testing.T)
}
return &stage.StageResult{AnalyzeState: &stage.AnalyzeStateProjection{
Session: session,
Invocation: map[string]manifest.AnalyzeArtifactRecord{
"quest_log": failed,
"session_recap": completed,
},
}}, errors.New("quest log failed")
}}
@@ -141,8 +133,7 @@ func TestExecuteStagesRejectsInvalidAnalyzeProjectionWithoutReplacingPriorState(
invalid := appAnalyzeRecord("session_recap", manifest.AnalyzeArtifactCurrent, m.RunID, time.Now().UTC())
invalid.Output.Checksum = "invalid"
return &stage.StageResult{AnalyzeState: &stage.AnalyzeStateProjection{
Session: map[string]manifest.AnalyzeArtifactRecord{"session_recap": invalid},
Invocation: map[string]manifest.AnalyzeArtifactRecord{"session_recap": invalid},
Session: map[string]manifest.AnalyzeArtifactRecord{"session_recap": invalid},
}}, errors.New("analysis failed")
}}
@@ -182,7 +173,6 @@ func TestExecuteStagesRollsBackAnalyzeAuthorityWhenProjectionSaveFails(t *testin
"player_handout": prior,
"session_recap": newRecord,
},
Invocation: map[string]manifest.AnalyzeArtifactRecord{"session_recap": newRecord},
}}, nil
}}
@@ -227,8 +217,7 @@ func TestExecuteStagesRejectsContradictoryAnalyzeResultWithError(t *testing.T) {
return &stage.StageResult{
Outputs: []artifacts.Ref{{Kind: "session_recap", RelativePath: "artifacts/session-recap.md"}},
AnalyzeState: &stage.AnalyzeStateProjection{
Session: map[string]manifest.AnalyzeArtifactRecord{"session_recap": record},
Invocation: map[string]manifest.AnalyzeArtifactRecord{"session_recap": record},
Session: map[string]manifest.AnalyzeArtifactRecord{"session_recap": record},
},
}, errors.New("analysis failed")
}}

28
internal/stage/analyze.go
View File

@@ -151,22 +151,18 @@ func (analyzeStage) Run(ctx context.Context, env *Env, m *manifest.Manifest) (*S
reusedArtifacts := []map[string]any{}
reusedSeen := map[string]struct{}{}
sessionRecords := manifest.CloneAnalyzeArtifactCollection(workPlan.ProjectedRecords)
invocationRecords := make(map[string]manifest.AnalyzeArtifactRecord)
priorCurrentRecords := make(map[string]manifest.AnalyzeArtifactRecord)
for _, item := range reconciliation.Ordered {
if item.Stored != nil && item.Stored.Status == manifest.AnalyzeArtifactCurrent {
priorCurrentRecords[item.Key] = *item.Stored
}
}
invocationKeys := make(map[string]struct{}, len(workPlan.ExecutionOrder)+len(workPlan.ReusedCurrent))
currentKeys := make(map[string]struct{}, len(workPlan.ExecutionOrder)+len(workPlan.ReusedCurrent))
for _, item := range workPlan.ExecutionOrder {
invocationKeys[item.Key] = struct{}{}
currentKeys[item.Key] = struct{}{}
}
for _, item := range workPlan.ReusedCurrent {
invocationKeys[item.Key] = struct{}{}
if record, ok := sessionRecords[item.Key]; ok {
invocationRecords[item.Key] = record
}
currentKeys[item.Key] = struct{}{}
}
for _, item := range workPlan.ExecutionOrder {
@@ -185,7 +181,6 @@ func (analyzeStage) Run(ctx context.Context, env *Env, m *manifest.Manifest) (*S
"",
executionErr,
sessionRecords,
invocationRecords,
), executionErr
}
priorRecord, hadPriorRecord := priorCurrentRecords[item.Key]
@@ -199,7 +194,6 @@ func (analyzeStage) Run(ctx context.Context, env *Env, m *manifest.Manifest) (*S
fingerprint,
err,
sessionRecords,
invocationRecords,
), err
}
@@ -236,18 +230,16 @@ func (analyzeStage) Run(ctx context.Context, env *Env, m *manifest.Manifest) (*S
fingerprint,
recordErr,
sessionRecords,
invocationRecords,
), recordErr
}
sessionRecords[plan.Name] = record
invocationRecords[plan.Name] = record
sourceID, ok := runtimeCatalog.SourceIDForConfiguredKey(plan.Name)
if !ok {
catalogErr := fmt.Errorf("analyze: source id not found for artifact %q", plan.Name)
return failedAnalyzeResult(
execution, item, artifactCfg, fingerprint, catalogErr,
sessionRecords, invocationRecords,
sessionRecords,
), catalogErr
}
if err := runtimeCatalog.MarkAvailableGeneratedEvidence(
@@ -261,14 +253,14 @@ func (analyzeStage) Run(ctx context.Context, env *Env, m *manifest.Manifest) (*S
catalogErr := fmt.Errorf("analyze: mark generated artifact %q available: %w", sourceID, err)
return failedAnalyzeResult(
execution, item, artifactCfg, fingerprint, catalogErr,
sessionRecords, invocationRecords,
sessionRecords,
), catalogErr
}
if !hadPriorRecord || !sameAnalyzeOutputIdentity(priorRecord, record) {
staleUnscheduledAnalyzeDependents(
env.Config.Pipeline.Scriptorium.Artifacts,
plan.Name,
invocationKeys,
currentKeys,
sessionRecords,
)
}
@@ -295,8 +287,7 @@ func (analyzeStage) Run(ctx context.Context, env *Env, m *manifest.Manifest) (*S
GeneratedConfigs: dedupeAndSortPaths(generatedConfigs),
Metadata: metadata,
AnalyzeState: &AnalyzeStateProjection{
Session: sessionRecords,
Invocation: manifest.CloneAnalyzeArtifactCollection(invocationRecords),
Session: sessionRecords,
},
}, nil
}
@@ -308,7 +299,6 @@ func failedAnalyzeResult(
fingerprint string,
cause error,
sessionRecords map[string]manifest.AnalyzeArtifactRecord,
invocationRecords map[string]manifest.AnalyzeArtifactRecord,
) *StageResult {
record := manifest.AnalyzeArtifactRecord{
Key: item.Key,
@@ -332,11 +322,9 @@ func failedAnalyzeResult(
}
}
sessionRecords[item.Key] = record
invocationRecords[item.Key] = record
staleAnalyzeDependents(artifactCfgMap(execution), item.Key, sessionRecords)
return &StageResult{AnalyzeState: &AnalyzeStateProjection{
Session: manifest.CloneAnalyzeArtifactCollection(sessionRecords),
Invocation: manifest.CloneAnalyzeArtifactCollection(invocationRecords),
Session: manifest.CloneAnalyzeArtifactCollection(sessionRecords),
}}
}

8
internal/stage/analyze_failure_test.go
View File

@@ -31,7 +31,7 @@ func TestAnalyzeAdapterFailureProjectsCompletedAndFailedArtifacts(t *testing.T)
}
keys := []string{"alpha", "beta", "gamma"}
for index, key := range keys {
record, exists := result.AnalyzeState.Invocation[key]
record, exists := result.AnalyzeState.Session[key]
switch {
case index < failAt-1:
if !exists || record.Status != manifest.AnalyzeArtifactCurrent || record.Output == nil {
@@ -43,7 +43,7 @@ func TestAnalyzeAdapterFailureProjectsCompletedAndFailedArtifacts(t *testing.T)
}
default:
if exists {
t.Fatalf("unattempted %s appeared in invocation: %#v", key, record)
t.Fatalf("unattempted %s appeared in reconciled session state: %#v", key, record)
}
}
}
@@ -86,8 +86,8 @@ func TestAnalyzeMiddleFailureStalesDependentsAndPreservesUnrelatedCurrent(t *tes
got.Fingerprint != unrelated.Fingerprint || got.Output == nil || got.Output.Checksum != unrelated.Output.Checksum {
t.Fatalf("unrelated record = %#v, want preserved %#v", got, unrelated)
}
if _, attempted := result.AnalyzeState.Invocation["gamma"]; attempted {
t.Fatal("dependent target was reported as attempted")
if result.AnalyzeState.Session["gamma"].Status != manifest.AnalyzeArtifactStale {
t.Fatalf("dependent target state = %#v, want stale", result.AnalyzeState.Session["gamma"])
}
if old := m.Stages["analyze"].AnalyzeArtifacts["beta"]; old.Output == nil {
t.Fatal("fixture did not retain old canonical evidence before failure")

18
internal/stage/analyze_incremental_execution_test.go
View File

@@ -35,8 +35,8 @@ func TestAnalyzeReusesCurrentArtifactWithoutScriptorium(t *testing.T) {
if got := second.Metadata["reused_current"]; !reflect.DeepEqual(got, []string{"session_recap"}) {
t.Fatalf("reused current = %#v, want session_recap", got)
}
if second.AnalyzeState.Invocation["session_recap"].Status != manifest.AnalyzeArtifactCurrent {
t.Fatalf("invocation state = %#v", second.AnalyzeState.Invocation)
if second.AnalyzeState.Session["session_recap"].Status != manifest.AnalyzeArtifactCurrent {
t.Fatalf("session state = %#v", second.AnalyzeState.Session)
}
}
@@ -70,8 +70,8 @@ func TestAnalyzePartialForcePreservesUnrelatedCurrentRecord(t *testing.T) {
if got := second.AnalyzeState.Session["player_handout"]; !reflect.DeepEqual(got, priorDependent) {
t.Fatalf("unrelated dependent changed = %#v, want %#v", got, priorDependent)
}
if _, attempted := second.AnalyzeState.Invocation["player_handout"]; attempted {
t.Fatal("unselected dependent appeared in invocation state")
if _, current := second.AnalyzeState.Session["player_handout"]; !current {
t.Fatal("unselected dependent was removed from session state")
}
}
@@ -102,9 +102,9 @@ func TestAnalyzeExposesReusedPrerequisiteToScheduledDependent(t *testing.T) {
if got := fake.RunRequests[0].InputPaths["recap"]; got != filepath.Join(paths.ArtifactsDir, "session_recap.md") {
t.Fatalf("reused prerequisite input = %q", got)
}
if second.AnalyzeState.Invocation["session_recap"].Status != manifest.AnalyzeArtifactCurrent ||
second.AnalyzeState.Invocation["player_handout"].Status != manifest.AnalyzeArtifactCurrent {
t.Fatalf("invocation records = %#v, want reused prerequisite and produced dependent", second.AnalyzeState.Invocation)
if second.AnalyzeState.Session["session_recap"].Status != manifest.AnalyzeArtifactCurrent ||
second.AnalyzeState.Session["player_handout"].Status != manifest.AnalyzeArtifactCurrent {
t.Fatalf("session records = %#v, want reused prerequisite and produced dependent", second.AnalyzeState.Session)
}
}
@@ -131,8 +131,8 @@ func TestAnalyzeChangedOutputStalesUnselectedDependents(t *testing.T) {
if dependent.Status != manifest.AnalyzeArtifactStale || dependent.Output != nil || dependent.OutputSize != 0 {
t.Fatalf("dependent record = %#v, want stale without output", dependent)
}
if _, attempted := second.AnalyzeState.Invocation["player_handout"]; attempted {
t.Fatal("changed-output invalidation executed the unselected dependent")
if second.AnalyzeState.Session["player_handout"].Status != manifest.AnalyzeArtifactStale {
t.Fatalf("dependent state = %#v, want stale", second.AnalyzeState.Session["player_handout"])
}
}

8
internal/stage/analyze_test.go
View File

@@ -62,7 +62,7 @@ func TestAnalyzeGeneratesSessionRecapFromTrimmedTranscript(t *testing.T) {
t.Fatalf("session_id var = %q, want sticky narratio session id", req.Vars["session_id"])
}
if result.AnalyzeState == nil || result.AnalyzeState.Invocation["session_recap"].Status != manifest.AnalyzeArtifactCurrent {
if result.AnalyzeState == nil || result.AnalyzeState.Session["session_recap"].Status != manifest.AnalyzeArtifactCurrent {
t.Fatalf("analyze state = %#v, want current session_recap", result.AnalyzeState)
}
if len(result.Logs) != 2 {
@@ -316,7 +316,7 @@ func TestAnalyzeUsesRunLocalPathsAndMaterializesCanonical(t *testing.T) {
if !strings.Contains(fake.RunRequests[0].OutputPath, filepath.Join("runs", m.RunID, "analyze", "outputs")) {
t.Fatalf("run output path = %q, want run-local path", fake.RunRequests[0].OutputPath)
}
if result.AnalyzeState == nil || result.AnalyzeState.Invocation["session_recap"].Output == nil {
if result.AnalyzeState == nil || result.AnalyzeState.Session["session_recap"].Output == nil {
t.Fatalf("analyze state = %#v, want session recap output evidence", result.AnalyzeState)
}
if err := requireNonEmptyFile(filepath.Join(paths.ArtifactsDir, "session_recap.md"), "materialized output"); err != nil {
@@ -686,7 +686,7 @@ func TestAnalyzeAppliesSelectedArtifactsFilter(t *testing.T) {
if fake.RunRequests[0].PromptID != "dnd.player_handout" {
t.Fatalf("prompt id = %q, want dnd.player_handout", fake.RunRequests[0].PromptID)
}
if result.AnalyzeState == nil || len(result.AnalyzeState.Invocation) != 1 || result.AnalyzeState.Invocation["player_handout"].Status != manifest.AnalyzeArtifactCurrent {
if result.AnalyzeState == nil || len(result.AnalyzeState.Session) != 1 || result.AnalyzeState.Session["player_handout"].Status != manifest.AnalyzeArtifactCurrent {
t.Fatalf("analyze state = %#v, want only current player_handout", result.AnalyzeState)
}
}
@@ -1391,7 +1391,7 @@ func TestAnalyzeRecordsRefsAndMetadata(t *testing.T) {
if err != nil {
t.Fatalf("Run() error = %v", err)
}
record, ok := result.AnalyzeState.Invocation["session_recap"]
record, ok := result.AnalyzeState.Session["session_recap"]
if !ok || record.Output == nil || record.Output.LocalPath != "artifacts/session_recap.md" {
t.Fatalf("analyze output record = %#v, want session recap evidence", record)
}

7
internal/stage/stage.go
View File

@@ -132,9 +132,8 @@ type StageResult struct {
AnalyzeState *AnalyzeStateProjection
}
// AnalyzeStateProjection carries analyze-owned reconciled session authority and
// the invocation subset evaluated by the current run.
// AnalyzeStateProjection carries the reconciled session authority proposed for
// the runner's existing end-of-stage session-manifest save.
type AnalyzeStateProjection struct {
Session map[string]manifest.AnalyzeArtifactRecord
Invocation map[string]manifest.AnalyzeArtifactRecord
Session map[string]manifest.AnalyzeArtifactRecord
}