scriptorium/docs/roadmap/implementation.md

# Promptkit v0.9.0 Adoption Implementation Plan

## Status And Authority

Planned. Implement the stages below in numeric order. The accepted feature
scope and target state are defined by the
[Promptkit v0.9.0 adoption roadmap](promptkit-v0.9.0-adoption.md), and the
durable product boundary is defined by
[ADR 0004](../adr/0004-definition-boundary.md). If this plan and either source
conflict, the ADR governs product boundaries and the feature roadmap governs
scope.

Follow the architecture, documentation, and testing policies under
`docs/policy/` throughout the work. In particular:

- use only Promptkit's public root package and its released v0.9.0 tag;
- do not add a workspace, module replacement, vendored Promptkit source,
  direct catalog import, or Scriptorium reimplementation of Promptkit parsing
  and validation;
- keep default tests deterministic, offline, and free of real credentials or
  provider calls;
- update each current-state contract in the same stage that implements its
  behavior; and
- prefer a small number of boundary and integration tests over duplicating
  Promptkit's own framework test matrix.

At the start of every stage, inspect the working tree and preserve unrelated
changes. At the end of every stage, run the focused tests named in that stage,
`gofmt` changed Go files, and run `git diff --check`. Do not proceed while a
stage's completion conditions are unmet.

## Stage 1: Adopt The Promptkit v0.9.0 Compatibility Baseline

**Completion: Complete.**

Make the released dependency the build baseline before adding new application
surface area.

### Implementation

1. Update `go.mod` to require
   `gitea.maximumdirect.net/eric/promptkit v0.9.0`, then run `go mod tidy` so
   `go.mod` and `go.sum` contain the module graph selected by that tag. Accept
   Promptkit's OpenRouter and Rakestrawhome catalog modules only as transitive
   dependencies; do not import them from Scriptorium.
2. In the HTTP request DTO, change `model.reasoning_effort` from `string` to
   `*string`, and pass the pointer directly to
   `promptkit.ExecutionTargetOverride.ReasoningEffort`. This is required by the
   v0.9.0 API and preserves omitted versus explicit-empty input. Keep the JSON
   field itself a string when present.
3. Compile every Promptkit public struct literal against v0.9.0. Keep literals
   keyed, do not restore the removed `RunRequest.Metadata` field, and do not
   add compatibility wrappers around Promptkit.
4. Audit maintained prompt, profile, schema, configuration, and HTTP examples
   against v0.9.0. Correct only actual incompatibilities: endpoints must be
   absolute HTTP(S) URLs, roles must be `developer`, `system`, `user`, or
   `assistant`, positive repair budgets must be at most three and paired with
   `basic`, `json`, or `json_schema`, content and schema paths must remain
   within their Promptkit source rules, and extra parameters must be
   JSON-compatible and avoid reserved provider keys.
5. Replace v0.1.0 links and current-version statements in current-state
   documentation with tagged v0.9.0 links. Do not rewrite historical version
   statements in accepted ADRs. Update the owning documentation to state the
   v0.9.0 credential and provider behavior: an optional missing environment
   credential may result in an unauthenticated provider request, while a
   profile that declares credentials required still fails as an invalid request
   when no source is supplied and with `ErrAPIKeyEnvMissing` when its selected
   environment source is empty; unset optional provider controls are omitted;
   and a positive repair budget can add provider calls, latency, token use, and
   cost.
6. Preserve the architecture guard for forbidden replacements, workspaces,
   vendor trees, former facade packages, and Promptkit `internal` imports.
   Extend it only if the dependency graph introduces a new realistic bypass
   that the existing assertions do not cover.

### Verification

- Update HTTP mapping tests for the pointer-valued reasoning field, including
  separate omitted and explicit-empty cases, and update `docs/api.md` with the
  HTTP three-state behavior in this stage. Stage 4 adds the corresponding CLI
  control and session-ID support.
- Run `go test ./...`, `go test -race ./...`, `go vet ./...`, and
  `go build ./cmd/scriptorium` against the tagged module with no local
  replacement.
- Run the existing maintained render example and any example-validation tests.
- Confirm that a search for `v0.1.0` finds only legitimate historical material,
  and that searches for `replace`, `go.work`, and Promptkit `/internal/`
  imports do not reveal a prohibited dependency path.

### Completion Conditions

The repository builds and all existing behavior tests pass on Promptkit v0.9.0;
current documentation identifies v0.9.0 as the framework contract; no local or
private Promptkit integration mechanism has been introduced.

## Stage 2: Remove Definition-Compatibility Restrictions In Run And Render

**Completion: Complete.**

Allow the executable adapters to address every valid directory-backed prompt
definition without imposing input requirements of their own.

### Implementation

1. Add `promptVersion string` to the shared run/render request configuration.
   Register `--prompt-version <version>` for both `run` and `render`, include it
   in usage text, and map it without local normalization to
   `promptkit.RunRequest.PromptVersion`.
2. Make `--input` optional for `run` and `render`. Do not weaken
   `parseMappings` for callers that actually supplied mappings; instead, skip
   that parser when no input flag was supplied and pass a nil or empty input
   map to Promptkit. Promptkit remains responsible for declared-required and
   template-referenced input failures.
3. In `POST /v1/runs`, remove the adapter check that rejects a nil or empty
   `inputs` object. Preserve strict JSON decoding and all validation of input
   references that are present. Continue mapping the already-supported
   `prompt_version` field exactly once into `RunRequest.PromptVersion`.
4. Keep extra supplied inputs legal and retain the CLI file-reader and HTTP
   artifact containment and size policies.
5. Update `docs/cli.md` and `docs/api.md` in this stage: prompt ID remains
   required, prompt version is optional, inputs are optional at the adapter
   boundary, and Promptkit decides whether the selected definition requires
   them. Update the shortest request examples only where necessary to avoid
   implying that all prompts need inputs.

### Verification

- Add CLI behavior tests proving that an omitted version selects a sole prompt
  definition, an explicit version selects the requested definition, and an
  omitted version fails clearly when a prompt ID has multiple versions.
- Add CLI and HTTP adapter coverage for: a prompt with no declared inputs,
  omitted optional inputs, a missing declared-required input, and a missing
  template-referenced input. Use a capturing runner for pure request mapping
  and one real Promptkit engine integration fixture for the definition-driven
  failures; do not reproduce Promptkit's parser tests.
- Retain tests for malformed supplied mappings, invalid HTTP input references,
  artifact containment, and request-size limits.
- Run `go test ./internal/adapter/cli ./internal/adapter/http` and the maintained
  render workflow.

### Completion Conditions

Both executable interfaces can select a specific prompt version and can submit
no inputs; only Promptkit rejects missing definition-required data.

## Stage 3: Add Engine-Scoped Custom Backend Configuration

Add the application-owned configuration needed for Promptkit profiles to select
custom backend IDs and capacity policies.

### Configuration Contract

Use this exact strict-YAML shape:

```yaml
backends:
  local-gpu:
    endpoint: http://localhost:11434/v1
    api_key_env: LOCAL_GPU_API_KEY
    extra_params:
      provider_option: enabled
    concurrency_limit: 2
    queue_capacity: 0
```

The mapping key is the backend ID. `endpoint` is required by the effective
contract. `api_key_env`, `extra_params`, `concurrency_limit`, and
`queue_capacity` are optional. `concurrency_limit` uses zero as Promptkit's
unlimited value. Represent `queue_capacity` as `*int` so omission uses
Promptkit's default queue capacity and explicit zero disables queuing. There
are no backend CLI overrides and no raw API-key field.

### Implementation

1. Add `Backends map[string]BackendConfig` to the on-disk `config.Config` and a
   `BackendConfig` with exactly the five YAML fields above. Add a resolved
   `BackendSettings` value containing `ID` plus the corresponding fields, and
   carry `[]BackendSettings` on `AppSettings`.
2. During config application, sort the YAML map keys and build the resolved
   slice in that order. Preserve the `queue_capacity` pointer and the complete
   `extra_params` value. The deterministic order makes multi-error diagnostics
   and tests stable; it is not backend precedence.
3. Do not duplicate Promptkit's backend validators in `internal/config`.
   Strict YAML decoding owns unknown-field and type errors. Promptkit
   `NewEngine` owns blank/reserved/duplicate IDs, endpoint and environment-name
   rules, reserved or invalid extra parameters, and capacity relationships.
4. Refactor CLI engine construction around a small shared engine-settings value
   containing prompt, profile, and schema directories plus resolved backends.
   Convert every `BackendSettings` to `promptkit.Backend`, register it with
   `promptkit.WithBackend`, and use the same constructor for `run`, `render`,
   and `serve`. Wrap construction failures with application-configuration and
   engine-initialization context while preserving `errors.Is` identities.
5. Retain one immutable backend registry per constructed engine. Do not create
   registries per HTTP request, and do not add direct imports of Promptkit's
   catalog modules; Promptkit's built-ins remain available automatically.
6. Document `backends` in `docs/config.md`, including key identity,
   queue-capacity presence, validation ownership, optional environment
   credentials, and the raw-secret prohibition. Update
   `docs/internal/adapters.md` and `docs/internal/sources.md` to describe the
   implemented mapping and engine assembly. Add the exact shape to
   `examples/config.full.yml`; keep the minimal config minimal.

### Verification

- In `internal/config`, test strict decoding of all fields, unknown and raw
  secret-field rejection, deterministic resolved ordering, JSON-compatible
  nested `extra_params`, and omitted versus explicit-zero `queue_capacity`.
- At the CLI/engine boundary, test a valid custom backend selected by a custom
  profile through `render`, and representative Promptkit validation failures:
  reserved backend ID, invalid endpoint, invalid capacity relationship, and a
  reserved `extra_params` key. Assert the stable error identity or a small
  diagnostic fragment, not Promptkit's full text.
- Test that endpoint-only profiles and built-in profiles still work without a
  `backends` section.
- Run `go test ./internal/config ./internal/adapter/cli` and render with the
  complete example without contacting a provider.

### Completion Conditions

Every command constructs its engine from the same resolved custom-backend
configuration, Promptkit remains the validator, and profiles can select custom,
built-in, or endpoint-only execution targets.

## Stage 4: Expose Session And Presence-Aware Reasoning Controls

Complete the request mapping for direct session identifiers and the v0.9.0
reasoning override semantics.

### Implementation

1. Add `--session-id <id>` and `--reasoning-effort <value>` to both `run` and
   `render`, with no aliases. Map the session value to
   `RunRequest.SessionID`.
2. Track flag presence for `--reasoning-effort`. Omission leaves
   `ExecutionTargetOverride.ReasoningEffort` nil, a nonblank value supplies a
   pointer to that value, and `--reasoning-effort=` supplies a pointer to the
   empty string. Include reasoning presence when deciding whether to allocate
   the enclosing execution override.
3. Add optional `session_id` to the strict HTTP run-request DTO and map it to
   `RunRequest.SessionID` without application-level normalization. Keep
   `model.reasoning_effort` pointer-valued so omitted, nonblank, and empty JSON
   strings remain distinct.
4. Add optional `session_id` to successful HTTP metadata and populate it from
   the effective `RunResult.SessionID`, not directly from the request. Use
   `omitempty`; absence means no effective session ID.
5. Continue to reject unknown HTTP fields and do not add `api_key`, appended
   messages, or request-level output-contract fields.
6. Update `docs/cli.md` and `docs/api.md` with the exact flag/field names,
   inheritance/replacement/clearing semantics, the non-secret nature and
   Promptkit length rules of session IDs, and effective-session response
   behavior.

### Verification

- CLI tests must distinguish an omitted reasoning flag, a nonblank replacement,
  and an explicit empty clear by observing the resulting prepared run through
  `render`. Add a mapping assertion that an explicit empty value still creates
  an execution override.
- HTTP tests must capture and distinguish the same three reasoning states and
  continue to reject non-string values under the strict DTO contract. Treat
  JSON `null` like omission, matching ordinary pointer-field decoding; the
  documented explicit clear remains the empty JSON string.
- CLI and HTTP tests must cover a direct session ID, a definition-rendered
  session ID when no direct value is supplied, no effective session ID, and an
  invalid overlong direct ID. Assert that successful HTTP metadata reports the
  engine's effective result.
- Run `go test ./internal/adapter/cli ./internal/adapter/http ./internal/format`.

### Completion Conditions

CLI and HTTP callers can inherit, replace, or clear reasoning effort and can
supply a direct session ID, with Promptkit retaining normalization and
validation ownership.

## Stage 5: Present Backend Identity And Map Capacity Outcomes

Expose Promptkit's selected routing identity and make overload behavior a
stable application contract.

### Implementation

1. In prepared-run text output, add `selected_backend_id` immediately after
   `selected_profile_id` when it is nonempty. Preserve direct JSON marshaling of
   `promptkit.PreparedRun`; v0.9.0 already supplies `selected_backend_id` and
   `effective_model_params.backend_id` JSON fields.
2. In the CLI run summary, append `backend=<id>` when
   `RunResult.SelectedBackendID` is nonempty. Omit the token for endpoint-only
   profiles rather than deriving an ID from the endpoint.
3. Add `selected_backend_id,omitempty` to HTTP metadata and
   `backend_id,omitempty` to `metadata.model_params`, mapping them from
   `RunResult.SelectedBackendID` and `ExecutionTarget.BackendID` respectively.
   Keep the two values independently mapped so an upstream inconsistency is not
   hidden by Scriptorium.
4. In HTTP error mapping, classify `promptkit.ErrCapacityExceeded` before the
   general generation failure and return HTTP 503 with code
   `capacity_exceeded` and message `model backend capacity is exhausted`. Do
   not emit `Retry-After` and do not expose `CapacityError` details.
5. In CLI `run`, classify the same identity as the existing runtime-error exit
   status 1 and print the stable safe diagnostic
   `run error: model backend capacity is exhausted`. Keep all other runtime
   error handling unchanged.
6. Preserve `serveCommand`'s one-engine/one-handler construction outside the
   request path. Do not add per-request engine construction; that would split
   capacity state and violate ADR 0004.
7. Update `docs/cli.md`, `docs/api.md`, and `docs/operations.md` with backend
   identity, endpoint-only omission, capacity status, lack of retry timing,
   shared per-engine admission, and the distinction between transient
   in-flight state and durable workflow state.

### Verification

- Formatter, CLI, and HTTP tests must cover a registered backend and an
  endpoint-only profile. Assert supplied backend IDs and empty/omitted identity
  rather than endpoint-derived guesses.
- Add HTTP error-table coverage proving capacity is 503
  `capacity_exceeded`, provider failure remains 502 `llm_failed`, and neither
  response leaks wrapped provider or capacity diagnostics.
- Add one offline concurrent HTTP integration test using a single real
  Promptkit engine, a custom backend with `concurrency_limit: 1` and explicit
  `queue_capacity: 0`, and a blocking fake LLM client. Hold the first request
  in generation, issue a second request, assert the second receives the 503
  contract, release the first, assert it succeeds, and assert observed peak
  generation concurrency is one. This test owns the Scriptorium shared-engine
  interaction; do not replicate Promptkit's FIFO or broader scheduler suite.
- Run `go test -race ./internal/adapter/http ./internal/adapter/cli ./internal/format`.

### Completion Conditions

All presentations report actual backend identity when present, endpoint-only
profiles remain identity-free, and one server instance enforces one engine's
backend capacity across concurrent requests.

## Stage 6: Add Stable Prompt Inspection Output And CLI

Introduce reusable application-owned inspection presentation, then expose
prompt inspection without model execution.

### Presentation Contract

1. Generalize the internal prepared-run format selector to an internal
   `OutputFormat` with `text` and `json` values, a text default, and one parser.
   Update config and prepared-run call sites to use it without changing
   existing prepared-run bytes or `defaults.render_format` behavior.
2. Define a Scriptorium-owned prompt-inspection DTO; never marshal
   `promptkit.PromptInspection` directly. JSON contains these fields in this
   shape, including empty strings and an empty array where applicable:

   ```json
   {
     "prompt_id": "example",
     "prompt_version": "1",
     "prompt_hash": "opaque",
     "default_profile_id": "",
     "inputs": [
       {
         "name": "source",
         "required": true,
         "content_type": "text/plain",
         "description": "Input text"
       }
     ],
     "output_contract": {
       "format": "text",
       "validation_mode": "none",
       "schema_path": "",
       "repair_attempts": 0
     }
   }
   ```

3. Preserve Promptkit's declared input order. Emit JSON with deterministic
   indentation and one trailing newline. Text output uses the same field order,
   renders `inputs: []` when empty, lists input records in declaration order,
   quotes free-form descriptions safely, includes empty declared values, and
   ends in one newline. The prompt hash is opaque and must not be interpreted.

### CLI Contract And Implementation

1. Add `inspect` to top-level dispatch and usage, with implementation in a new
   focused CLI file rather than further expanding `run.go`.
2. Implement:

   ```text
   scriptorium inspect prompt --prompt ID [--prompt-version VERSION]
       [--config PATH] [--prompt-dir DIR] [--format text|json] [--out PATH]
   ```

   `--prompt` is required. All other flags are optional. Positional arguments
   and unknown inspection modes are errors. There are no deprecated aliases.
3. Apply ordinary config discovery and CLI-over-file precedence. Require an
   effective prompt directory, pass the optional version directly to
   `Engine.InspectPrompt`, and construct the engine through the shared settings
   path so configured backends are validated consistently. Do not require a
   profile or schema directory for this operation.
4. Default inspection format to text independently of
   `defaults.render_format`; `--format` is the only inspection format override.
   Reuse ordinary output-file handling. Write output only after successful
   inspection and formatting. Use exit 0 for success and exit 1 for parsing,
   configuration, load, formatting, or write failure.
5. Update `docs/cli.md`, `docs/internal/adapters.md`, and the internal format
   component description. Make clear that inspection loads and normalizes a
   definition but does not resolve a profile, load a schema, render templates,
   reserve backend capacity, or call a model.

### Verification

- Formatter tests cover deterministic text and JSON, declared order, empty
  inputs/default profile/schema path, all nonempty fields, and nil inspection
  handling. Prefer structural JSON assertions plus a few stable text fragments
  over large snapshots.
- CLI tests cover required flags, rejected positional arguments, config source
  selection, `--prompt-dir` precedence, sole and explicit multi-version
  selection, default and explicit formats, stdout versus `--out`, no partial
  output on failure, and Promptkit prompt-not-found/load identities.
- Use a fake or deliberately failing model client to prove inspection performs
  no generation; do not assert Promptkit's internal call sequence.
- Run `go test ./internal/format ./internal/adapter/cli`.

### Completion Conditions

`inspect prompt` produces stable Scriptorium text or JSON for every valid
directory-backed v0.9.0 prompt definition without inputs or model access, and
prepared-run presentation remains backward compatible.

## Stage 7: Add Stable Profile Inspection Output And CLI

Complete the inspection command family with inherited, built-in, custom, and
endpoint-only profile support.

### Presentation Contract

1. Define a Scriptorium-owned profile-inspection DTO; never marshal
   `promptkit.ProfileInspection` directly. JSON has the following complete
   shape. Keep empty strings and zero numeric values because Promptkit reports
   unresolved optional controls as zero, and normalize absent `extra_params`
   to an empty object:

   ```json
   {
     "profile_id": "example",
     "effective_model_params": {
       "backend_id": "",
       "endpoint": "https://example.test/v1",
       "model": "model-name",
       "temperature": 0,
       "max_tokens": 0,
       "top_p": 0,
       "timeout_seconds": 0,
       "service_tier": "",
       "reasoning_effort": "",
       "api_key_env": "",
       "extra_params": {}
     },
     "api_key_required": false
   }
   ```

2. Text output follows the same order, nests effective model parameters,
   encodes extra-parameter values as deterministic compact JSON with sorted
   keys, includes an empty `backend_id` for endpoint-only profiles, and ends in
   one newline. Never read or print an environment-variable value or direct
   API key.

### CLI Contract And Implementation

1. Implement the second mode:

   ```text
   scriptorium inspect profile --profile ID
       [--config PATH] [--profile-dir DIR] [--format text|json] [--out PATH]
   ```

   `--profile` is required. The common inspection parse, format, output, and
   exit rules from Stage 6 apply. No prompt, prompt directory, schema directory,
   or input is required, so built-in profiles are inspectable with only their
   ID.
2. Resolve config and `--profile-dir` with normal precedence, construct the
   engine through the shared settings path so custom backends participate, and
   call `Engine.InspectProfile` exactly once.
3. Update `docs/cli.md`, `docs/config.md`, and internal adapter/source docs.
   Explain that profile inspection resolves inheritance and backend defaults
   but does not read credentials, load a prompt, reserve capacity, or call a
   model. Document that zero provider-control values may mean unspecified,
   consistent with Promptkit v0.9.0.

### Verification

- Formatter tests cover deterministic text and JSON, registered and empty
  backend identity, optional environment-variable names, `api_key_required`,
  nested extra parameters, and the absence of credential values.
- CLI tests cover built-in, inherited custom, custom-backend, and endpoint-only
  profiles; config and `--profile-dir` precedence; default and explicit format;
  stdout and `--out`; missing and invalid profiles; no partial output; and no
  requirement for `prompt_dir`.
- Include a sentinel environment secret in a test and assert it appears in no
  inspection output or diagnostic while its variable name may appear.
- Run `go test ./internal/format ./internal/adapter/cli ./internal/config`.

### Completion Conditions

`inspect profile` reports the complete safe effective profile view for every
valid directory-backed v0.9.0 profile and configured backend combination
without requiring a prompt or contacting a model.

## Stage 8: Complete Cross-Feature Conformance And Release Validation

Close the roadmap with representative definition-level integration coverage,
copyable examples, documentation alignment, and full validation. Do not defer
contract documentation from earlier stages to this stage; this is a final
cross-check.

### Implementation And Documentation

1. Maintain one compact shared fixture set under
   `testdata/promptkit-v0.9/` with `prompts`, `profiles`, `schemas`, and any
   message content files needed by CLI and HTTP integration tests. Through a
   small set of composable definitions, cover:

   - inline and file-backed messages;
   - `developer`, `system`, `user`, and `assistant` roles and message cache
     control;
   - required and optional inputs plus template-referenced values;
   - prompt versions, default profiles, and session-ID templates;
   - text and JSON output plus `none`, `basic`, `json`, and `json_schema`
     validation, a schema path, and a valid positive repair budget;
   - standalone and inherited profiles;
   - built-in, configured custom, and endpoint-only backends;
   - model controls, service tier, reasoning effort, timeout, optional
     credential environment names, and nested JSON-compatible extra
     parameters.

   Exercise these fixtures through Scriptorium's `render` and inspection
   boundaries. Assert representative effective outcomes; do not independently
   decode YAML or exhaustively retest every Promptkit field combination.
2. Add a copyable `examples/render-v0.9-features.sh` workflow that uses only
   `render` and `inspect` operations to demonstrate prompt-version selection,
   a direct session ID, reasoning replacement or clearing, a configured custom
   backend, and both inspection modes without provider access. Keep
   `examples/config.full.yml` complete and keep raw credentials out of every
   example.
3. Review every canonical current-state owner named by the feature roadmap.
   Remove stale claims that inputs are always required, that every command
   needs `prompt_dir`, or that v0.1.0 semantics still apply. Ensure CLI, API,
   configuration, operations, internal adapter/source, development, and
   overview documents link instead of duplicating one another's contracts.
   Change architecture policy or internal component boundaries only if the
   implementation actually changed those durable responsibilities.
4. Verify that all explicit non-goals remain absent: appended messages,
   retained prepared handles, HTTP inspection routes, alternative Promptkit
   source kinds, request output-contract replacement, public provider details,
   direct catalog imports, raw API keys, and durable workflow state.
5. After every completion criterion below passes, change the feature roadmap's
   Status from `Planned` to `Complete`. Do not mark it complete for a partial
   implementation.

### Final Validation

Run all of the following from the repository root:

```bash
gofmt -w $(git ls-files '*.go')
go test ./...
go test -race ./...
go vet ./...
go build ./cmd/scriptorium
go run ./cmd/scriptorium render --config ./examples/config.yml --prompt generic.markdown_summary --input transcript=./examples/fixtures/transcript.md --input glossary=./examples/fixtures/glossary.yml
bash ./examples/render-v0.9-features.sh
git diff --check
```

Also verify the architecture guard, strict example decoding, documentation
links, executable permissions on shell examples, and absence of secrets. Review
the suite for redundant upstream-semantic tests and retain only tests that
protect Scriptorium mappings, presentation, configuration, concurrency
integration, or complete workflows.

### Completion Conditions

Every roadmap validation criterion is satisfied on the tagged dependency; all
current-state documentation and examples describe implemented behavior; the
full application supports every Promptkit v0.9.0 feature expressible in its
directory-backed prompt and profile definitions; and the ADR 0004 non-goals
remain intact.

## Open Questions

None. The external syntax, application mappings, validation ownership,
inspection wire shapes, concurrency behavior, testing boundaries, and stage
order required for implementation are fixed above.