Files

Eric Rakestraw c648d1de33 Refresh operator documentation and CLI help

2026-11-07 23:31:14 +00:00

49 KiB

Raw Blame History

Command line reference

This reference describes the standard commands available to applications using CommandKit. Replace example in commands with your application's executable name. Applications choose which optional commands and aliases to install; use example --help to see what yours provides.

Start with configuration inspection, PromptKit resource listings, or PromptKit inspection. Shared output controls apply across the suite. Flags such as --campaign in examples are application-defined, not built into CommandKit.

Go application developers should use the consumer guide for installation and API details.

Help

The root help command lists registered commands and their summaries. Help is available through a bare invocation, --help, -h, and help:

example
example --help
example -h
example help
example help config show
example config --help

help [COMMAND...] resolves an exact registered path. A command group without an execution handler displays its help when invoked by itself. A runnable command with children runs its handler when invoked without help; add --help to inspect that command and its children.

Help shows available commands, flags, defaults, and positional arguments, including which values are required or repeatable. It does not run the selected command. Unknown flags and malformed supplied values still produce usage errors, but you do not need to supply values required only for execution or satisfy execution-only combinations of inputs.

Version

version reports the application identity by default:

example version
example version --scope commandkit
example version --format yaml

The supported flags are:

Flag	Values	Default	Meaning
`--scope`	`app`, `commandkit`	`app`	Select the caller's application identity or the embedded CommandKit module identity.
`--format`	`text`, `yaml`, `json`	automatic	Select the output format; automatic uses text for version.

Text output is NAME VERSION, followed by (REVISION) when the selected identity has a revision. An empty application version is displayed as dev; the revision remains empty when omitted.

CommandKit identity uses the version of its module in build information. A versioned module replacement supplies the effective version. A local or versionless replacement, an untagged module, or missing build information displays dev. The CommandKit identity name is CommandKit; dependency build metadata does not supply a revision.

YAML and JSON output use these fields, including an empty revision:

name: ExampleApp
version: dev
revision: ""

Shell completion

Applications may install shell completion explicitly. When installed, generate and load the script for the executable name chosen by the application:

# Bash, after initializing bash-completion
source <(app completion bash)
# Zsh, after initializing its completion system with compinit
source <(app completion zsh)
# Fish
app completion fish | source

Bash requires the bash-completion package and its helpers in the current shell. If your startup files do not initialize it, source the package's initialization file before loading the generated script. For example, on systems that install it at the following path:

source /usr/share/bash-completion/bash_completion
source <(app completion bash)

Use the path provided by your Bash completion installation.

The application selects the executable basename at installation; use that same name to invoke the generated script. Completion offers canonical command and flag names, static values, and file or directory paths where declarations provide hints. A fully entered alias is resolved to its canonical command; alias spellings and partial alias paths are not suggested. Completion does not run the selected command or load its data through CommandKit's callbacks. The application may still do work during startup. See the consumer guide for installation and hint declarations. Script generation follows the reporting policy, including quiet mode.

Reporting

Use --format to choose report output and --quiet, --verbose, or --debug to control which messages appear:

example config show --format json
example config validate --verbose

Command results and diagnostic messages use separate output destinations, chosen by the application. Diagnostics include warnings, progress, errors, and usage guidance.

The shared --format flag accepts text, yaml, and json. Without it, commands with a text template use text. Other reports use text for a single string, number, boolean, or nonempty list of strings, and YAML for structured values, empty collections, and null. Explicit --format text also uses YAML for structured values without a text template.

An invalid or empty format is a usage error. The flag does not reformat raw application output, help, completion scripts, or diagnostic messages.

Modes and visibility

Each invocation starts in ordinary mode. Only one of --quiet, --verbose, and --debug can be active; if several are supplied, the last enabled one wins. They accept presence or an explicit boolean value. A false occurrence clears the selected mode only when it names the current mode, returning to ordinary mode; it does not restore an earlier selection. Invalid boolean values are usage errors, even if a later flag would select another mode.

Category	`--quiet`	No mode flag	`--verbose`	`--debug`
Command output	Suppressed	Shown	Shown	Shown
Failure explanation	Shown	Shown	Shown	Shown
Warning	Suppressed	Suppressed	Shown	Shown
Info	Suppressed	Suppressed	Shown	Shown
Debug	Suppressed	Suppressed	Suppressed	Shown
Progress, unless enabled explicitly	Suppressed	Suppressed	Suppressed	Shown

--progress independently enables Progress; --progress=false disables it. Both accept the existing boolean syntax, and the last occurrence wins. An explicit setting overrides the mode default, except quiet always suppresses Progress. Without an explicit setting, debug enables Progress; ordinary and verbose modes leave it disabled. Progress does not enable Info, Warning, or Debug.

Placement and aliases

Reporting flags are available on application commands, the root, canonical groups, help, version, and alias-only discovery help. Place them after the complete command path; they may be interspersed with operands after that path. For example:

example export --quiet
example config show --verbose
example promptkit test all --debug --progress=false
example help config show --quiet
example version --quiet

Reporting flags do not go before or between command-path components. Tokens consumed as values and tokens after -- retain their normal meaning.

The reserved long spellings are --quiet, --verbose, --debug, --progress, and --format; application declarations cannot reuse them. There are no built-in short spellings. Applications may define aliases such as -q → --quiet, -v → --verbose, and -d → --debug; canonical and aliased occurrences share the same ordering. Thus example export --verbose -q selects quiet mode.

Channels and messages

Reports use the result destination. Warnings, information, progress, debug messages, failure explanations, and usage guidance use the diagnostic destination. Simple Warning, Debug, and terminal failure messages begin with WARN:, DEBUG:, and ERROR: respectively. Info, Progress, and command output have no prefix. For multiline messages, the prefix appears on the opening line; structured details keep their own formatting without duplicate labels. Progress is streamed as unprefixed lines; terminal redraw and progress bars are not provided. A failed outcome with no message or detail uses the fallback ERROR: Command failed..

Quiet mode suppresses successful help and version output, as it does other command output. Usage guidance and routing, parsing, and operational failures remain visible in every mode. The application decides whether to report a recovered problem as a warning.

Output failures

Quiet mode hides output but still prepares and renders it, so formatting errors can still fail a command. Suppression itself is not a delivery failure.

A formatting or write failure changes an otherwise successful or usage result to status 1. Application-specific results retain their classification and nonzero status; a zero status becomes 1 on output failure. Output already written remains visible. See outcome status for status conventions and the consumer guide for Go error access.

Registered command syntax

Options are parsed after the complete command path and may be interspersed with declared operands. Flags are local to the selected command. Use -- to end option parsing; all later tokens are operands.

Value flags accept --name value and --name=value. Boolean flags accept presence as true and explicit values such as --enabled=false. A singleton flag accepts repeated occurrences; the last successfully parsed occurrence supplies its value. Every supplied occurrence is converted, so an earlier malformed value remains an error. Repeatable flags preserve occurrence order, with each occurrence treated as one value, including strings containing commas.

An explicitly supplied empty, false, or zero value is different from omitting a flag and using its default. Help and version do not require command inputs.

Application-owned aliases

Applications can install command and flag aliases alongside their canonical commands. Aliases are shortcuts to existing commands or flags. In this reference, “canonical” means the original command or flag name.

Command alias paths are sequences of command tokens relative to the application root. The executable name, flags, flag values, and operands are not part of an alias path. Matching uses complete tokens and chooses the longest matching path. A group alias preserves its remaining command tokens, and aliases can compose through chains. For example, list collections can route to promptkit list collections, while pk can route to the promptkit group. The target must resolve to a canonical command or group.

Aliases do not rewrite operands or supply arguments. In particular, promptkit test KIND remains one canonical command whose KIND is an operand; all is an operand value, not a child command. smoke → promptkit test is a valid shortcut because it preserves the caller's KIND value.

Flag aliases can be application-wide or scoped to a canonical command path. An application-wide alias is effective wherever its canonical target flag is available and is ignored elsewhere. A scoped alias requires its canonical command and target flag to be installed. The selected canonical command determines applicable scoped aliases, so the alias also works when that command is reached through another command path. A scoped mapping takes precedence over an application-wide mapping with the same spelling.

Only tokens parsed as flag names participate in flag aliasing. Values, operands, and tokens following -- remain unchanged. This includes values that look like flags, such as --message=--hlep. An alias has the same value parsing, default, required-value rules, and repetition behavior as its original flag. Mixed spellings preserve occurrence order: singleton flags use the last successfully parsed occurrence, every supplied occurrence is converted, and repeatable flags accumulate values in order.

Flag aliases accept one long spelling or one individual short spelling. Short aliases use the canonical short-option cluster and attached-value grammar:

Input	Interpretation
`-vq`	Two boolean flags, `--verbose` and `--quiet`.
`-cblue` or `-c blue`	Value `blue` for a value-taking `-c` flag.
`-vqcblue`	`-v`, `-q`, then `--color=blue`.
`-cvq`	Value `vq` for a value-taking `-c` flag.
`-v=false`	Explicit false for the boolean flag `-v`.
`-c=`	An explicitly supplied empty string.

Boolean flags use presence as true; an explicit boolean value requires =. The first value-taking short flag consumes the remaining cluster suffix or the next argument. A consumed value is never interpreted as an alias. The canonical short-option empty-value form -c= supplies an empty value, consistently with --color= and -c "".

Help reached through an alias describes the canonical command's flags and operands while using the entered route in usage. It identifies the canonical target, lists applicable aliases separately, and supports global help aliases such as --hlep → --help. Help for an alias-only prefix lists its routes without creating a command group; the bare incomplete prefix points to that help. Displaying this help does not run the target command or load its data through CommandKit's callbacks.

Usage diagnostics preserve the command path and flag spelling the caller used. When a command alias resolved successfully, usage identifies its canonical target while retaining the entered path in the diagnostic. An unresolved route receives no invented canonical label. An invalid flag value is reported using the spelling you entered and identifies the occurrence that failed.

Configuration scopes

A scope is a named configuration, such as app for application settings or promptkit for engine settings. List the scopes and operations your application provides:

example config scopes
example config scopes --format yaml

The configuration suite supports these tasks where enabled by the application:

Command	Task
`config scopes`	Find available configurations and operations.
`config validate`	Check configuration and inspect validation findings.
`config show`	View the effective settings.
`config sources`	Find the files and other sources used to load settings.
`config explain`	Discover paths or find where a value came from.
`config diff`	Compare two configurations.

Scope discovery does not load configuration. It lists only operations installed for each scope, along with supported optional features. A scope can appear even if it has no configuration loader. Text output uses one scope: operation, operation line per scope; structured output contains a scopes list with name, operations, and any supported features. Scopes appear in this order: app, promptkit, then other names alphabetically.

Temporary overrides

For scopes that support them, repeat --set POINTER=JSON to override values for one invocation. For example:

example config show --set '/server/port=8080'

config diff also accepts --against-set POINTER=JSON for its right side. Assignments reach the loader in order, labeled with the corresponding flag. The first = separates the JSON Pointer from its value, so these flags cannot address keys containing =; JSON string values can contain it. Values may be any JSON type, including null, false, zero, and empty containers. ConfigKit checks override paths and handles conflicts during loading.

For --scope all, every selected scope must support overrides. The bundled PromptKit configuration scope does not currently support them. Applications choose supported features and loaders as described in the consumer guide.

Validate configuration

config validate checks the selected configuration. It uses the app scope by default. Use --scope all to check every scope, when all support validation.

example config validate
example config validate --scope promptkit --format yaml
example config validate --scope all
example config validate --scope all --details --severity warning --code schema.version

The default output is concise text: SCOPE: valid for each successful scope. YAML and JSON reports contain a scopes list with each attempted scope and its outcome: valid, failed, or canceled.

The command checks that all selected scopes support validation before loading any of them. With --scope all, it stops at the first load failure or cancellation and reports the scopes reached. Reporting happens after loading has stopped. Warnings use the diagnostic destination and do not by themselves make validation fail.

Detailed validation findings

--details includes validation findings and the source references needed to interpret them. Each scope's detail state is unavailable, empty, filtered, or present. Repeat --severity error|warning or --code CODE to filter findings: values within a filter match any, and the two filters narrow together. Filters require --details and never change the validation outcome. Detailed output suppresses ordinary warnings associated with its native findings; other messages and terminal failure explanations remain visible. An explicitly empty --config or --overlay differs from omission; the application's loader decides how to interpret it. Help and invalid syntax do not load configuration through the scope providers.

Show effective configuration

config show displays the settings after loading and resolution, using the provider's safe view with secrets redacted. The default scope is app.

example config show
example config show --scope promptkit
example config show --scope all
example config show --path /server/addr --format json

The report preserves value types, empty collections, and redaction. Automatic formatting uses text for a string, number, boolean, or nonempty string list and YAML for other values; --format selects text, YAML, or JSON.

--scope all loads every selected scope before producing one mapping keyed by scope name, even when only one scope is registered. Selecting one scope directly returns its configuration without a scope wrapper. A load failure prevents report output; conversion and write failures follow the shared output failure rules.

For a scope that supports path-selected show, --path POINTER displays only that safe effective value. An explicitly empty pointer selects the root, and a present null remains a value. Missing and historical-only paths fail; path selection cannot be combined with --scope all.

List configuration sources

config sources lists the files and other sources used to load the selected configuration. The default scope is app; --scope all is unsupported.

example config sources
example config sources --source-id source-1 --format yaml

Use an ID from the source list with --source-id to select one entry; an unknown ID fails. Source IDs belong to that load. The command reads the recorded list once, without reloading configuration. If loading fails but a source list is available, it still reports the list and retains the failure. Otherwise it reports that source information is unavailable.

Automatic output is text. Use --format yaml or --format json for structured output.

Source report fields

scope: app
sources:
  - id: source-1
    label: "application config"
    path: /etc/example/config.yml
    layer: main
    kind: file
    underlay_index: 0
    imported_from: null
    destination: /server
    placement: mapping
    destination_is_fallback: false
    version_evidence:
      path: /version
      version:
        present: true
        null: false
        text: "1"
      location:
        source_id: source-1
        line: 2
        column: 1
      redacted: false
    empty_import_selections: []

Each source includes id, label, path, layer, kind, underlay_index, and imported_from, plus the native destination pointer, placement, destination_is_fallback, nullable version_evidence, and empty_import_selections. Placement is mapping or sequence_item. The import location is null when absent; otherwise it has source_id, line, and column. Version evidence preserves path, present, null, text, optional source location, and redacted as returned by ConfigKit. Values and source order are preserved, including empty fields, zero coordinates, false and null observations, and synthetic sources without paths. Absent version evidence is null. Each empty-import record has a source-local path, a location mapping (source_id, line, column), and a selector; records preserve their returned order. No records are empty_import_selections: []. An empty inventory is sources: []. Text output begins with the scope heading and prints the same source fields, quoting labels and paths. These fields come from ConfigKit's source records; producing the report does not reload files or rerun version checks. See the ConfigKit v1.1.0 source records for native evidence definitions.

Explain a configuration path

config explain --path POINTER explains the effective and historical value at a JSON Pointer path. An explicitly empty path selects the root. config explain --list discovers paths when the selected scope supports path discovery. An invocation with neither mode displays help; the two modes cannot be combined. The default scope is app; --format yaml or --format json selects structured output, and automatic output uses text. --scope all is unsupported.

example config explain --path /server/addr
example config explain --path /profiles/team~1summary --format yaml
example config explain --path= --format yaml
example config explain --list --under /profiles --historical

Listing normally includes current paths. --historical adds paths that only occurred in history; --under POINTER selects a subtree, and --explicit=true or --explicit=false filters current paths by native explicitness. Historical-only paths do not match an explicitness filter. These options require --list. The report contains the scope and each selected path's current presence and, when present, its explicitness. Path discovery does not include values or history.

Pointer tokens use JSON Pointer escaping: ~1 represents / and ~0 represents ~. A malformed pointer is a usage error with status 2 and is rejected before loading configuration. A well-formed path that is unknown in the selected configuration is an operational error with status 1. Text, YAML, and JSON show the same explanation, available field metadata and metadata history, and source labels.

Explanation report fields

Structured output uses these root fields:

scope: app
path: /server/addr
effective:
  present: true
  redacted: false
  yaml: "localhost:8080\n"
origin:
  layer: main
  locations:
    - source_id: source-1
      line: 4
      column: 9
  operation: ""
  inheritance: []
history: []
secret_reference: null

Snapshot yaml values remain strings, including null and empty fragments. Origin and history retain source locations, operation names, inheritance paths, ordered reasons and dependencies. Secret references contain only the owning environment-variable name; effective and historical secret fragments remain redacted. Empty location, inheritance, history, reason, and dependency lists are serialized as [].

Compare configuration selections

config diff shows the differences between two configurations of the same scope. It uses app by default; --scope all is unsupported.

example config diff --config before.yml --against-config after.yml
example config diff --config candidate.yml

The second example compares candidate.yml with the configuration the application ordinarily loads without overrides.

--config and --overlay select the left side. --against-config and --against-overlay select the right side. Each omitted field uses that side's ordinary application loading behavior; the right side does not inherit the left side's flags. An explicitly empty string remains an explicit selection. Application-defined selectors affect the left side, and their --against-* counterparts affect the right side when available. Equal selections are allowed and still run the comparison.

The application must enable comparison for the selected scope. The bundled ConfigKit and PromptKit configuration adapters both support it.

Automatic output is YAML. The shared --format also accepts YAML, JSON, and text; text uses the same structured representation. For example:

example config diff --config base.yml --against-overlay candidate
example config diff --config base.yml --overlay current --against-overlay ""
example config diff --scope promptkit --config before.yml --against-config after.yml
example config diff --config before.yml --against-config after.yml --details --check

Detailed differences and check mode

--details replaces the concise report with {scope, differences}, containing one entry per difference returned by the provider. Each entry has path, before, after, and hidden_changed; each snapshot has present, redacted, and value. An absent snapshot has present: false and a null value, distinct from a present null. Empty comparisons contain differences: [].

--check returns status 3 when differences exist and 0 when they do not. Without it, a completed comparison succeeds even when differences exist. Usage errors retain status 2, and operational failures retain status 1. The report is still delivered in check mode; a presentation failure remains separate evidence and does not replace a nonzero difference outcome.

Interpreting the comparison

For ConfigKit-backed scopes, comparison uses the effective values after defaults and resolution. A change only in where a value came from does not count as a value change. PromptKit uses ConfigKit for its comparison too. The concise report contains additions, changes, and deletions. Additions and changes keep the configuration's path structure and show right-side values from the comparison. Deletions are JSON Pointer paths. Explicit null is a value, not a deletion. Changed lists and regions replaced as a whole are shown as whole values.

ConfigKit considers sensitivity on both sides. A secret value can change even when both redacted snapshots look the same. For those ConfigKit-backed comparisons, the concise report omits separate redaction and hidden-change fields: a reported masked region changed, but not necessarily every value in it. Public strings that look like redaction markers remain ordinary strings. An empty comparison succeeds with this shape:

additions: {}
changes: {}
deletions: []

The concise report is a partial view with secrets concealed. Do not load it as configuration or apply it as a patch. Changes involving an empty or null value at the configuration root can be ambiguous because the report has no wrapper identifying the whole-root value. Null values and deletions of individual fields remain distinct.

Without --check, successful comparisons return status 0, including empty results. Unsupported scopes and formats are usage errors. Application input checks can report usage or operational failures. Load and comparison failures use status 1.

The bundled adapters load the left side before the right. Diagnostics identify the side and its sources; left-side diagnostics are retained if right loading fails. Custom application loaders and caches determine whether sources are read again for each side. See the consumer guide for adapter and loader responsibilities.

The complete report is prepared before writing. Preparation failures and cancellation noticed before delivery leave the report output empty. Once writing starts, cancellation does not interrupt the writer or undo delivered bytes; a write failure can leave partial output.

PromptKit definition inspection

Use promptkit show to inspect a stored prompt, profile, or backend without running a prompt or probing the model provider. Applications can enable these commands independently.

example promptkit show prompt customer/summary
example promptkit show profile customer-support --format json
example promptkit show backend local --format yaml

Each command takes one resource ID and accepts --config and --overlay. Reports show PromptKit's inspection data. Automatic output and explicit text use YAML for these structured values; --format json selects JSON.

Inspection report details

PromptKit's JSON field names and omission rules apply. Absent optional fields are omitted; explicit zero values and empty collections remain present. Prompt messages are captured content, not rendered requests, and the prompt hash is opaque.

Profile reports describe the selected profile before backend completion or request overrides. Backend reports describe configured backend fields. An api_key_env value names an environment variable; it is not a credential value or a claim that one is available. Neither report resolves references or performs a network probe. See the PromptKit resource inspection contracts. Omitting the ID is a usage error. Lookup failures and cancellation follow the shared operational failure rules.

Resource namespace selection

When enabled by the application, namespace flags choose the namespace for a short resource ID. Omitting a flag keeps the application's default; an explicitly empty value clears that default. Already qualified IDs pass through unchanged. If these flags are absent from help, the application controls namespace selection through its engine loader.

Operation	Available namespace flags
`show prompt`	`--llm-prompt-namespace`
`show profile`	`--llm-profile-namespace`
`show backend`	`--llm-backend-namespace`
`requirements show`	`--llm-prompt-namespace`, `--llm-profile-namespace`
`requirements validate`, `show prepared`, `test`	All three, including `--llm-backend-namespace`

Catalog namespace filters remain separate; configuration inspection has no resource namespace selectors.

PromptKit convenience aliases

Applications can enable shorter flag spellings within the PromptKit commands. Each applicable --llm-NAME flag also accepts --NAME; --llm-input-text additionally accepts --llm-input and --input for literal text. Use --llm-input-file or --input-file for a file reference. The original name and its aliases behave identically, including when repeated or supplied with an empty value.

Use help to see whether your application enables these aliases. They do not apply to commands outside promptkit unless the application installs them there as well. See the consumer guide for installation choices.

PromptKit prepared-request preview

promptkit show prepared PROMPT_ID previews the rendered messages and request settings without calling the model. When enabled by the application, it accepts the same configuration, application selector, profile, session, variable, input, and execution flags as request validation:

example promptkit show prepared briefing --llm-input-text topic=roadmap --format json

The command returns PromptKit's prepared-run report; automatic and explicit text output use YAML. Preparation can read resources and include rendered application content in the report. It does not guarantee that a later model call will succeed. Application loaders and callbacks may have their own effects. The PromptKit Prepare API owns the prepared value and upstream preparation behavior.

PromptKit requirements

promptkit requirements show lists the inputs and variables a prompt may need. Supply one prompt ID. Variables can help PromptKit determine which conditional inputs are needed:

example promptkit requirements show briefing --campaign spring
example promptkit requirements show briefing --llm-profile editorial --llm-var mode=full
example promptkit requirements show briefing --llm-session-id session-42 --llm-var note= --format yaml

The command accepts:

Flag	Values	Default	Meaning
`--config`	path string	omitted	Select the configuration input; explicit presence is retained.
`--overlay`	path string	omitted	Select one overlay; explicit presence is retained. Repeated occurrences use the last value.
Application selectors	application-defined	per selector	Additional flags provided by your application.
`--llm-profile`	string	empty	Pass the profile ID to PromptKit; empty uses PromptKit's selection rule.
`--llm-session-id`	string	empty	Pass session context to PromptKit.
`--llm-var`	repeatable `KEY=VALUE`	none	Supply a variable to help discover required inputs.
`--format`	`text`, `yaml`, `json`	automatic	Select the requirements report format; automatic and text use YAML for this structured report.

These commands work with PromptKit directly and have no built-in --scope flag. PromptKit interprets an empty profile or session value according to its own selection rules. Profile selection establishes the reported profile identity rather than changing the discovered requirements. An enabled profile namespace qualifies an explicitly supplied short profile ID; it does not rewrite the prompt's stored default reference. Requirements inspection does not require the selected profile to be execution-ready. See the PromptKit v0.20.0 consumer guide for configuration, selection, and requirements behavior. For built-in flags other than --llm-var, repeating a flag uses its last value. Application-defined flags follow their own repetition rules.

Each --llm-var occurrence must contain =. CommandKit splits at the first separator and preserves the exact key and remaining value. Empty keys and values, whitespace, commas, and additional = characters are retained. All occurrences are parsed; duplicate keys use the last value, while any malformed occurrence is a usage error even if a later assignment is valid.

Requirements report

The report identifies the selected prompt and profile, whether discovery is complete, and the inputs, variables, and issues found. False values and empty inventories are retained. Input metadata preserves null versus an empty string. Variable entries contain name, kind, supplied, and explanation. supplied reports whether the exact variable name was provided, including an explicitly empty value, independently of kind and complete. Explanations and issue descriptions are upstream-owned human-readable summaries, not machine-readable details.

For a prompt with the template {{if .mode}}{{input "notes"}}{{end}} and no input declarations, omitting mode can produce a partial report:

prompt_id: briefing
profile_id: editorial
complete: false
inputs:
  - name: notes
    kind: conditional
    declared: false
    content_type: null
    description: null
    explanation: This reference is not known to be reached on every execution path.
variables:
  - name: mode
    kind: required
    supplied: false
    explanation: ""
issues:
  - code: missing_variable
    input_name: ""
    variable_name: mode
    description: A missing variable prevents selecting a dependency-bearing branch.

PromptKit determines these findings and explanations. A partial report can still be a successful command result (status 0); read complete and issues to understand what remains unknown. A reported issue alone does not fail this command. Load or selection failures, cancellation, and report delivery failures use status 1. Invalid syntax, arguments, variable assignments, or formats use status 2.

Concrete request validation

promptkit requirements validate checks whether PromptKit can prepare a request with the inputs you supply. It accepts one prompt ID and does not call the model. Applications can enable it independently of requirements discovery:

example promptkit requirements validate briefing --campaign spring --llm-input-file topic=topic.txt
example promptkit requirements validate briefing --llm-profile editorial --llm-session-id session-42 --llm-var mode=full --llm-input-file notes=notes.txt

The command accepts:

Flag	Values	Default	Meaning
`--config`	path string	omitted	Select the configuration input; explicit presence is retained.
`--overlay`	path string	omitted	Select one overlay; explicit presence is retained. Repeated occurrences use the last value.
Application selectors	application-defined	per selector	Additional flags provided by your application.
`--llm-profile`	string	empty	Pass the profile ID to PromptKit.
`--llm-session-id`	string	empty	Pass session context to PromptKit.
`--llm-var`	repeatable `KEY=VALUE`	none	Supply caller variables for the concrete request.
`--llm-input-text`	repeatable `NAME=TEXT`	none	Supply text for a named prompt input.
`--llm-input-file`	repeatable `NAME=PATH`	none	Supply a file for a named prompt input.
`--format`	`text`, `yaml`, `json`	automatic	Select the success confirmation format; automatic and text use YAML for this structured report.

Validation also accepts the execution controls from --llm-endpoint through --llm-extra-params listed under smoke testing. For built-in flags other than --llm-var, --llm-input-text, and --llm-input-file, repeating a flag uses its last value. Explicitly empty --config and --overlay values remain distinct from omitted flags. PromptKit decides how empty profile and session values affect selection.

Each --llm-var, --llm-input-text, and --llm-input-file occurrence must contain =. CommandKit splits at the first separator and preserves the exact key/name and remaining value. PromptKit validates empty names and values. Text is passed as a literal input. File paths are passed unchanged to PromptKit; argument parsing does not read the files or normalize, expand, or rebase paths. Every occurrence is parsed; for each input name the last text or file occurrence wins, regardless of form.

On success, the confirmation contains only the requested prompt ID and valid: true:

prompt_id: briefing
valid: true

Validation delegates concrete request preparation to PromptKit. Preparation may resolve prompts and profiles, read input files and schemas, render messages and session context, and check configuration and credentials where applicable. It does not make a model call or retain a prepared execution snapshot. A successful report describes this preparation invocation; it does not certify provider acceptance or guarantee later execution. See PromptKit's Engine.ValidateRequest API.

Request construction, engine loading, and validation failures use status 1, retaining available diagnostics. Malformed syntax, missing arguments, and unsupported formats use status 2. Messages follow the common reporting policy; report delivery failures follow the shared output failure rules.

PromptKit catalogs

Use promptkit list to find available resources. Your application chooses which listings to enable:

example promptkit list prompts
example promptkit list profiles --config settings.yml --campaign spring
example promptkit list backends --namespace team
example promptkit list namespaces --kind prompt --kind profile
example promptkit list collections

All listings accept --config, --overlay, application-defined selector flags, and the shared --format flag. Automatic output uses text for nonempty string lists and YAML for other inventories; explicit text uses the same rendering for structured inventories. Prompt, profile, and backend listings accept namespace selection:

Binding	Effect
omitted `--namespace` and `--all-namespaces`	Use PromptKit's default namespace filter.
`--namespace team`	List resources in that namespace.
`--namespace ""`	Select the unqualified namespace.
`--all-namespaces`	List resources from all namespaces.
`--all-namespaces=false`	Do not select an override; a supplied namespace still applies, otherwise use PromptKit's default.

If --namespace is repeated, its last value is used. Supplying a namespace while --all-namespaces=true is active is a usage error before loading resources. Namespace listings accept repeatable --kind KIND; omitting --kind uses PromptKit's default kind selection. Repeated values are forwarded in order. Collections have no namespace or kind filter.

Listing results

Each command returns a list in the provider's order, without an outer wrapper. The entries are:

Prompt entries are {id}.
Profiles, backends, and namespaces are strings.
Collection entries are {namespace, version?}, where version projects PromptKit's optional release version.
An empty list renders as [].

- id: team/briefing

- autumn
- spring

A catalog result is an inventory, not a readiness check or execution result. The owning provider controls membership and order. Namespace filters are passed to PromptKit without local membership interpretation. See the PromptKit v0.20.0 consumer guide for resource and selection definitions.

PromptKit smoke testing

promptkit test KIND runs a PromptKit smoke test using the application's configured client or network probes. Unlike inspection and validation, these tests can contact a provider and request model output. The application must enable smoke testing.

example promptkit test text_inference --campaign spring --llm-prompt briefing --llm-profile spring
example promptkit test all --campaign spring --llm-prompt briefing --llm-profile spring
example promptkit test authentication --llm-backend local

The test kinds are:

Kind	Purpose
`reachability`	Check whether the endpoint can be reached.
`authentication`	Check authentication using the configured diagnostic support.
`text_inference`	Check text generation.
`structured_output`	Check structured output.
`all`	Run the four tests above in that order.

Support and applicability depend on PromptKit and the configured provider. An unsupported or unknown observation is not a positive result.

Interpreting smoke results

Status 0 means the requested operation and reporting completed without error. It does not mean every check passed: inspect the reported facts, reason, and whether the test was supported, applicable, and completed. For all, all four operations must return without error for status 0. Individual errors use status 1; the sequence continues after them but stops on cancellation, preserving results already obtained. A report returned with an error is still shown. Invalid syntax, arguments, formats, or conflicting namespace flags use status 2.

Smoke-test flags

Flag	Values	Default	Meaning
`--config`, `--overlay`	path strings	omitted	Select PromptKit configuration; supplied presence is retained.
Application selectors	application-defined	per selector	Additional flags provided by your application.
`--llm-backend`	string	empty	Select a backend by ID.
`--llm-prompt`	string	empty	Select a prompt by ID.
`--llm-profile`	string	empty	Select a profile by ID.
`--llm-endpoint`	string	omitted	Override the execution endpoint.
`--llm-model`	string	omitted	Override the execution model.
`--llm-temperature`	float	omitted	Override execution temperature.
`--llm-max-tokens`	integer	omitted	Override execution maximum tokens.
`--llm-top-p`	float	omitted	Override execution top-p.
`--llm-timeout-seconds`	integer	omitted	Override the execution timeout.
`--llm-service-tier`	string	omitted	Override the execution service tier.
`--llm-reasoning-effort`	string	omitted	Override execution reasoning effort.
`--llm-api-key-env`	string	omitted	Name the environment variable containing the API key.
`--llm-transient-retries`	integer	omitted	Set the retry count for transient inference failures.
`--llm-extra-params`	JSON object	omitted	Supply provider parameters as a whole map.
`--format`	`text`, `yaml`, `json`	automatic	Select the report format; automatic and text use YAML for these structured reports.

These execution flags override settings for the selected operation. The --llm-api-key-env value is a variable name, not a credential value. The extra-parameters flag accepts one JSON object, including an empty object; PromptKit determines how it replaces existing parameters and validates it. Integer JSON numbers use signed or unsigned 64-bit integers where representable; decimal and exponent numbers use floating point.

The application chooses how to load configuration, supply credentials, and configure its client and probes. It may customize the request before passing it to PromptKit; consult its documentation for application-specific behavior.

Smoke report fields

A single report contains PromptKit's findings. False, unknown/null, and incomplete values are preserved; CommandKit does not add a readiness score. The report includes kind, supported, applicable, completed, reason, target, facts, timing, attempts, usage, and failure. Its target contains selected IDs, selection source, endpoint, effective settings, and settings-presence flags. Attempt entries retain their own facts, timing, failure, and token-usage evidence. Durations use Go duration strings; token counts preserve value, known, and complete.

For example, this report says the configured HTTP diagnostic is unavailable:

kind: reachability
supported: false
applicable: null
completed: true
reason: http_diagnostics_unavailable
target:
  backend_id: local
  profile_id: spring
  prompt_id: briefing
  default_source_backend_id: ""
  selection_source: profile
  endpoint: https://example.invalid/v1
  effective_settings:
    backend_id: local
    endpoint: https://example.invalid/v1
    model: spring-model
    temperature: 0
    max_tokens: 0
    top_p: 0
    timeout_seconds: 600
    service_tier: ""
    reasoning_effort: ""
    api_key_env: ""
    extra_params:
      campaign: spring
  settings_presence:
    temperature: false
    max_tokens: false
    top_p: false
    timeout_seconds: false
facts:
  credentials_supplied: null
  endpoint_reachable: null
  credentials_accepted: null
  response_received: null
  generation_completed: null
  text_returned: null
  expected_text_matched: null
  structured_output_requested: null
  valid_json: null
  schema_matched: null
timing:
  started_at: 2026-10-17T19:05:36Z
  duration: 35.685µs
  capacity_wait: null
  request_duration: null
  retry_delay: null
  evaluation_duration: null
attempts: []
usage:
  prompt_tokens: {value: 0, known: true, complete: true}
  completion_tokens: {value: 0, known: true, complete: true}
  total_tokens: {value: 0, known: true, complete: true}
  cached_tokens: {value: 0, known: true, complete: true}
  cache_write_tokens: {value: 0, known: true, complete: true}
failure: ""

Aggregate output is one mapping with ordered tests entries. Each entry has selected kind, nullable report, and optional error text. The inner report keeps the kind returned by PromptKit. This cancellation example shows two attempted operations with no reports; later kinds are absent because cancellation stops the sequence:

tests:
  - kind: reachability
    report: null
    error: "reachability operation failed"
  - kind: authentication
    report: null
    error: "context canceled"

Report formatting and write failures follow the shared output failure rules. Application developers can inspect operation causes separately from delivery failures through the reporting API.

Native configuration, selection, and diagnostic meanings remain owned by PromptKit. See the v0.20.0 consumer guide, smoke API, and engine API.

Outcome status

Status	Standard meaning
`0`	The operation succeeded.
`1`	Loading, execution, cancellation, or output failed.
`2`	The command syntax or supplied inputs were invalid.
`3`	`config diff --check` found differences.

Applications may define additional results and decide how CommandKit's status maps to the process exit code. A command's successful status does not replace reading its report: partial requirements discovery and unsupported smoke checks can return status 0.

An output failure changes a successful or usage result to status 1. An application-specific result retains its classification and nonzero status; a zero status becomes 1 on output failure. Go callers can inspect operation errors and output errors separately; see the consumer guide.

49 KiB Raw Blame History