Add bounded sources and strict YAML parsing
This commit is contained in:
79
docs/config.md
Normal file
79
docs/config.md
Normal file
@@ -0,0 +1,79 @@
|
||||
# Configuration Input Contract
|
||||
|
||||
This document owns ConfigKit's currently implemented source-input and YAML syntax
|
||||
contracts. The parser and owned document representation are private; a public typed
|
||||
loading lifecycle has not been implemented yet.
|
||||
|
||||
## Sources
|
||||
|
||||
ConfigKit exposes opaque sources through these constructors:
|
||||
|
||||
| Constructor | Current contract |
|
||||
| --- | --- |
|
||||
| `File(path)` | Requires a nonempty path, opens a regular file for each consumption, and closes only the library-owned file handle. Missing, inaccessible, unreadable, directory, and other nonregular inputs fail. |
|
||||
| `Bytes(label, data)` | Requires a nonempty, non-secret diagnostic label and snapshots bytes during construction. Later caller mutation cannot affect the source. |
|
||||
| `Reader(label, input)` | Requires a nonempty, non-secret label and non-nil reader. The reader is claimed once across all copies of the source, consumed but never closed, and cannot be reused. |
|
||||
| `NoSource()` | Represents an explicit absence of input and creates no source record. Its defaults-only lifecycle behavior will become available with `Load`. |
|
||||
|
||||
The zero `Source` is invalid. Constructor argument failures are retained inside the
|
||||
opaque source and become safe structured diagnostics when the source is consumed.
|
||||
File sources can observe file changes between consumptions; byte sources cannot.
|
||||
|
||||
Each source is limited to 8 MiB. A load will be limited to 64 MiB across all source
|
||||
and override input when those layers are implemented. Reads request at most the
|
||||
remaining allowance plus one byte so excess data is rejected rather than silently
|
||||
truncated. Data returned together with a read error is not accepted as partial input.
|
||||
|
||||
## YAML documents
|
||||
|
||||
The private YAML boundary currently accepts exactly one nonempty document whose root
|
||||
is a mapping. Comments and block or flow collections are supported. It rejects:
|
||||
|
||||
- empty input or an empty document;
|
||||
- every additional document, including an empty one;
|
||||
- duplicate decoded mapping keys;
|
||||
- non-string mapping keys;
|
||||
- explicit tags, anchors, aliases, and merge keys;
|
||||
- a quoted or plain mapping key equal to `<<`; and
|
||||
- input exceeding the fixed depth or node limits.
|
||||
|
||||
Mapping key comparison is exact: it performs no case folding or Unicode
|
||||
normalization. The private document tree retains declaration order, exact validated
|
||||
number spelling, source identifiers, and separate key/value line and column
|
||||
locations. It contains no upstream parser nodes or types.
|
||||
|
||||
Root depth is zero. Input depth is limited to 256 and each parsed input tree to
|
||||
1,000,000 nodes, counting mapping keys as input nodes. The upstream parser also has
|
||||
a private calibrated depth guard; passing that guard never bypasses ConfigKit's tree
|
||||
limits.
|
||||
|
||||
## Scalars
|
||||
|
||||
Scalar meaning is determined by ConfigKit rather than upstream coercion:
|
||||
|
||||
| Input form | Meaning |
|
||||
| --- | --- |
|
||||
| Quoted, literal, or folded scalar | String |
|
||||
| Plain `true` or `false` | Boolean |
|
||||
| Plain `null`, `~`, or an empty mapping value | Null |
|
||||
| JSON decimal number | Number, retaining its exact spelling |
|
||||
| Date/time-looking or other ordinary plain text | String |
|
||||
|
||||
JSON numbers permit an optional minus, a zero or nonzero decimal integer without a
|
||||
leading zero, an optional fraction with at least one digit, and an optional decimal
|
||||
exponent. Hexadecimal, octal, digit separators, leading plus, leading/trailing-dot
|
||||
forms, incomplete exponents, and non-finite values are rejected. Legacy boolean
|
||||
words (`yes`, `no`, `on`, and `off`) and non-lowercase variants of the reserved
|
||||
boolean/null words are rejected unless quoted.
|
||||
|
||||
Mapping keys use the same scalar classification and must classify as strings. Quote
|
||||
keys that otherwise classify as a boolean, number, or null.
|
||||
|
||||
## Safe failures
|
||||
|
||||
Source failures use stable `source.*` codes and YAML failures use stable `yaml.*`
|
||||
codes declared by the root package. Diagnostics contain safe classifications,
|
||||
semantic paths when applicable, and available source coordinates. Raw parser,
|
||||
conversion, reader-error, and input text is not forwarded. Source labels and
|
||||
diagnostic messages are application-visible and therefore must never contain
|
||||
secrets; terminal control characters are escaped by `WriteDiagnostics`.
|
||||
@@ -53,6 +53,14 @@ Assigned to the YAML/source implementation:
|
||||
ConfigKit's scalar grammar and limits, and replace raw parser errors with safe
|
||||
diagnostics. Narratio's application decoders and YAML node types are not copied.
|
||||
|
||||
Implementation record (2026-09-06): the labeled byte/reader ownership and strict
|
||||
single-document cases above were generalized into `source.go`, `parse.go`, and
|
||||
`internal/yaml`. No production line was copied verbatim. ConfigKit adds byte and
|
||||
tree ceilings, shared atomic reader claims, regular-file checks, exact numeric
|
||||
lexemes, parser-depth calibration, ConfigKit-owned locations, safe diagnostic
|
||||
codes, and an AST import-boundary regression. Application typed decoding, parser
|
||||
errors, YAML node types, and session-specific loaders were not retained.
|
||||
|
||||
### Strict composition
|
||||
|
||||
Assigned to document composition:
|
||||
|
||||
@@ -5,10 +5,15 @@ module and package documentation boundary. Its implemented public primitives are
|
||||
|
||||
- immutable semantic `Path` construction and strict JSON Pointer parsing;
|
||||
- detached provenance and diagnostic record types with empty `Metadata` queries;
|
||||
- canonical, control-safe diagnostic rendering and structured `LoadError` access.
|
||||
- canonical, control-safe diagnostic rendering and structured `LoadError` access;
|
||||
- opaque file, byte, reader, and no-input source construction and bounded reads;
|
||||
- a parser-independent ordered tree in `internal/document`; and
|
||||
- the sole upstream YAML integration in `internal/yaml`, which translates validated
|
||||
input into that owned tree and exposes no parser types outside its boundary.
|
||||
|
||||
It intentionally exposes no loading, source, parsing, or decoding API yet. Private
|
||||
metadata population will be integrated with loading when that lifecycle exists.
|
||||
The [configuration input contract](../config.md) owns implemented source and YAML
|
||||
behavior. ConfigKit intentionally exposes no typed loading or decoding API yet.
|
||||
Private metadata population will be integrated when that lifecycle exists.
|
||||
|
||||
The source-extraction inventory records the application foundations assigned to
|
||||
future implementation work:
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
## Status and ownership
|
||||
|
||||
Status: implementation in progress; Stages 1–2 are complete and later stages are
|
||||
Status: implementation in progress; Stages 1–3 are complete and later stages are
|
||||
pending.
|
||||
|
||||
Audience: a `gpt-5.6-terra` coding agent implementing one stage per prompt, in
|
||||
@@ -383,7 +383,30 @@ Acceptance:
|
||||
|
||||
## Stage 3 — Extract source reading and enforce YAML policy
|
||||
|
||||
Status: pending.
|
||||
Status: complete.
|
||||
|
||||
Completion evidence (2026-09-06):
|
||||
|
||||
- Added opaque source constructors and bounded file/byte/reader acquisition in
|
||||
[`source.go`](../../source.go), including byte snapshots, regular-file checks,
|
||||
deterministic I/O codes, caller-owned reader lifetime, and one atomic claim
|
||||
shared by source copies. `NoSource` creates no source record.
|
||||
- Pinned `go.yaml.in/yaml/v4 v4.0.0-rc.6` solely behind `internal/yaml` and added
|
||||
the parser-independent ordered tree in `internal/document`. The adapter uses the
|
||||
streaming loader, checks for a second document explicitly, applies a calibrated
|
||||
private parser depth plugin, enforces owned depth/node limits, preserves numeric
|
||||
spelling and locations, and returns only value-free internal issues.
|
||||
- Generalized the strict-source/document cases recorded in the [extraction
|
||||
inventory](../internal/extraction.md), created the [current configuration input
|
||||
contract](../config.md), and added an AST import-boundary regression. No typed
|
||||
decoder, composition, filesystem graph, or public `Load` was introduced.
|
||||
- Tests cover real file failures and reopening, byte ownership, fragmented/error
|
||||
readers, exact/overflow byte limits, concurrent copied-source claims, all YAML
|
||||
policy categories, trailing/empty documents, duplicate locations, lexical wide
|
||||
numbers, block/flow depth calibration, node limits, safe parser errors, and
|
||||
adapter isolation. Verified `go test ./...`, `go vet ./...`, `go build ./...`,
|
||||
`go mod tidy -diff`, and `git diff --check` with Go 1.26.7 on linux/amd64. No
|
||||
deviations were required; typed binding and decoding remain later work.
|
||||
|
||||
Read: feature [YAML policy](initial.md#yaml-and-scalar-policy),
|
||||
[source constructors](initial.md#entry-point-and-reusable-application-definition).
|
||||
|
||||
@@ -423,31 +423,10 @@ configuration is never returned as a successful, usable result.
|
||||
|
||||
## YAML and scalar policy
|
||||
|
||||
The supported model is mappings with string keys, sequences, scalars, and null,
|
||||
with YAML comments and block/flow syntax. Require exactly one document with a
|
||||
mapping root. Reject empty input, additional documents (including empty ones),
|
||||
duplicate mapping keys, explicit tags, anchors, aliases, and merge keys (`<<`).
|
||||
Mapping keys must resolve to strings under this scalar policy; quote keys
|
||||
that would otherwise be booleans, numbers, or null. Compare decoded string keys
|
||||
without case folding or Unicode normalization. A quoted `"<<"` is also reserved and rejected.
|
||||
|
||||
Use a deliberately limited scalar vocabulary independent of parser defaults:
|
||||
|
||||
- Quoted and block scalars are strings.
|
||||
- Plain `true` and `false` are booleans; plain `null`, `~`, and an empty value
|
||||
are null. Other case variants of these reserved literals are rejected when used as
|
||||
plain values.
|
||||
- Numbers use JSON decimal syntax: optional minus, integer part with no leading
|
||||
zero except zero itself, optional fraction, and optional decimal exponent.
|
||||
- Reject YAML numeric extensions such as hexadecimal, octal, digit separators,
|
||||
leading plus, and non-finite values. Reject legacy boolean words such as
|
||||
`yes`, `no`, `on`, and `off` unless quoted (case-insensitively).
|
||||
- Date/time-looking plain text remains a string, never an implicit timestamp.
|
||||
Other ordinary plain text is a string.
|
||||
|
||||
Unsupported scalar forms must produce a policy error rather than inherit a
|
||||
parser conversion. Preserve numeric lexical information until decoding so
|
||||
large integers are not rounded through an intermediate floating-point value.
|
||||
Source ownership, the supported YAML subset, scalar grammar, fixed input limits,
|
||||
and safe failure behavior are implemented. The [current configuration input
|
||||
contract](../config.md) now owns those definitions. Typed scalar conversion into
|
||||
application values remains target work in [typed decoding](#typed-decoding).
|
||||
|
||||
## Paths and presence
|
||||
|
||||
@@ -770,11 +749,6 @@ type FilesystemContext struct {
|
||||
BaseDir string
|
||||
Root string
|
||||
}
|
||||
|
||||
func File(path string) Source
|
||||
func Bytes(label string, data []byte) Source
|
||||
func Reader(label string, input io.Reader) Source
|
||||
func NoSource() Source
|
||||
```
|
||||
|
||||
`Load` accepts zero or one options value; multiple values are an options error,
|
||||
@@ -792,11 +766,11 @@ result. Callbacks must not retain these contexts or configuration pointers; shar
|
||||
closures must be safe for concurrent calls if the application loads concurrently.
|
||||
ConfigKit does not synchronize application-owned captured state.
|
||||
|
||||
`Source` is opaque with the constructors above, not an arbitrary provider
|
||||
interface. `Bytes` snapshots its input; `Reader` is consumed during the load and
|
||||
is not closed by ConfigKit. A reader source is single-use. File sources are read
|
||||
on each load. Constructor argument errors surface as load diagnostics. Applications must use
|
||||
non-secret source labels because labels appear in diagnostics and source reports.
|
||||
`Source` and its constructors are implemented; [`source.go`](../../source.go) and
|
||||
the [configuration input contract](../config.md#sources) own their current behavior.
|
||||
The loading and filesystem-option behavior below remains target work.
|
||||
Applications must use non-secret source labels because labels appear in diagnostics
|
||||
and source reports.
|
||||
Filesystem context follows the [source rules](#explicit-imports-and-source-identity):
|
||||
a file's own directory supplies its base, and an explicit context can set its
|
||||
root. Supplying `BaseDir` for a file is an options error; bytes/readers require
|
||||
|
||||
Reference in New Issue
Block a user