Record the test suite remediation results
This commit is contained in:
@@ -37,7 +37,8 @@ the current design from becoming an accidental suite-wide pattern.
|
|||||||
it at `/usr/local/bin/seriatim`.
|
it at `/usr/local/bin/seriatim`.
|
||||||
- Rebuild when the source checkout changes or the binary is absent.
|
- Rebuild when the source checkout changes or the binary is absent.
|
||||||
- Converge the installed binary to `root:root` ownership and mode `0755`.
|
- Converge the installed binary to `root:root` ownership and mode `0755`.
|
||||||
- Verify the installed version in repository-local integration tests.
|
- Verify an explicitly selected immutable Seriatim version through the native
|
||||||
|
CLI in repository-local integration tests.
|
||||||
- Keep the role interface, README, metadata, and focused local tests consistent
|
- Keep the role interface, README, metadata, and focused local tests consistent
|
||||||
with the simplified behavior.
|
with the simplified behavior.
|
||||||
- Remove obsolete build-state files and implementation machinery from the
|
- Remove obsolete build-state files and implementation machinery from the
|
||||||
@@ -213,7 +214,8 @@ used as templates for another role.
|
|||||||
- Detect arbitrary binary modification independently of reported version.
|
- Detect arbitrary binary modification independently of reported version.
|
||||||
- Protect or recover from unexpected symbolic links and malformed managed
|
- Protect or recover from unexpected symbolic links and malformed managed
|
||||||
paths.
|
paths.
|
||||||
- Add dedicated Ubuntu release and architecture test matrices.
|
- Add broader Ubuntu release and architecture test coverage only if future
|
||||||
|
platform behavior warrants it.
|
||||||
|
|
||||||
These are candidates for separately selected features if operational
|
These are candidates for separately selected features if operational
|
||||||
experience or a changed threat model justifies them.
|
experience or a changed threat model justifies them.
|
||||||
|
|||||||
@@ -75,8 +75,9 @@ rendered rather than mistaken for omission and are left for Audita to validate.
|
|||||||
|
|
||||||
Installation and configuration converge independently. A configuration-only
|
Installation and configuration converge independently. A configuration-only
|
||||||
change does not rebuild the binary, and an immediate repeat reports no changes.
|
change does not rebuild the binary, and an immediate repeat reports no changes.
|
||||||
Repository-local tests execute Audita to verify the installed version and
|
Repository-local tests validate managed configurations through Audita. The
|
||||||
validate both minimal and representative managed configurations.
|
current CLI has no native release-identity command, so the suite deliberately
|
||||||
|
does not substitute a raw source-commit proxy for version coverage.
|
||||||
|
|
||||||
## Repository Ownership
|
## Repository Ownership
|
||||||
|
|
||||||
|
|||||||
@@ -92,8 +92,8 @@ final value with ordinary Ansible/Jinja expressions.
|
|||||||
An immediate second application reports no changes. A changed Git checkout or
|
An immediate second application reports no changes. A changed Git checkout or
|
||||||
missing binary causes one build; configuration changes converge independently
|
missing binary causes one build; configuration changes converge independently
|
||||||
through the ordinary `ansible.builtin.template` operation. Repository-local
|
through the ordinary `ansible.builtin.template` operation. Repository-local
|
||||||
tests execute the installed CLI to verify the selected build and validate the
|
tests select immutable `v0.5.0` explicitly before checking its native CLI
|
||||||
managed configuration.
|
output, and validate the managed configuration.
|
||||||
|
|
||||||
## Repository Ownership
|
## Repository Ownership
|
||||||
|
|
||||||
@@ -151,10 +151,9 @@ managed configuration.
|
|||||||
`become: true`.
|
`become: true`.
|
||||||
- `community.general` supplies Snap management and is pinned to version
|
- `community.general` supplies Snap management and is pinned to version
|
||||||
`13.3.0` in the role's collection requirements.
|
`13.3.0` in the role's collection requirements.
|
||||||
- Tag `v0.5.0`, commit
|
- Tag `v0.5.0` remains buildable with the same Go requirement, package, linker
|
||||||
`61436d7c186af57b06004e090ceac898e83cdb71`, remains buildable with the same
|
symbol, and configuration schema and is the explicit immutable release used
|
||||||
Go requirement, package, linker symbol, and configuration schema and may be
|
for focused native version-selection coverage.
|
||||||
used for focused checkout-change coverage before restoring `v0.6.0`.
|
|
||||||
|
|
||||||
## Fixed Public Interface
|
## Fixed Public Interface
|
||||||
|
|
||||||
@@ -289,8 +288,8 @@ its configuration.
|
|||||||
validates successfully with `notarius config validate`.
|
validates successfully with `notarius config validate`.
|
||||||
- Explicit falsey non-null values, including zero and false, are rendered
|
- Explicit falsey non-null values, including zero and false, are rendered
|
||||||
rather than omitted; semantic validity remains Notarius's responsibility.
|
rather than omitted; semantic validity remains Notarius's responsibility.
|
||||||
- Repository-local tests receive `notarius v0.6.0` from the installed CLI and
|
- Repository-local tests explicitly select `v0.5.0`, receive `notarius v0.5.0`
|
||||||
successfully validate the managed configuration.
|
from the installed CLI, and successfully validate the managed configuration.
|
||||||
- An immediate second role run reports no changes.
|
- An immediate second role run reports no changes.
|
||||||
- A missing binary causes one rebuild followed by an unchanged run.
|
- A missing binary causes one rebuild followed by an unchanged run.
|
||||||
- A changed checkout causes one rebuild followed by an unchanged run.
|
- A changed checkout causes one rebuild followed by an unchanged run.
|
||||||
@@ -340,4 +339,5 @@ through `notarius_pipelines` and select the other field-specific values.
|
|||||||
- Validate configuration during production convergence.
|
- Validate configuration during production convergence.
|
||||||
- Manage a dedicated runtime account, secret environment, state paths,
|
- Manage a dedicated runtime account, secret environment, state paths,
|
||||||
service, or schedule if later operating requirements select them.
|
service, or schedule if later operating requirements select them.
|
||||||
- Add dedicated Ubuntu release and architecture test matrices.
|
- Add broader Ubuntu release and architecture test coverage only if future
|
||||||
|
platform behavior warrants it.
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
# Test Suite Audit Findings
|
# Test Suite Audit Findings
|
||||||
|
|
||||||
Status: complete; remediation pending review
|
Status: remediation implemented; pending review
|
||||||
|
|
||||||
## Executive Summary
|
## Executive Summary
|
||||||
|
|
||||||
@@ -256,5 +256,20 @@ explicit release selection where natively observable, and representative
|
|||||||
configuration behavior.
|
configuration behavior.
|
||||||
|
|
||||||
The audit made no changes to production roles or test implementations.
|
The audit made no changes to production roles or test implementations.
|
||||||
Remediation should begin only after these findings and the Audita
|
|
||||||
release-identity recommendation have been reviewed.
|
## Remediation Status
|
||||||
|
|
||||||
|
The recommended remediation was implemented and is pending review. No
|
||||||
|
production role behavior changed. The role commits are:
|
||||||
|
|
||||||
|
| Repository | Remediation commit | Final integration result |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `ansible-role-seriatim` | `e08c63c` | Pass |
|
||||||
|
| `ansible-role-audita` | `d2d0b9d` | Pass |
|
||||||
|
| `ansible-role-notarius` | `cfecf37` | Pass |
|
||||||
|
|
||||||
|
Final sequential validation on the approved Ubuntu 26.04 `x86_64` proxy ran
|
||||||
|
each repository's `tests/integration` entry point successfully. Each cleanup
|
||||||
|
path restored the default deployment, and the role repositories were clean
|
||||||
|
after their remediation commits. The existing classifications remain the
|
||||||
|
authoritative rationale for this implemented suite shape.
|
||||||
|
|||||||
@@ -4,7 +4,7 @@ Last updated: 2026-09-09
|
|||||||
|
|
||||||
Current milestone: MVP standalone application roles
|
Current milestone: MVP standalone application roles
|
||||||
|
|
||||||
Current focus: Review and commit the staged test-suite remediation plan.
|
Current focus: Review the implemented test-suite remediation.
|
||||||
|
|
||||||
| ID | Deliverable | Owner | State | Evidence or next step |
|
| ID | Deliverable | Owner | State | Evidence or next step |
|
||||||
| --- | --- | --- | --- | --- |
|
| --- | --- | --- | --- | --- |
|
||||||
@@ -21,7 +21,7 @@ Current focus: Review and commit the staged test-suite remediation plan.
|
|||||||
| F0002 | Install and configure Audita | ansible-role-audita | done | Field-specific configuration completed in `e61f02f` and `847c4c3`; full suite passed |
|
| F0002 | Install and configure Audita | ansible-role-audita | done | Field-specific configuration completed in `e61f02f` and `847c4c3`; full suite passed |
|
||||||
| F0003 | Install and configure Notarius | ansible-role-notarius | done | Implemented and validated through remediation commit `eb4f511` |
|
| F0003 | Install and configure Notarius | ansible-role-notarius | done | Implemented and validated through remediation commit `eb4f511` |
|
||||||
| A0001 | Audit completed standalone-role test suites | workspace and standalone roles | done | Findings recorded in `docs/roadmap/test-suite-audit-findings.md` |
|
| A0001 | Audit completed standalone-role test suites | workspace and standalone roles | done | Findings recorded in `docs/roadmap/test-suite-audit-findings.md` |
|
||||||
| R0001 | Simplify completed standalone-role test suites | workspace and standalone roles | ready | Execute `docs/roadmap/implementation.md` in four stages |
|
| R0001 | Simplify completed standalone-role test suites | workspace and standalone roles | in_progress | Pending review; role commits `e08c63c`, `d2d0b9d`, and `cfecf37` passed their full integration suites |
|
||||||
|
|
||||||
## Updating This File
|
## Updating This File
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user