48 lines
3.0 KiB
Markdown
48 lines
3.0 KiB
Markdown
# Testing House Style
|
|
|
|
This policy is normative for creating, changing, reviewing, and auditing tests
|
|
in this workspace.
|
|
|
|
- Use a classical state-based approach, also commonly described as Chicago or
|
|
Detroit style. Exercise the real role through its public interface and
|
|
observe the resulting deployed system.
|
|
- Test behavior owned by the role, not the current arrangement of tasks that
|
|
implements it. A useful test should normally remain valid after an internal
|
|
refactor that preserves the same public behavior.
|
|
- Require an immediate second role application to report no changes. Clean
|
|
repeated convergence is a first-class role outcome, not a test of any one
|
|
module.
|
|
- Distinguish configurable behavior from current defaults. To prove version
|
|
selection, explicitly request a version and verify that the application
|
|
reports that version. Do not assert that the same version remains the role's
|
|
default. For example, a useful test sets `audita_version: v0.9.0` and verifies
|
|
that the installed Audita reports `v0.9.0`; a harmful test asserts that
|
|
`v0.9.0` is the role's current default. The former continues to prove version
|
|
selection after the default advances to `v0.10.0`, while the latter fails
|
|
merely because of the intended update.
|
|
- Use a default invocation as a smoke test when useful, but do not snapshot
|
|
exact default values, selected commits, complete default configuration
|
|
documents, or counts of default variables. Those assertions are change
|
|
detectors that turn intentional updates into unrelated test failures.
|
|
- Do not assert task names, task order, individual `changed` or `skipped`
|
|
results, or other implementation traces. Inspect the final system and the
|
|
overall clean-repeat result instead.
|
|
- Do not test documented behavior of `ansible.builtin`, widely used
|
|
`community.general`, or other trusted modules. In particular, do not
|
|
independently verify file existence, ownership, permissions, package state,
|
|
checkout state, or template deployment merely to repeat module arguments.
|
|
- Test role-authored decisions only when they express meaningful public
|
|
behavior. Examples include an explicitly selected application version, a
|
|
caller-supplied build executable, configuration rendering with representative
|
|
non-null and falsey values, and composition of caller-supplied mappings.
|
|
- Verify application outcomes with the application's native interface when
|
|
practical. Prefer executing the installed application or its configuration
|
|
validator over reconstructing its rules in Ansible assertions.
|
|
- Self-evident declarative guards, including straightforward platform
|
|
assertions, do not require a negative test matrix.
|
|
- Prefer one representative case for a behavior over exhaustive combinations.
|
|
Every additional test must protect a requirement or a plausible regression,
|
|
not merely increase coverage.
|
|
- Keep syntax checking and linting as static validation. They complement the
|
|
behavioral suite but do not justify tests that freeze implementation details.
|