livery.workshop.testing¶
livery.workshop.testing ¶
What an extension's steps and categories are judged against: the conformance kit.
An extension declares steps and category tables that the gate then relies on. The kit states what the gate relies on, one clause at a time, and judges an extension against it: a livery.workshop.testing.Subject names the extension, the sets of extensions to probe it with and the steps it registers, each livery.workshop.testing.Clause returns the livery.workshop.testing.Violations the subject commits, and livery.workshop.testing.CLAUSES lists them in order. An extension's own suite runs every clause on its subject after the extension's registrations ran:
import pytest
from livery.workshop.testing import CLAUSES, Clause, Subject
SUBJECT = Subject("acme.extension", sets=(("acme",),), steps=(...))
@pytest.mark.parametrize("clause", CLAUSES, ids=lambda clause: clause.name)
def test_the_extension_conforms(clause: Clause) -> None:
assert clause.judge(SUBJECT) == []
The workshop's own steps run the same clauses through livery.workshop.testing.builtin_subject, so the kit cannot drift from what the gate enforces.
Classes:
| Name | Description |
|---|---|
Clause |
One thing the gate relies on of what an extension registers. |
Subject |
What an extension registers, as the kit judges it. |
Violation |
One way a subject breaks a clause. |
Functions:
| Name | Description |
|---|---|
builtin_subject |
The workshop's own steps as a subject, for its own suite. |
judge |
Every violation subject commits, clause by clause; empty when it conforms. |
Attributes:
| Name | Type | Description |
|---|---|---|
CLAUSES |
tuple[Clause, ...]
|
Every clause, in the order the kit judges them. |
CLAUSES
module-attribute
¶
CLAUSES: tuple[Clause, ...] = (Clause(FRAGMENT_OWNER, 'A per-package configuration file has one owner among the steps whose fragments name an extension.', _fragment_owner), Clause(CATEGORY_TABLE, "A path's category is the most specific rule's, the nearer table winning a tie between tables, the extension later in composition order first and the base's table last, and two rules of one table never tie.", _category_table), Clause(STEP_ORDER, 'Every step a step runs after is registered or declared by an installed extension, and following the steps it runs after never leads back to it.', _step_order), Clause(DECLARATION_VALIDATES, "An extension's extension.toml holds only the keys the workshop declares, each of its type, and every reference in it names a function its module defines.", _declaration_validates), Clause(REFERENCES_REGISTER_NOTHING, "Importing a module a declaration's reference names registers nothing: the declaration says what the extension adds, and the mount registers it.", _references_register_nothing), Clause(ENTRIES_NAME_DEFINED_TASKS, "Every task a CI job's entries name is one the extension defines, read from its sources: a group and a task decorated in it.", _entries_name_defined_tasks), Clause(CLAIMS_NAME_CATEGORIES, "Every category a step's claims name is one the workshop knows, and every suffix starts with a dot.", _claims_name_categories), Clause(WORDS_ARE_ANSWERABLE, "A step in words runs a tool it requires, every placeholder its words name is one the engine or its matrix answers, every matrix key is named in its words, and a package's placeholder sits in a step that judges one package at a time.", _words_are_answerable), Clause(PLUGIN_IS_AN_ENTRY_POINT, 'The plugin an extension.toml names is a footman.tasks entry point an installed distribution declares.', _plugin_is_an_entry_point), Clause(REQUIREMENTS_IN_METADATA, 'Every extension an extension.toml requires is a dependency of its wheel.', _requirements_in_metadata), Clause(DECLARATION_DOCUMENTED, "An extension's extension.toml carries an [extension] description, and every verb its plugin defines, every contract key, template, template option, step option and environment variable it declares carries its doc line.", _declaration_documented), Clause(TEMPLATE_COMPOSES, "Every extension a template lists composes something in a package, unless another it lists requires it, and the set of the template's list holds every extension its chain needs.", _template_composes), Clause(FRAGMENT_DRIFT, "A step's fragments render, the composed pyproject.toml parses, and a per-package file matches its render until a person edits it, when the drift gate names it.", _fragment_drift), Clause(WITHDRAWN_FILE, "A per-package file whose step is withdrawn, and which no other step renders for the package's extensions, is removed when unedited and kept when edited.", _withdrawn_file), Clause(WALK_ORDER, 'Under --fix every step of a phase the gate runs that has a fix mode rewrites before any judge starts and is not judged again, and every other step of such a phase is judged.', _walk_order), Clause(GATE_LINES, 'Every step of a phase the gate runs names the extension that registered it, by the name a workspace lists it under, which the gate prints, and a step with no file to read is named and never started.', _gate_lines))
Every clause, in the order the kit judges them.
__all__
module-attribute
¶
Clause
dataclass
¶
Subject
dataclass
¶
What an extension registers, as the kit judges it.
The records are the ones the extension registered, and the kit reads the registries as they stand, so the extension's registrations run before the kit does.
Attributes:
| Name | Type | Description |
|---|---|---|
extension |
str
|
The extension's package, |
sets |
tuple[tuple[str, ...], ...]
|
The sets of extensions the kit probes, each as the list a
package writes for it, |
steps |
tuple[StepRecord, ...]
|
The steps the extension registers. |
Violation
dataclass
¶
One way a subject breaks a clause.
Attributes:
| Name | Type | Description |
|---|---|---|
clause |
str
|
The clause's name. |
where |
str
|
What the violation concerns: a set, a step, a file or a path. |
reason |
str
|
What is wrong, and what the gate does because of it. |
Methods:
| Name | Description |
|---|---|
__str__ |
The violation as one line: clause, what it concerns, reason. |
builtin_subject ¶
builtin_subject() -> Subject
The workshop's own steps as a subject, for its own suite.
Its sets are the empty one, whose package the base's table answers for alone, and, for each installed extension that composes something in a package (livery.workshop._extensions.composes_per_package), the set of that extension alone.
Source code in packages/workshop/src/livery/workshop/testing/_conformance.py
judge ¶
Every violation subject commits, clause by clause; empty when it conforms.