Skip to content

livery.workshop.testing

livery.workshop.testing

What an extension's steps and categories are judged against: the conformance kit.

An extension declares steps and category tables that the gate then relies on. The kit states what the gate relies on, one clause at a time, and judges an extension against it: a livery.workshop.testing.Subject names the extension, the sets of extensions to probe it with and the steps it registers, each livery.workshop.testing.Clause returns the livery.workshop.testing.Violations the subject commits, and livery.workshop.testing.CLAUSES lists them in order. An extension's own suite runs every clause on its subject after the extension's registrations ran:

import pytest

from livery.workshop.testing import CLAUSES, Clause, Subject

SUBJECT = Subject("acme.extension", sets=(("acme",),), steps=(...))


@pytest.mark.parametrize("clause", CLAUSES, ids=lambda clause: clause.name)
def test_the_extension_conforms(clause: Clause) -> None:
    assert clause.judge(SUBJECT) == []

The workshop's own steps run the same clauses through livery.workshop.testing.builtin_subject, so the kit cannot drift from what the gate enforces.

Classes:

Name Description
Clause

One thing the gate relies on of what an extension registers.

Subject

What an extension registers, as the kit judges it.

Violation

One way a subject breaks a clause.

Functions:

Name Description
builtin_subject

The workshop's own steps as a subject, for its own suite.

judge

Every violation subject commits, clause by clause; empty when it conforms.

Attributes:

Name Type Description
CLAUSES tuple[Clause, ...]

Every clause, in the order the kit judges them.

CLAUSES module-attribute

CLAUSES: tuple[Clause, ...] = (Clause(FRAGMENT_OWNER, 'A per-package configuration file has one owner among the steps whose fragments name an extension.', _fragment_owner), Clause(CATEGORY_TABLE, "A path's category is the most specific rule's, the nearer table winning a tie between tables, the extension later in composition order first and the base's table last, and two rules of one table never tie.", _category_table), Clause(STEP_ORDER, 'Every step a step runs after is registered or declared by an installed extension, and following the steps it runs after never leads back to it.', _step_order), Clause(DECLARATION_VALIDATES, "An extension's extension.toml holds only the keys the workshop declares, each of its type, and every reference in it names a function its module defines.", _declaration_validates), Clause(REFERENCES_REGISTER_NOTHING, "Importing a module a declaration's reference names registers nothing: the declaration says what the extension adds, and the mount registers it.", _references_register_nothing), Clause(ENTRIES_NAME_DEFINED_TASKS, "Every task a CI job's entries name is one the extension defines, read from its sources: a group and a task decorated in it.", _entries_name_defined_tasks), Clause(CLAIMS_NAME_CATEGORIES, "Every category a step's claims name is one the workshop knows, and every suffix starts with a dot.", _claims_name_categories), Clause(WORDS_ARE_ANSWERABLE, "A step in words runs a tool it requires, every placeholder its words name is one the engine or its matrix answers, every matrix key is named in its words, and a package's placeholder sits in a step that judges one package at a time.", _words_are_answerable), Clause(PLUGIN_IS_AN_ENTRY_POINT, 'The plugin an extension.toml names is a footman.tasks entry point an installed distribution declares.', _plugin_is_an_entry_point), Clause(REQUIREMENTS_IN_METADATA, 'Every extension an extension.toml requires is a dependency of its wheel.', _requirements_in_metadata), Clause(DECLARATION_DOCUMENTED, "An extension's extension.toml carries an [extension] description, and every verb its plugin defines, every contract key, template, template option, step option and environment variable it declares carries its doc line.", _declaration_documented), Clause(TEMPLATE_COMPOSES, "Every extension a template lists composes something in a package, unless another it lists requires it, and the set of the template's list holds every extension its chain needs.", _template_composes), Clause(FRAGMENT_DRIFT, "A step's fragments render, the composed pyproject.toml parses, and a per-package file matches its render until a person edits it, when the drift gate names it.", _fragment_drift), Clause(WITHDRAWN_FILE, "A per-package file whose step is withdrawn, and which no other step renders for the package's extensions, is removed when unedited and kept when edited.", _withdrawn_file), Clause(WALK_ORDER, 'Under --fix every step of a phase the gate runs that has a fix mode rewrites before any judge starts and is not judged again, and every other step of such a phase is judged.', _walk_order), Clause(GATE_LINES, 'Every step of a phase the gate runs names the extension that registered it, by the name a workspace lists it under, which the gate prints, and a step with no file to read is named and never started.', _gate_lines))

Every clause, in the order the kit judges them.

__all__ module-attribute

__all__ = ['CLAUSES', 'Clause', 'Subject', 'Violation', 'builtin_subject', 'judge']

Clause dataclass

Clause(name: str, rule: str, judge: Callable[[Subject], list[Violation]])

One thing the gate relies on of what an extension registers.

Attributes:

Name Type Description
name str

The clause's name, fragment-owner.

rule str

The rule in one sentence.

judge Callable[[Subject], list[Violation]]

The violations a subject commits; empty when it conforms.

name instance-attribute

name: str

rule instance-attribute

rule: str

judge instance-attribute

Subject dataclass

Subject(extension: str, sets: tuple[tuple[str, ...], ...] = (), steps: tuple[StepRecord, ...] = ())

What an extension registers, as the kit judges it.

The records are the ones the extension registered, and the kit reads the registries as they stand, so the extension's registrations run before the kit does.

Attributes:

Name Type Description
extension str

The extension's package, acme.extension: where its extension.toml sits, which the kit reads as the mount does.

sets tuple[tuple[str, ...], ...]

The sets of extensions the kit probes, each as the list a package writes for it, ("acme",). The category-table clause judges a package listing each, and the gate's walk runs over a workspace holding one such package for each.

steps tuple[StepRecord, ...]

The steps the extension registers.

extension instance-attribute

extension: str

sets class-attribute instance-attribute

sets: tuple[tuple[str, ...], ...] = ()

steps class-attribute instance-attribute

steps: tuple[StepRecord, ...] = ()

Violation dataclass

Violation(clause: str, where: str, reason: str)

One way a subject breaks a clause.

Attributes:

Name Type Description
clause str

The clause's name.

where str

What the violation concerns: a set, a step, a file or a path.

reason str

What is wrong, and what the gate does because of it.

Methods:

Name Description
__str__

The violation as one line: clause, what it concerns, reason.

clause instance-attribute

clause: str

where instance-attribute

where: str

reason instance-attribute

reason: str

__str__

__str__() -> str

The violation as one line: clause, what it concerns, reason.

Source code in packages/workshop/src/livery/workshop/testing/_conformance.py
def __str__(self) -> str:
    """The violation as one line: clause, what it concerns, reason."""
    return f"{self.clause}: {self.where}: {self.reason}"

builtin_subject

builtin_subject() -> Subject

The workshop's own steps as a subject, for its own suite.

Its sets are the empty one, whose package the base's table answers for alone, and, for each installed extension that composes something in a package (livery.workshop._extensions.composes_per_package), the set of that extension alone.

Source code in packages/workshop/src/livery/workshop/testing/_conformance.py
def builtin_subject() -> Subject:
    """The workshop's own steps as a subject, for its own suite.

    Its sets are the empty one, whose package the base's table answers
    for alone, and, for each installed extension that composes
    something in a package
    ([livery.workshop._extensions.composes_per_package][]), the set of
    that extension alone.
    """
    from livery.workshop._extensions import composes_per_package, installed_extensions

    composing = [name for name in installed_extensions() if composes_per_package(name)]
    return Subject(
        BASE_EXTENSION,
        sets=((), *((name,) for name in composing)),
        steps=tuple(
            record
            for record in steps_by_name().values()
            if record.extension == BASE_EXTENSION
        ),
    )

judge

judge(subject: Subject) -> list[Violation]

Every violation subject commits, clause by clause; empty when it conforms.

Source code in packages/workshop/src/livery/workshop/testing/_conformance.py
def judge(subject: Subject) -> list[Violation]:
    """Every violation *subject* commits, clause by clause; empty when it conforms."""
    return [violation for clause in CLAUSES for violation in clause.judge(subject)]