Skip to content

livery.toolroom.bench._tasks

livery.toolroom.bench._tasks

Keep the tool records honest — fm tools ….

The bridge never goes stale, because it transcribes nothing. A tool's record can: it describes the tool at the versions read, and tools move. These tasks close that gap by reading the installed tools into the records and by failing a check when a tool and its record disagree. A stub is a rendering of a record, written by the index build and materialised into a workspace by fm toolroom.restub; nothing here writes one into a package.

fm toolroom.list                  what footman curates, and what's installed
fm toolroom.spec ruff             what one tool says about itself, right now
fm toolroom.read                  read the installed tools into their records
fm toolroom.audit                 which tools have moved past their record
fm toolroom.color                 how footman forces colour, per tool

A reading is a snapshot, not a contract: sync takes one, audit says which tools have released a newer version since. Being behind is news rather than a fault, so audit reports and exits zero unless you ask for --strict, and the readings are retaken by the refresh rather than the moment a tool ships. A snapshot only ever moves forward: a tool that isn't installed, is missing from a --prefix, or reads older than the record already holds is named and left alone — a check that quietly covered three of thirteen would be worse than no check.

Attributes:

Name Type Description
tasks Group

tasks module-attribute

tasks: Group = Group('toolroom', help='Keep the tool records honest')

OBSERVATION_SCHEMA module-attribute

OBSERVATION_SCHEMA = 1

The observation document's shape. Bumped when a reader must know.

index_tasks module-attribute

index_tasks = tasks.group('index', help='The published index of the tool records')

STUBS_MODULE module-attribute

STUBS_MODULE = 'toolroom_stubs'

The module the docs pages' stubs are rendered as, for the docs build's renderer.

__all__ module-attribute

__all__ = ['tasks']

HostCheck dataclass

HostCheck(tool: str, version: str, outcome: str, detail: str)

One tool's executable check on this host.

Attributes:

Name Type Description
tool str

The record's name.

version str

The version checked, the newest with a build for the host.

outcome str

ok, skipped for a tool with no build for this host, or failed.

detail str

What happened, in the words a reviewer acts on.

Methods:

Name Description
__str__

tool instance-attribute

tool: str

version instance-attribute

version: str

outcome instance-attribute

outcome: str

detail instance-attribute

detail: str

__str__

__str__() -> str
Source code in packages/toolroom-bench/src/livery/toolroom/bench/_tasks.py
def __str__(self) -> str:
    at = f" {self.version}" if self.version else ""
    return f"  {self.tool}{at}: {self.outcome}, {self.detail}"

Gathered dataclass

Gathered(platform: str, observations: dict[str, dict[str, dict[str, Any]]], holes: dict[str, list[str]] = dict(), unreachable: dict[str, str] = dict(), skipped: list[str] = list())

What one platform saw, as data — the document a matrix leg hands on.

Deliberately portable rather than a CI internal: written by fm toolroom.gather --out=…, copied off a Windows box by hand if that is how the week goes, and folded by fm toolroom.assemble wherever the store lives. Self-describing, because the machine that reads it is not the machine that wrote it.

Methods:

Name Description
document

Attributes:

Name Type Description
platform str

Who looked — the one fact every observation in this document shares.

observations dict[str, dict[str, dict[str, Any]]]

tool -> version -> {date, tag, surface}: what this platform found.

holes dict[str, list[str]]

Releases this platform meant to read and could not. Carried rather

unreachable dict[str, str]

Indexes that would not answer here. Carried so the assembler can tell

skipped list[str]

Tools with no index, or no history to add to, on this platform.

platform instance-attribute

platform: str

Who looked — the one fact every observation in this document shares.

observations instance-attribute

observations: dict[str, dict[str, dict[str, Any]]]

tool -> version -> {date, tag, surface}: what this platform found.

holes class-attribute instance-attribute

holes: dict[str, list[str]] = dataclasses.field(default_factory=dict)

Releases this platform meant to read and could not. Carried rather than dropped: the assembler reports them, a later run fills them.

unreachable class-attribute instance-attribute

unreachable: dict[str, str] = dataclasses.field(default_factory=dict)

Indexes that would not answer here. Carried so the assembler can tell "this leg saw nothing new" from "this leg could not look".

skipped class-attribute instance-attribute

skipped: list[str] = dataclasses.field(default_factory=list)

Tools with no index, or no history to add to, on this platform.

document

document() -> dict[str, Any]
Source code in packages/toolroom-bench/src/livery/toolroom/bench/_tasks.py
def document(self) -> dict[str, Any]:
    return {"schema": OBSERVATION_SCHEMA, **dataclasses.asdict(self)}

Owed dataclass

Owed(releases: dict[str, list[str]], unreachable: dict[str, str], skipped: list[str])

What a gather would read, without reading any of it.

Attributes:

Name Type Description
releases dict[str, list[str]]

Versions this platform owes a reading of, per tool.

unreachable dict[str, str]

Indexes that would not answer. Not the same as nothing to do — a

skipped list[str]

Tools with no index to read, or no record to add to.

total int

releases instance-attribute

releases: dict[str, list[str]]

Versions this platform owes a reading of, per tool.

unreachable instance-attribute

unreachable: dict[str, str]

Indexes that would not answer. Not the same as nothing to do — a walk that cannot see an index cannot say the index has nothing new.

skipped instance-attribute

skipped: list[str]

Tools with no index to read, or no record to add to.

total property

total: int

Refreshed dataclass

Refreshed(read: dict[str, list[str]], events: dict[str, list[str]], unreachable: dict[str, str], skipped: list[str], holes: dict[str, list[str]] = dict(), wrote_changelog: bool = False, release: bool = False, additions_only: bool = False, ingest: dict[str, list[str]] = dict(), ingest_ok: bool = True, artifacts: dict[str, list[str]] = dict())

What a refresh found — the release decision, as data.

Returned rather than printed, so fm --json toolroom.refresh hands a scheduled job the same answer a person reads.

Methods:

Name Description
__post_init__

Attributes:

Name Type Description
read dict[str, list[str]]

Releases newly observed, per tool, oldest first.

events dict[str, list[str]]

The subset of those that are newer than anything seen before and

unreachable dict[str, str]

Indexes that would not answer, and why. Not the same as a tool with

skipped list[str]

Tools with no index to read, or no record to add to.

holes dict[str, list[str]]

Releases that were listed but could not be observed — an install that

wrote_changelog bool

Whether the events reached CHANGELOG.md. False with nothing to say,

release bool

Whether any tool's surface moved — decision 4, in one line. A field

additions_only bool

Whether every surface change across every announced event only ADDED

ingest dict[str, list[str]]

Per announced tool, the ingest verification's lines: the verdict,

ingest_ok bool

Whether every ingest check passed on every host of every event

artifacts dict[str, list[str]]

Per tool read from a forge tier, the artifact step's lines: the

armed bool

Whether the pull request arms: additions only and every check passed.

read instance-attribute

read: dict[str, list[str]]

Releases newly observed, per tool, oldest first.

events instance-attribute

events: dict[str, list[str]]

The subset of those that are newer than anything seen before and changed the tool's surface. This is the release decision and the CHANGELOG line at once — so a backfill, which reaches only downwards, is saved and reported but never announced.

unreachable instance-attribute

unreachable: dict[str, str]

Indexes that would not answer, and why. Not the same as a tool with nothing new — see _toolfetch.Unreachable.

skipped instance-attribute

skipped: list[str]

Tools with no index to read, or no record to add to.

holes class-attribute instance-attribute

holes: dict[str, list[str]] = field(default_factory=dict)

Releases that were listed but could not be observed — an install that failed, a binary that would not describe itself. A hole is not an error: the record stays whole by construction, a later run fills it, and until then a change the missing release carried reads as arriving at the next release actually read.

wrote_changelog class-attribute instance-attribute

wrote_changelog: bool = False

Whether the events reached CHANGELOG.md. False with nothing to say, and false when the file has no [Unreleased] section to write into — which a caller should notice rather than assume the notes got written.

release class-attribute instance-attribute

release: bool = False

Whether any tool's surface moved — decision 4, in one line. A field rather than a property, and recomputed from events on construction: dataclasses.asdict serialises fields only, and this is the one value the scheduled job reads out of fm --json toolroom.refresh.

additions_only class-attribute instance-attribute

additions_only: bool = False

Whether every surface change across every announced event only ADDED to a tool's surface — nothing dropped, no verb withdrawn. Half of the graded release trigger's green light: additions cannot break a caller. False when there are no events at all (nothing to ship is not "safe to ship"). Rewordings count as additions-safe: they change what a stub says, never what a tool accepts.

ingest class-attribute instance-attribute

ingest: dict[str, list[str]] = field(default_factory=dict)

Per announced tool, the ingest verification's lines: the verdict, each finding and each host's structural diff, for every event version that has a host (livery.toolroom.bench._ingest). Empty for a tool whose versions have no host, since there is nothing to stage.

ingest_ok class-attribute instance-attribute

ingest_ok: bool = True

Whether every ingest check passed on every host of every event version: the other half of the green light. True when nothing had a host to check, so a tool read but never downloaded arms as before.

artifacts class-attribute instance-attribute

artifacts: dict[str, list[str]] = field(default_factory=dict)

Per tool read from a forge tier, the artifact step's lines: the hosts each fresh version gained, the assets they came from, and any version whose artifacts could not be recorded.

armed property

armed: bool

Whether the pull request arms: additions only and every check passed.

__post_init__

__post_init__() -> None
Source code in packages/toolroom-bench/src/livery/toolroom/bench/_tasks.py
def __post_init__(self) -> None:
    object.__setattr__(self, "release", any(self.events.values()))

default_prefix

default_prefix() -> Path

Where provisioned tools live when no --prefix names another place.

A toolroom-bench room in footman's data directory: durable, never touched by the cache collector, and moved wholesale by FOOTMAN_DATA_DIR (or XDG_DATA_HOME). One machine-level home, instead of a .tools-latest in whichever directory the command happened to run from, and beside the store's own toolroom home, never inside it: the store's bin directory is not a provisioned prefix.

Source code in packages/toolroom-bench/src/livery/toolroom/bench/_tasks.py
def default_prefix() -> Path:
    """Where provisioned tools live when no `--prefix` names another place.

    A `toolroom-bench` room in footman's data directory: durable, never
    touched by the cache collector, and moved wholesale by
    `FOOTMAN_DATA_DIR` (or `XDG_DATA_HOME`). One machine-level home,
    instead of a `.tools-latest` in whichever directory the command
    happened to run from, and beside the store's own `toolroom` home,
    never inside it: the store's bin directory is not a provisioned
    prefix.
    """
    return data_dir() / "toolroom-bench"

list_

list_(show: Annotated[Literal['all', 'installed', 'missing'], doc('which tools to list (default: all, present or not)')] = 'all') -> None

The curated tools: version, in-process capability, stub state.

Every curated tool is listed by default, absent ones included — the version column says not installed. --show installed narrows to what this machine can actually run, --show missing to what it can't.

A present tool whose version will not read is a different fact from an absent one, and the column says which: unreadable (timed out after 30s), never a false not installed — the CI contradiction that conflation produced (a row both passing the installed filter and claiming absence) is exactly what the diagnosis channel exists to prevent.

Source code in packages/toolroom-bench/src/livery/toolroom/bench/_tasks.py
@tasks.task(name="list")
def list_(
    show: Annotated[
        Literal["all", "installed", "missing"],
        doc("which tools to list (default: all, present or not)"),
    ] = "all",
) -> None:
    """The curated tools: version, in-process capability, stub state.

    Every curated tool is listed by default, absent ones included — the
    version column says `not installed`. `--show installed` narrows to what
    this machine can actually run, `--show missing` to what it can't.

    A *present* tool whose version will not read is a different fact from an
    absent one, and the column says which: `unreadable (timed out after
    30s)`, never a false `not installed` — the CI contradiction that
    conflation produced (a row both passing the installed filter and
    claiming absence) is exactly what the diagnosis channel exists to
    prevent.
    """
    on = wants_color(sys.stdout)
    rows: list[tuple[str, str, str, str]] = []
    shown = [
        (driver, here)
        for driver in _drivers.DRIVERS
        for here in (_drivers.installed(driver),)
        if not (show == "missing" and here) and not (show == "installed" and not here)
    ]
    # Every version is one spawn, read side by side so the table waits
    # for the slowest tool and not for the sum of them; a stalled read
    # is tried again alone, with a longer budget, before it is a fact.
    present = [driver.name for driver, here in shown if here]
    versions = _drivers.read_versions(present)
    for driver, here in shown:
        version = why = ""
        if here:
            version, why = versions[driver.name]
        capable = _drivers.in_process_capable(driver.name) if here else False
        mode = "in-process" if driver.in_process else ("capable" if capable else "—")
        record = "yes" if _record_path(driver.key).exists() else "no"
        blank = f"unreadable ({why})" if here else "not installed"
        rows.append((driver.key, version or blank, mode, record))
    width = max((len(r[0]) for r in rows), default=4)
    print(
        styled(f"{'tool'.ljust(width)}  version      in-process  record", "bold", on=on)
    )
    for key, version, mode, record in rows:
        print(f"{key.ljust(width)}  {version:<12} {mode:<11} {record}")

spec

spec(name: Annotated[str, doc('a curated tool: ruff, uv, mkdocs, …')], verb: Annotated[str, doc('one verb, dotted for nesting (compose.up)')] = '') -> None

Print what a tool says about itself, as footman reads it.

Source code in packages/toolroom-bench/src/livery/toolroom/bench/_tasks.py
@tasks.task
def spec(
    name: Annotated[str, doc("a curated tool: ruff, uv, mkdocs, …")],
    verb: Annotated[str, doc("one verb, dotted for nesting (compose.up)")] = "",
) -> None:
    """Print what a tool says about itself, as footman reads it."""
    driver = _drivers.find(name)
    if driver is None:
        raise SystemExit(f"no driver for {name!r}; try `{_prog()} toolroom.list`")
    if not _drivers.installed(driver):
        raise SystemExit(f"{driver.name} is not installed")
    on = wants_color(sys.stdout)
    extracted = _extract(driver)
    print(
        styled(f"{extracted.name} {extracted.version}", "bold", on=on), extracted.help
    )
    for one in extracted.verbs:
        if verb and one.name != verb:
            continue
        label = one.name or "(the tool itself)"
        print(styled(f"\n  {label}", "cyan", on=on), f"— {len(one.options)} options")
        for option in one.options:
            negation = f"  off → {option.negation}" if option.negation else ""
            print(f"    {option.name:<28} {option.type_name:<10}{negation}")

read

read(only: Annotated[str, doc('regenerate just this tool')] = '', prefix: Annotated[str, doc("read binaries from this prefix's bin/")] = '') -> None

Read the tools installed on this machine into their records.

A reading is a snapshot: what one tool accepted at one version, on one machine. Point --prefix at a fm toolroom.provision directory to take that snapshot from the isolated latest set instead of whatever this machine has — a dev environment's pytest carries its plugins' flags too, and those do not belong in a record whose driver never asked for them.

A tool that isn't installed keeps the record that is checked in — there is nothing to read it from, and a record that exists beats one that was emptied because a laptop happened to be missing a binary.

Source code in packages/toolroom-bench/src/livery/toolroom/bench/_tasks.py
@tasks.task
def read(
    only: Annotated[str, doc("regenerate just this tool")] = "",
    prefix: Annotated[str, doc("read binaries from this prefix's bin/")] = "",
) -> None:
    """Read the tools installed on this machine into their records.

    A reading is a *snapshot*: what one tool accepted at one version, on
    one machine. Point `--prefix` at a `fm toolroom.provision` directory to
    take that snapshot from the isolated latest set instead of whatever
    this machine has — a dev environment's pytest carries its plugins'
    flags too, and those do not belong in a record whose driver never
    asked for them.

    A tool that isn't installed keeps the record that is checked in —
    there is nothing to read it from, and a record that exists beats one
    that was emptied because a laptop happened to be missing a binary.
    """
    with _on_path(prefix):
        _sync(only, _prefix_root(prefix))

audit

audit(only: Annotated[str, doc('check just this tool')] = '', fix: Annotated[bool, doc('take a fresh snapshot instead of reporting')] = False, prefix: Annotated[str, doc("read binaries from this prefix's bin/")] = '', strict: Annotated[bool, doc('exit non-zero when a snapshot is behind')] = False) -> dict[str, object]

Report which tools have moved on since their record's newest reading.

A record holds what one tool accepted at the versions it was read from. Tools keep releasing, and footman promises no particular speed at following them — so a tool showing up here means a newer version exists, not that anything is wrong. Every stubbed verb ends in **flags: Any, so the bridge already speaks a flag the record has never heard of; only the hint is behind. --fix takes a fresh reading, --strict gives automation something to trip on, and --prefix asks the question against a provisioned latest set rather than this machine.

A snapshot only ever moves forward, so two readings are worth less than the record already checked in and are named and left alone: a tool missing from --prefix (a partial provision must not read as drift, and the host's copy is not the answer), and one whose version is older than the record holds (a machine behind the one that took the snapshot has nothing to add). Neither counts as behind — they are unanswered.

One finding here is a fault, and always exits non-zero: footman's negation and wrapper tables are read by the runtime, so a disagreement there means a task emits the wrong command today.

Source code in packages/toolroom-bench/src/livery/toolroom/bench/_tasks.py
@tasks.task
def audit(
    only: Annotated[str, doc("check just this tool")] = "",
    fix: Annotated[bool, doc("take a fresh snapshot instead of reporting")] = False,
    prefix: Annotated[str, doc("read binaries from this prefix's bin/")] = "",
    strict: Annotated[bool, doc("exit non-zero when a snapshot is behind")] = False,
) -> dict[str, object]:
    """Report which tools have moved on since their record's newest reading.

    A record holds what one tool accepted at the versions it was read
    from. Tools keep releasing, and footman promises no particular speed
    at following them — so a tool showing up here means a newer version
    exists, **not** that anything is wrong. Every stubbed verb ends in
    `**flags: Any`, so the bridge already speaks a flag the record has
    never heard of; only the *hint* is behind. `--fix` takes a fresh
    reading, `--strict` gives automation something to trip on, and
    `--prefix` asks the question against a provisioned latest set rather
    than this machine.

    A snapshot only ever moves **forward**, so two readings are worth less
    than the record already checked in and are named and left alone: a
    tool missing from `--prefix` (a partial provision must not read as
    drift, and the host's copy is not the answer), and one whose version
    is older than the record holds (a machine behind the one that took
    the snapshot has nothing to add). Neither counts as behind — they are
    unanswered.

    One finding here *is* a fault, and always exits non-zero: footman's
    negation and wrapper tables are read by the runtime, so a disagreement
    there means a task emits the wrong command today.
    """
    with _on_path(prefix):
        return _audit(only, fix, strict, _prefix_root(prefix))

color

color(only: Annotated[str, doc('probe just this tool')] = '', write: Annotated[bool, doc("regenerate toolroom's _colordata.py")] = True, prefix: Annotated[str, doc("probe binaries from this prefix's bin/")] = '') -> None

Probe how footman forces colour for each installed tool, and regenerate the colour data.

footman spawns over pipes (no PTY), so it forces colour into the tools it spawns — by the environment (FORCE_COLOR/NO_COLOR) for the modern set, by the tool's own switch for the few that ignore it. Which is which is probed, not assumed: each tool is run with colour forced on and off, and the bytes read, so a direction is recorded env, flag (like git's -c color.ui=always), none, or unprobed (no trigger figured out).

Writes toolroom's _colordata.py, which the bridge reads for its forcing table and the docs read for the support table — and only ever from a --prefix, a fm toolroom.provision directory. A verdict is a claim about a release, so it has to come from a release someone fetched on purpose: without a prefix this machine's own binaries answer, at whatever versions it happens to carry and missing whatever it never installed. That reads fine as a table on a terminal, which is why it still prints one; it is not something to check in.

Source code in packages/toolroom-bench/src/livery/toolroom/bench/_tasks.py
@tasks.task
def color(
    only: Annotated[str, doc("probe just this tool")] = "",
    write: Annotated[bool, doc("regenerate toolroom's _colordata.py")] = True,
    prefix: Annotated[str, doc("probe binaries from this prefix's bin/")] = "",
) -> None:
    """Probe how footman forces colour for each installed tool, and regenerate
    the colour data.

    footman spawns over pipes (no PTY), so it forces colour into the tools it
    spawns — by the environment (`FORCE_COLOR`/`NO_COLOR`) for the modern set, by
    the tool's own switch for the few that ignore it. Which is which is *probed*,
    not assumed: each tool is run with colour forced on and off, and the bytes
    read, so a direction is recorded `env`, `flag` (like git's
    `-c color.ui=always`), `none`, or `unprobed` (no trigger figured out).

    Writes toolroom's `_colordata.py`, which the bridge reads for its forcing
    table and the docs read for the support table — and only ever from a
    `--prefix`, a `fm toolroom.provision` directory. A verdict is a claim about a
    *release*, so it has to come from a release someone fetched on purpose:
    without a prefix this machine's own binaries answer, at whatever versions
    it happens to carry and missing whatever it never installed. That reads
    fine as a table on a terminal, which is why it still prints one; it is not
    something to check in.
    """
    root = _prefix_root(prefix)
    with _on_path(prefix):
        _color_probe_and_write(only, write, wants_color(sys.stdout), root)

verify_host

verify_host(*, store: _Installer | None = None, only: str = '', run: Callable[[list[str]], tuple[int, str]] | None = None, extract: Callable[[Driver], ToolSpec] | None = None) -> list[HostCheck]

The executable checks on this host: each downloaded tool installs, runs, reads.

For each curated tool whose record is a downloaded kind, the newest version with a build for this host is supplied through store, its first entry point is run with the driver's help flag, and its surface is extracted from the installed tree and must describe the tool. A download with no entry point is not a program, and the presence of the files its env names is the whole check. A tool with no build for this host is skipped and says so. store, run and extract are the seams the tests drive; the defaults are the bench's store, a subprocess with a two-minute limit, and the bench's own extractor.

Source code in packages/toolroom-bench/src/livery/toolroom/bench/_tasks.py
def verify_host(
    *,
    store: _Installer | None = None,
    only: str = "",
    run: Callable[[list[str]], tuple[int, str]] | None = None,
    extract: Callable[[_drivers.Driver], ToolSpec] | None = None,
) -> list[HostCheck]:
    """The executable checks on this host: each downloaded tool installs, runs, reads.

    For each curated tool whose record is a downloaded kind, the newest
    version with a build for this host is supplied through *store*,
    its first entry point is run with the driver's help flag, and its
    surface is extracted from the installed tree and must describe the
    tool. A download with no entry point is not a program, and the
    presence of the files its env names is the whole check. A tool with
    no build for this host is skipped and says so.
    *store*, *run* and *extract* are the seams the tests drive; the
    defaults are the bench's store, a subprocess with a two-minute
    limit, and the bench's own extractor.
    """
    import subprocess

    from livery.toolroom.store import DOWNLOAD_KINDS, StoreError

    engine = store or _bench_store()
    checks: list[HostCheck] = []
    for driver in _drivers.DRIVERS:
        if only and driver.key != only:
            continue
        record = _surfaces.load(_record_path(driver.key))
        if record is None or record.kind not in DOWNLOAD_KINDS:
            continue
        version = next(
            (
                delta.version
                for delta in reversed(record.deltas)
                if engine.host in delta.hosts
            ),
            "",
        )
        if not version:
            checks.append(
                HostCheck(driver.key, "", "skipped", f"no build for {engine.host}")
            )
            continue
        try:
            ensured = engine.ensure(record, version)
        except StoreError as error:
            checks.append(HostCheck(driver.key, version, "failed", f"install: {error}"))
            continue
        if not ensured.deployment.entry_points:
            # A download with nothing on PATH (cmake-conan's provider)
            # is reached through its env: the files it names present in
            # the installed tree is the whole check, nothing runs.
            named = [Path(value) for value in ensured.env.values()]
            missing = [path.name for path in named if not path.exists()]
            checks.append(
                HostCheck(
                    driver.key,
                    version,
                    "failed" if missing else "ok",
                    f"{', '.join(missing)} not in the installed tree"
                    if missing
                    else f"{len(named)} file(s) present, a file and not a program",
                )
            )
            continue
        entry = ensured.tool_dir / ensured.deployment.entry_points[0]
        try:
            code, tail = (run or _run_entry)([str(entry), driver.help_flag])
        except OSError as error:
            checks.append(
                HostCheck(
                    driver.key,
                    version,
                    "failed",
                    f"{entry.name} did not start: {error}",
                )
            )
            continue
        except subprocess.TimeoutExpired:
            checks.append(
                HostCheck(
                    driver.key,
                    version,
                    "failed",
                    f"{entry.name} {driver.help_flag} did not return in time",
                )
            )
            continue
        if code != 0:
            checks.append(
                HostCheck(
                    driver.key,
                    version,
                    "failed",
                    f"{entry.name} {driver.help_flag} exited {code}: {tail}",
                )
            )
            continue
        with _bin_on_path(entry.parent):
            spec = (extract or _extract)(driver)
        if not _describes_itself(spec):
            checks.append(
                HostCheck(
                    driver.key,
                    version,
                    "failed",
                    "the surface does not describe the tool",
                )
            )
            continue
        options = sum(len(verb.options) for verb in spec.verbs)
        checks.append(
            HostCheck(
                driver.key,
                version,
                "ok",
                f"{entry.name} runs, {options} option(s) read",
            )
        )
    return checks

tools_verify_host

tools_verify_host(only: Annotated[str, doc('check just this tool')] = '') -> None

Install, run and read every downloaded tool on this host; red when one fails.

The six-host verification point's task: for each curated tool of a downloaded kind, the newest version with a build for this host installs through the bench's store, its entry point runs with the driver's help flag, and its surface extracts to a reading that describes the tool. A tool with no build for this host is skipped and says so.

Source code in packages/toolroom-bench/src/livery/toolroom/bench/_tasks.py
@tasks.task(name="verify-host")
def tools_verify_host(
    only: Annotated[str, doc("check just this tool")] = "",
) -> None:
    """Install, run and read every downloaded tool on this host; red when one fails.

    The six-host verification point's task: for each curated tool of a
    downloaded kind, the newest version with a build for this host
    installs through the bench's store, its entry point runs with the
    driver's help flag, and its surface extracts to a reading that
    describes the tool. A tool with no build for this host is skipped
    and says so.
    """
    checks = verify_host(only=only)
    for check in checks:
        print(check)
    failed = [check for check in checks if check.outcome == "failed"]
    counted = f"{len(checks) - len(failed)} of {len(checks)} tool(s) pass"
    if failed:
        fail(
            f"{counted}: {', '.join(check.tool for check in failed)} failed on this"
            " host"
        )
    print(f"  {counted}")

colour_page

colour_page() -> str

The colour support page, rendered from the checked-in colour data.

Reads toolroom's probed table (livery.toolroom.tools.colour_controls), not a live probe, so a docs build needs nothing on PATH and the page says exactly what ships — the same rule the per-tool pages follow. fm toolroom.color refreshes the data; the page follows on the next build.

Source code in packages/toolroom-bench/src/livery/toolroom/bench/_tasks.py
def colour_page() -> str:
    """The colour support page, rendered from the checked-in colour data.

    Reads toolroom's probed table ([livery.toolroom.tools.colour_controls][]),
    not a live probe, so a docs build needs nothing on PATH and the page
    says exactly what ships — the same rule the per-tool pages follow.
    `fm toolroom.color` refreshes the data; the page follows on the next
    build.
    """
    controls = _tools.colour_controls()
    lines = [
        "| Tool | Colour on | Colour off |",
        "| ---- | --------- | ---------- |",
    ]
    for key in sorted(controls):
        _argv0, on, off, flag_on, flag_off, _pre = controls[key]
        if on == "n/a" and off == "n/a":
            lines.append(f"| `{key}` | *(pass-through wrapper)* | |")
            continue
        on_cell = f"`{' '.join(flag_on)}`" if on == "flag" and flag_on else _ON_WORD[on]
        off_cell = (
            f"`{' '.join(flag_off)}`" if off == "flag" and flag_off else _OFF_WORD[off]
        )
        lines.append(f"| `{key}` | {on_cell} | {off_cell} |")
    return _COLOUR_PAGE.format(table="\n".join(lines))

prime

prime(only: Annotated[str, doc('prime just this tool')] = '', count: Annotated[int, doc('how many releases back to read')] = 20, keep: Annotated[bool, doc('leave the throwaway environments behind')] = False, prefix: Annotated[str, doc("drive the tiers from this prefix's bin/")] = '') -> None

Read past releases into the option history, deepening each chain.

Reaches below each tool's floor, up to --count releases further back. Nothing already written is touched, and a release the chain already has is skipped — so a prime interrupted by a rate limit is resumed by running it again.

The releases are gathered in parallel, a bounded wave at a time — installing a release and reading its --help depends on no other release, and the chain assembles whatever order the observations arrive in. A release that will not install, or whose binary will not describe itself, is a hole: named in the report, filled by a later run, and never the end of the tool's walk.

--prefix points at a fm toolroom.provision directory, and the tiers are driven from its binaries. That is not the same nicety it is on sync: uv carries CPython's download index inside itself, so a stale uv reports a stale newest python and the walk silently starts too low.

Source code in packages/toolroom-bench/src/livery/toolroom/bench/_tasks.py
@tasks.task
def prime(
    only: Annotated[str, doc("prime just this tool")] = "",
    count: Annotated[int, doc("how many releases back to read")] = 20,
    keep: Annotated[bool, doc("leave the throwaway environments behind")] = False,
    prefix: Annotated[str, doc("drive the tiers from this prefix's bin/")] = "",
) -> None:
    """Read past releases into the option history, deepening each chain.

    Reaches below each tool's floor, up to `--count` releases further back.
    Nothing already written is touched, and a release the chain already has
    is skipped — so a prime interrupted by a rate limit is resumed by
    running it again.

    The releases are gathered **in parallel**, a bounded wave at a time —
    installing a release and reading its `--help` depends on no other
    release, and the chain assembles whatever order the observations arrive
    in. A release that will not install, or whose binary will not describe
    itself, is a **hole**: named in the report, filled by a later run, and
    never the end of the tool's walk.

    `--prefix` points at a `fm toolroom.provision` directory, and the tiers are
    driven from *its* binaries. That is not the same nicety it is on `sync`:
    uv carries CPython's download index inside itself, so a stale uv reports
    a stale newest python and the walk silently starts too low.
    """
    import shutil
    import tempfile

    from livery.toolroom.bench import _toolfetch

    _bounce_bare_call("prime")
    scratch = Path(tempfile.mkdtemp(prefix="footman-prime-"))
    lines: list[str] = []
    try:
        with _on_path(prefix), _sandboxed(scratch):
            drivers, skipped = _curated(only, _toolfetch)
            listings, unreachable = _list_phase(drivers, _toolfetch)
            skipped += [f"{key} ({why})" for key, why in sorted(unreachable.items())]

            plans: dict[str, list[_toolfetch.Release]] = {}
            records: dict[str, Record] = {}
            for driver in drivers:
                if driver.key not in listings:
                    continue
                record = _surfaces.load(_record_path(driver.key))
                if record is None or not _surfaces.versions(record):
                    skipped.append(f"{driver.key} (no record — run `sync` first)")
                    continue
                planned, refused = _plan_prime(record, listings[driver.key], count)
                if refused:
                    lines.append(
                        f"{driver.key} +0 (from {_surfaces.floor(record)}) — {refused}"
                    )
                    continue
                records[driver.key] = record
                plans[driver.key] = planned

            work = [(d, r) for d in drivers if d.key in plans for r in plans[d.key]]
            surfaces = _gather(work, scratch)
            for driver in drivers:
                if driver.key not in plans:
                    continue
                record, fresh, holes = _assemble(
                    driver, records[driver.key], plans[driver.key], surfaces
                )
                note = f" — holes: {', '.join(holes)}" if holes else ""
                lines.append(
                    f"{driver.key} +{len(fresh)} (from {_surfaces.floor(record)}){note}"
                )
    finally:
        if not keep:
            shutil.rmtree(scratch, ignore_errors=True)
    for line in lines:
        print(line)
    if skipped:
        print(f"skipped: {', '.join(skipped)}")

gather

gather(only: Annotated[str, doc('gather just this tool')] = '', prefix: Annotated[str, doc("drive the tiers from this prefix's bin/")] = '', out: Annotated[str, doc('write the observation document here')] = '', count: Annotated[int, doc('also reach this many releases below the floor')] = 0) -> Gathered

Observe every release this platform has not yet accounted for.

Half of a refresh, and the half that must happen on the platform: a Linux box cannot tell you what a tool's --help says on Windows. It writes no store — only a document saying what this machine saw — so the three platforms of a matrix can run at once and nothing races for the files.

What it observes: every release newer than each tool's base, every hole the chain reports, and the base itself when this platform has not looked at it yet — that last one is what makes a new platform's coverage converge on the versions people are actually running, in one pass. --count also reaches below the floor, for deepening a platform's history the way prime does.

--out writes the document for another machine to fold; without it the document is returned (and printed under --json), which is what refresh uses when both halves run in one process.

Source code in packages/toolroom-bench/src/livery/toolroom/bench/_tasks.py
@tasks.task
def gather(
    only: Annotated[str, doc("gather just this tool")] = "",
    prefix: Annotated[str, doc("drive the tiers from this prefix's bin/")] = "",
    out: Annotated[str, doc("write the observation document here")] = "",
    count: Annotated[int, doc("also reach this many releases below the floor")] = 0,
) -> Gathered:
    """Observe every release this platform has not yet accounted for.

    Half of a refresh, and the half that must happen *on* the platform: a
    Linux box cannot tell you what a tool's `--help` says on Windows. It
    writes no store — only a document saying what this machine saw — so the
    three platforms of a matrix can run at once and nothing races for the
    files.

    What it observes: every release newer than each tool's base, every hole
    the chain reports, and the base itself when this platform has not looked
    at it yet — that last one is what makes a new platform's coverage
    converge on the versions people are actually running, in one pass.
    `--count` also reaches below the floor, for deepening a platform's
    history the way `prime` does.

    `--out` writes the document for another machine to fold; without it the
    document is returned (and printed under `--json`), which is what
    `refresh` uses when both halves run in one process.
    """
    import shutil
    import tempfile

    from livery.toolroom.bench import _toolfetch

    _bounce_bare_call("gather")
    scratch = Path(tempfile.mkdtemp(prefix="footman-gather-"))
    observations: dict[str, dict[str, dict[str, Any]]] = {}
    holes: dict[str, list[str]] = {}
    try:
        with _on_path(prefix), _sandboxed(scratch):
            work, skipped, unreachable = _work_to_do(only, count, _toolfetch)
            surfaces = _gather(work, scratch)
            for driver, release in work:
                surface = surfaces.get(driver.key, {}).get(release.version)
                if surface is None:
                    holes.setdefault(driver.key, []).append(release.version)
                    continue
                observations.setdefault(driver.key, {})[release.version] = {
                    "date": release.date,
                    "tag": release.tag,
                    "surface": surface,
                }
    finally:
        shutil.rmtree(scratch, ignore_errors=True)

    found = Gathered(
        platform=_platform(),
        observations=observations,
        holes={key: sorted(v) for key, v in holes.items()},
        unreachable=unreachable,
        skipped=skipped,
    )
    if out:
        target = Path(out).expanduser()
        target.parent.mkdir(parents=True, exist_ok=True)
        target.write_text(
            json.dumps(found.document(), indent=1, ensure_ascii=False) + "\n",
            encoding="utf-8",
        )
        print(f"wrote {target}")
    _report_gather(found)
    return found

owed

owed(only: Annotated[str, doc('ask about just this tool')] = '', prefix: Annotated[str, doc("drive the tiers from this prefix's bin/")] = '', count: Annotated[int, doc('also reach this many releases below the floor')] = 0) -> Owed

What this platform would read, without installing anything.

A gather provisions every tool and then discovers there is nothing to observe — which is most weeks, since a store that is current stays current until something ships. Listing is network and nothing else, so the question "is there any work" can be answered before the work is prepared for.

One tool must be current for the answer to be true: uv carries CPython's download index inside the binary, so a stale uv reports a stale newest python and this says "nothing owed" about a release it cannot see. Provision uv, ask this, and only then provision the rest.

An index that would not answer is not nothing to do — unreachable is reported separately for exactly that reason.

Source code in packages/toolroom-bench/src/livery/toolroom/bench/_tasks.py
@tasks.task
def owed(
    only: Annotated[str, doc("ask about just this tool")] = "",
    prefix: Annotated[str, doc("drive the tiers from this prefix's bin/")] = "",
    count: Annotated[int, doc("also reach this many releases below the floor")] = 0,
) -> Owed:
    """What this platform would read, without installing anything.

    A gather provisions every tool and then discovers there is nothing to
    observe — which is most weeks, since a store that is current stays
    current until something ships. Listing is network and nothing else, so
    the question "is there any work" can be answered before the work is
    prepared for.

    One tool must be current for the answer to be true: uv carries
    CPython's download index inside the binary, so a stale uv reports a
    stale newest python and this says "nothing owed" about a release it
    cannot see. Provision uv, ask this, and only then provision the rest.

    An index that would not answer is *not* nothing to do — `unreachable`
    is reported separately for exactly that reason.
    """
    from livery.toolroom.bench import _toolfetch

    _bounce_bare_call("owed")
    with _on_path(prefix):
        work, skipped, unreachable = _work_to_do(only, count, _toolfetch)
    releases: dict[str, list[str]] = {}
    for driver, release in work:
        releases.setdefault(driver.key, []).append(release.version)
    found = Owed(
        releases={k: sorted(v) for k, v in sorted(releases.items())},
        unreachable=unreachable,
        skipped=skipped,
    )
    if found.unreachable:
        for key, why in sorted(found.unreachable.items()):
            print(f"unreachable: {key} ({why})")
    if found.total:
        for tool, versions in found.releases.items():
            print(f"{tool}: {len(versions)} to read")
    print(f"owed: {found.total}")
    return found

assemble

assemble(documents: Annotated[list[str], doc('observation documents to fold in')], changelog: Annotated[bool, doc('write the events into CHANGELOG.md')] = True) -> Refreshed

Fold gathered observations into the store — the single-writer half.

Every platform's reading of one release is folded into one surface and one sidecar before the chain is touched, so a matrix run never writes the churn of an option being inserted, dropped and resurrected as each leg's turn comes. Then releases go in oldest first, and a release the chain already holds is merged rather than replaced.

One process, one owner per file. Three machines committing to tool-history/ on their own would be three whole-file conflicts a week: git is not a merge engine for this, and the algebra is.

Source code in packages/toolroom-bench/src/livery/toolroom/bench/_tasks.py
@tasks.task
def assemble(
    documents: Annotated[list[str], doc("observation documents to fold in")],
    changelog: Annotated[bool, doc("write the events into CHANGELOG.md")] = True,
) -> Refreshed:
    """Fold gathered observations into the store — the single-writer half.

    Every platform's reading of one release is folded into one surface and
    one sidecar *before* the chain is touched, so a matrix run never writes
    the churn of an option being inserted, dropped and resurrected as each
    leg's turn comes. Then releases go in oldest first, and a release the
    chain already holds is merged rather than replaced.

    One process, one owner per file. Three machines committing to
    `tool-history/` on their own would be three whole-file conflicts a week:
    git is not a merge engine for this, and the algebra is.
    """
    return _finish(
        _assemble_documents([_read_document(name) for name in documents]), changelog
    )

refresh

refresh(only: Annotated[str, doc('refresh just this tool')] = '', prefix: Annotated[str, doc("drive the tiers from this prefix's bin/")] = '', changelog: Annotated[bool, doc('write the events into CHANGELOG.md')] = True, submit: Annotated[bool, doc('commit what moved on a branch and open its pull request')] = False) -> Refreshed

Observe what is new on this platform, and fold it into the store.

gather then assemble, in one process — the whole job when one machine is the whole matrix, and exactly the two halves the weekly workflow runs on three machines and one assembler. There is no third code path: what runs locally is what runs in CI, with the document handed across a function call instead of an artifact.

Nothing new anywhere means nothing to release, and that is the whole exit condition. An index that would not answer is reported and exits non-zero rather than counting as "nothing new": a rename or a moved repo would otherwise make a tool silently untracked while the job kept reporting success.

With --submit, a refresh that moved a tool's surface is committed on the day's branch and submitted as a pull request: armed when every change only added to a surface, the graded trigger's green light, and left for a person otherwise. A day's branch that origin already holds with every file this refresh wrote means the day's refresh is already submitted. Nothing moved means no branch and no pull request. A refresh asked to submit submits from the history of origin's default branch alone, so on another checkout, a branch a rehearsal runs on among them, it observes nothing and says so.

Source code in packages/toolroom-bench/src/livery/toolroom/bench/_tasks.py
@tasks.task
def refresh(
    only: Annotated[str, doc("refresh just this tool")] = "",
    prefix: Annotated[str, doc("drive the tiers from this prefix's bin/")] = "",
    changelog: Annotated[bool, doc("write the events into CHANGELOG.md")] = True,
    submit: Annotated[
        bool, doc("commit what moved on a branch and open its pull request")
    ] = False,
) -> Refreshed:
    """Observe what is new on this platform, and fold it into the store.

    `gather` then `assemble`, in one process — the whole job when one
    machine is the whole matrix, and exactly the two halves the weekly
    workflow runs on three machines and one assembler. There is no third
    code path: what runs locally is what runs in CI, with the document
    handed across a function call instead of an artifact.

    Nothing new anywhere means nothing to release, and that is the whole
    exit condition. An index that would not answer is reported and exits
    non-zero rather than counting as "nothing new": a rename or a moved repo
    would otherwise make a tool silently untracked while the job kept
    reporting success.

    With `--submit`, a refresh that moved a tool's surface is
    committed on the day's branch and submitted as a pull request:
    armed when every change only added to a surface, the graded
    trigger's green light, and left for a person otherwise. A day's
    branch that origin already holds with every file this refresh
    wrote means the day's refresh is already submitted. Nothing moved
    means no branch and no pull request. A refresh asked to submit
    submits from the history of origin's default branch alone, so on
    another checkout, a branch a rehearsal runs on among them, it
    observes nothing and says so.
    """
    import livery.toolroom.tools as toolroom

    if submit and not _on_default_branch(
        toolroom.git.opts(nofail=True, recorded=False)
    ):
        print(
            "  the checkout is not on origin's default branch, so the refresh"
            " observes nothing: it submits from the default branch alone"
        )
        return Refreshed(read={}, events={}, unreachable={}, skipped=[])
    found = gather(only=only, prefix=prefix)
    finished = _finish(_assemble_documents([found.document()]), changelog)
    if submit:
        for line in submit_refresh(finished):
            print(line)
    return finished

submit_refresh

submit_refresh(found: Refreshed, *, git: Callable[..., Result] | None = None, submit: Callable[[list[str]], int] | None = None, on: date | None = None, dry_run: bool | None = None) -> list[str]

Commit a refresh that moved something and open its pull request.

The branch is chore/tools-refresh-<date>, cut from HEAD, and a refresh submits only from the history of origin's default branch: on another checkout it submits nothing and says so, and the files it wrote stand. When origin already holds the day's branch, from an earlier refresh on another machine or in another job, the refresh compares and moves nothing: when that branch holds every file this refresh wrote, with the same content, it submits nothing and says the day's refresh is already submitted; otherwise it stops, naming the branch and the files that differ, for a person to merge or close that pull request and run the refresh again. The checkout stays on its own tree either way: the run reads its code and declarations from the checkout after the refresh, and the day's branch can be cut from an older main. Origin is asked before anything in the checkout changes, so a refresh that cannot ask it leaves the checkout as the walk left it.

The pull request is armed when Refreshed.armed holds, every change an addition and every ingest check passed, and unarmed otherwise, so a dropped verb or a deployment that does not stand waits for a person. The commit body carries the ingest lines, so the pull request's summary is the structural diff. A refresh that moved nothing opens nothing and says so. Under a dry run nothing is committed or submitted: the lines say what would happen, the arming decision included.

git, submit, on and dry_run are the seams the tests drive; the defaults are the git tool handle, the runner's own submit verb, today, and the run's own dry-run flag. git answers each command with its exit code and output and never raises; a code this does not expect fails the refresh in git's own words.

Returns:

Type Description
list[str]

The lines to print, one per step.

Source code in packages/toolroom-bench/src/livery/toolroom/bench/_tasks.py
def submit_refresh(
    found: Refreshed,
    *,
    git: Callable[..., Result] | None = None,
    submit: Callable[[list[str]], int] | None = None,
    on: date | None = None,
    dry_run: bool | None = None,
) -> list[str]:
    """Commit a refresh that moved something and open its pull request.

    The branch is `chore/tools-refresh-<date>`, cut from HEAD, and a
    refresh submits only from the history of origin's default branch:
    on another checkout it submits nothing and says so, and the files it
    wrote stand. When origin already
    holds the day's branch, from an earlier refresh on another machine
    or in another job, the refresh compares and moves nothing: when
    that branch holds every file this refresh wrote, with the same
    content, it submits nothing and says the day's refresh is already
    submitted; otherwise it stops, naming the branch and the files that
    differ, for a person to merge or close that pull request and run
    the refresh again. The checkout stays on its own tree either way:
    the run reads its code and declarations from the checkout after the
    refresh, and the day's branch can be cut from an older main. Origin
    is asked before anything in the checkout changes, so a refresh that
    cannot ask it leaves the checkout as the walk left it.

    The pull request is armed when `Refreshed.armed` holds, every change
    an addition and every ingest check passed, and unarmed otherwise, so
    a dropped verb or a deployment that does not stand waits for a
    person. The commit body carries the ingest lines,
    so the pull request's summary is the structural diff. A refresh that
    moved nothing opens nothing and says so. Under a dry run nothing is
    committed or submitted: the lines say what would happen, the arming
    decision included.

    *git*, *submit*, *on* and *dry_run* are the seams the tests drive;
    the defaults are the git tool handle, the runner's own submit verb,
    today, and the run's own dry-run flag. *git* answers each command
    with its exit code and output and never raises; a code this does
    not expect fails the refresh in git's own words.

    Returns:
        The lines to print, one per step.
    """
    import livery.toolroom.tools as tools

    if not found.release:
        return ["  nothing moved: no branch, no pull request"]
    when = on or date.today()
    branch = f"chore/tools-refresh-{when:%Y%m%d}"
    moved = ", ".join(sorted(found.events))
    title = f"chore(toolroom): tool refresh {when:%Y-%m-%d}"
    reason = (
        "additions only, every ingest check passed"
        if found.armed
        else "a surface lost something, a person decides"
        if found.ingest_ok
        else "an ingest check found something, a person decides"
    )
    rehearsal = current().dry_run if dry_run is None else dry_run
    if rehearsal:
        return [
            f"  would refresh {moved} on {branch}",
            f"  would submit {'armed' if found.armed else 'unarmed'}: {reason}",
        ]
    run_git: Callable[..., Result] = git or tools.git.opts(nofail=True, recorded=False)
    body = "\n".join(
        [
            moved,
            *(
                line.strip()
                for key in sorted(found.ingest)
                for line in found.ingest[key]
            ),
        ]
    )
    message = f"{title}\n\n{body}"
    tracking = f"refs/remotes/origin/{branch}"
    held = _origin_holds(run_git, branch)
    if not _on_default_branch(run_git):
        return [
            "  the checkout is not on origin's default branch, so nothing is"
            " submitted: a refresh submits from the default branch alone, and"
            " the files it wrote stand in the checkout"
        ]
    if held:
        _git_step(run_git, "fetch", "origin", f"+refs/heads/{branch}:{tracking}")
    _git_step(run_git, "switch", "-c", branch)
    _git_step(run_git, "add", "-A")
    _git_step(run_git, "commit", "-m", message)
    if held:
        differ = _differing(run_git, tracking)
        if not differ:
            return [
                f"  the day's refresh is already submitted: {branch} on origin"
                " holds every file this one wrote, so nothing is submitted"
            ]
        fail(
            f"{branch} on origin holds an earlier refresh, and this one wrote"
            f" {', '.join(differ)} differently. Merge or close that pull"
            " request, then run the refresh again; this refresh's commit"
            f" stands on the local branch {branch}."
        )
    argv = [_prog(), "submit", f"--title={title}"]
    if found.armed:
        argv.append("--armed")
    code = (submit or _run_submit)(argv)
    if code != 0:
        fail(f"the refresh's submit exited {code}; the branch {branch} stands")
    return [
        f"  refreshed {moved} on {branch}",
        f"  submitted {'armed' if found.armed else 'unarmed'}: {reason}",
    ]

observe

observe(tool: str, version: str, tag: str = '', date: str = '', published: str = '', requires_python: str = '', scratch: str = '') -> dict[str, Any] | None

Install one release, read what it accepts, and throw it away.

The unit of the gather, pure in (tool, version): requests for the same release dedupe on the futures work key, and arrival order is nobody's business: the record places each version at its own position whatever order these finish in.

A real task deliberately, not a helper. The task boundary is what buys each observation its own environment: a body call copies the caller's overlay, so the PATH written around extraction here is this observation's alone, while the sandbox variables and prefix PATH the caller set flow in — and on into every subprocess the tiers spawn.

None — a release that would not install, or a binary that would not describe itself — is a hole for the caller to report, never an error: the chain stays contiguous by construction, and a later run fills it.

Source code in packages/toolroom-bench/src/livery/toolroom/bench/_tasks.py
@tasks.task(hidden=True)
def observe(
    tool: str,
    version: str,
    tag: str = "",
    date: str = "",
    published: str = "",
    requires_python: str = "",
    scratch: str = "",
) -> dict[str, Any] | None:
    """Install one release, read what it accepts, and throw it away.

    The unit of the gather, pure in (tool, version): requests for the same
    release dedupe on the futures work key, and arrival order is nobody's
    business: the record places each version at its own position whatever
    order these finish in.

    A real task deliberately, not a helper. The task boundary is what buys
    each observation its own environment: a body call copies the caller's
    overlay, so the `PATH` written around extraction here is this
    observation's alone, while the sandbox variables and prefix `PATH` the
    caller set flow in — and on into every subprocess the tiers spawn.

    `None` — a release that would not install, or a binary that would not
    describe itself — is a hole for the caller to report, never an error:
    the chain stays contiguous by construction, and a later run fills it.
    """
    from livery.toolroom.bench import _toolfetch

    driver = _drivers.find(tool)
    if driver is None or not scratch:  # pragma: no cover - engine-supplied
        return None
    # Every Release field crosses the task boundary, or a fix to the walk
    # silently reverts inside it: the publishing-window cutoff never reached
    # _install_pypi through here, so cmake 4.3.1 kept resolving at the near
    # edge and holing — while the identical install ran clean by hand.
    release = _toolfetch.Release(
        version=version,
        tag=tag,
        date=date,
        published=published,
        requires_python=requires_python,
    )
    into = Path(scratch) / f"{tool}-{version}"
    placed = _toolfetch.install(driver, release, into)
    if placed is None:
        _refuse_a_broken_environment(Path(scratch))
        return None
    try:
        with _reading(driver, placed):
            # The home this walk made, never the one a lookup would find.
            spec = _extract(driver, home=_fetched_home(driver, placed))
    finally:
        _discard(into)
    if not _describes_itself(spec):
        return None
    if spec.version and not _same_release(spec.version, version):
        # The one lie _describes_itself cannot see: a faithful description
        # of the *wrong* binary. When the release's own directory is missing
        # from PATH (or its install left no binary), the extractor resolves
        # some ambient tool instead and reads it under this release's label —
        # the help-path twin of the guard _from_click already carries. A
        # reading that names a different version is a hole, not an
        # observation.
        return None
    return _surfaces.surface_of(spec)

tools_verify

tools_verify(tool: Annotated[str, doc('the curated tool, as its record is named')], version: Annotated[str, doc('the version to check; the newest with a host when empty')] = '') -> None

Run the ingest checks on one version of a tool for every host, from here.

Each host's artifact is staged through the bench's store and looked at; the verdict, each finding and each host's structural diff against the version before it print, and a finding is red.

Source code in packages/toolroom-bench/src/livery/toolroom/bench/_tasks.py
@tasks.task(name="verify")
def tools_verify(
    tool: Annotated[str, doc("the curated tool, as its record is named")],
    version: Annotated[
        str, doc("the version to check; the newest with a host when empty")
    ] = "",
) -> None:
    """Run the ingest checks on one version of a tool for every host, from here.

    Each host's artifact is staged through the bench's store and looked
    at; the verdict, each finding and each host's structural diff
    against the version before it print, and a finding is red.
    """
    from livery.toolroom.bench import _ingest

    record = _surfaces.load(_record_path(tool))
    if record is None:
        fail(f"no record of {tool}; the records are {', '.join(_record_names())}")
    chosen = version or next(
        (delta.version for delta in reversed(record.deltas) if delta.hosts), ""
    )
    if not chosen:
        fail(f"{tool} has no version with a host; nothing to stage")
    report = _ingest.verify(record, chosen, store=_bench_store())
    for line in _ingest.summary(report):
        print(line)
    if not report.passed:
        fail(f"{tool} {chosen}: {len(report.findings)} finding(s)")

tools_artifacts

tools_artifacts(tool: Annotated[str, doc('the curated tool, as its record is named')], version: Annotated[str, doc('the version to record; the newest when empty')] = '') -> None

Record a version's release artifacts per host, then run the ingest checks.

For a tool read from a forge tier: the release's asset for each host is downloaded once, hashed and written on the version line, a host the release has no asset for is left absent, and the nine checks then run on every host recorded. The record is saved before the checks, so a finding is red with the artifacts in place for the fix. A version already read from another index, as a tool has before it moves to its own release, is addressed by its version when no forge tag is known for it.

Source code in packages/toolroom-bench/src/livery/toolroom/bench/_tasks.py
@tasks.task(name="artifacts")
def tools_artifacts(
    tool: Annotated[str, doc("the curated tool, as its record is named")],
    version: Annotated[str, doc("the version to record; the newest when empty")] = "",
) -> None:
    """Record a version's release artifacts per host, then run the ingest checks.

    For a tool read from a forge tier: the release's asset for each
    host is downloaded once, hashed and written on the version line,
    a host the release has no asset for is left absent, and the nine
    checks then run on every host recorded. The record is saved before
    the checks, so a finding is red with the artifacts in place for
    the fix. A version already read from another index, as a tool has
    before it moves to its own release, is addressed by its version
    when no forge tag is known for it.
    """
    from livery.toolroom.bench import _ingest, _toolfetch
    from livery.toolroom.store import RecordError

    try:
        record = _surfaces.load(_record_path(tool))
    except RecordError as error:
        fail(f"{_record_path(tool)}: {error}")
    if record is None:
        fail(f"no record of {tool}; the records are {', '.join(_record_names())}")
    driver = _drivers.find(tool)
    if driver is None:
        fail(f"no driver for {tool}")
    if driver.base:
        fail(
            f"{tool} is a view of {driver.name}'s binary bound to"
            f" `{' '.join(driver.base)}`; record {driver.name}"
        )
    chosen = version or (_surfaces.versions(record) or [""])[0]
    if not chosen:
        fail(f"{tool} has no version; read one first")
    tag = ""
    if _artifacts.lists_assets(driver):
        tag = next(
            (r.tag for r in _toolfetch.releases(driver) if r.version == chosen), ""
        )
    store = _bench_store()
    try:
        record, done = _artifacts.record_version(
            record, driver, chosen, tag, store=store
        )
    except _artifacts.ArtifactError as error:
        fail(str(error))
    _surfaces.save(record, _record_path(tool))
    for line in done.lines(tool):
        print(line)
    report = _ingest.verify(record, chosen, store=store)
    for line in _ingest.summary(report):
        print(line)
    if not report.passed:
        fail(f"{tool} {chosen}: {len(report.findings)} finding(s)")

index_build

index_build(into: Annotated[str, doc('the index root; omitted = docs/tools in the workspace')] = '', from_genesis: Annotated[bool, doc('ignore the build already there and materialise every tool')] = False) -> Built

Materialise every record into the index: a strongroom store and its pointer.

The index is a strongroom store served as static files, one tree per tool holding the tool axis, each version's deployment per host and its surface as one blob, and beside it pointer.json naming each tool's current tree. A build into a directory that already holds one reads its pointer and reuses every tool whose record did not move; --from-genesis ignores the pointer and rebuilds every tool, and two builds from genesis land the same objects and the same pointer.

Declared as a docs generator, so the site's build writes the index under docs/tools and the site's deploy serves it at tools/.

Source code in packages/toolroom-bench/src/livery/toolroom/bench/_tasks.py
@index_tasks.task(name="build")
def index_build(
    into: Annotated[
        str, doc("the index root; omitted = docs/tools in the workspace")
    ] = "",
    from_genesis: Annotated[
        bool, doc("ignore the build already there and materialise every tool")
    ] = False,
) -> _index.Built:
    """Materialise every record into the index: a strongroom store and its pointer.

    The index is a strongroom store served as static files, one tree per
    tool holding the tool axis, each version's deployment per host and
    its surface as one blob, and beside it `pointer.json` naming each
    tool's current tree. A build into a directory that already
    holds one reads its pointer and reuses every tool whose record did
    not move; `--from-genesis` ignores the pointer and rebuilds every
    tool, and two builds from genesis land the same objects and the same
    pointer.

    Declared as a docs generator, so the site's build writes the index
    under `docs/tools` and the site's deploy serves it at `tools/`.
    """
    target = (
        Path(into).expanduser().resolve()
        if into
        else _records_dir().parent / "docs" / "tools"
    )
    built = _index.build(_records_dir(), target, from_genesis=from_genesis)
    for name in built.rebuilt:
        print(f"built {name} {built.tools[name]}")
    if built.reused:
        print(f"reused {len(built.reused)}: {', '.join(built.reused)}")
    if built.dropped:
        print(f"dropped: {', '.join(built.dropped)}")
    print(f"pointer: {target / _index.POINTER} ({len(built.tools)} tools)")
    return built

golden_records

golden_records() -> list[Record]

The golden records, small and shaped to exercise the renderer.

Source code in packages/toolroom-bench/src/livery/toolroom/bench/_tasks.py
def golden_records() -> list[Record]:
    """The golden records, small and shaped to exercise the renderer."""
    return _index.load_records(_GOLDENS / "records")

golden_path

golden_path(record: Record, version: str) -> Path

Where record's stub at version is checked in.

Source code in packages/toolroom-bench/src/livery/toolroom/bench/_tasks.py
def golden_path(record: Record, version: str) -> Path:
    """Where *record*'s stub at *version* is checked in."""
    return _GOLDENS / "stubs" / record.name / f"{version}.pyi.golden"

goldens

goldens(check: Annotated[bool, doc('report what differs instead of writing')] = False) -> dict[str, list[str]]

Render the golden records, the golden stubs and the records' schema.

A golden is a small record shaped to exercise the renderer, and its stub at every version is checked in, rendered as a consumer renders the version it locks. The test compares each render with its golden byte for byte, so a change to the renderer fails the gate until this verb moves the goldens in the same change, which is what keeps a render change deliberate. records/record.schema.json is the same kind of file, exported from the record's shape and compared by a test, so it moves here too. --check names what would move and writes nothing.

Source code in packages/toolroom-bench/src/livery/toolroom/bench/_tasks.py
@tasks.task
def goldens(
    check: Annotated[bool, doc("report what differs instead of writing")] = False,
) -> dict[str, list[str]]:
    """Render the golden records, the golden stubs and the records' schema.

    A golden is a small record shaped to exercise the renderer, and its
    stub at every version is checked in, rendered as a consumer renders
    the version it locks. The test compares each render with its golden
    byte for byte, so a change to the renderer fails the gate until this
    verb moves the goldens in the same change, which is what keeps a
    render change deliberate. `records/record.schema.json` is the same
    kind of file, exported from the record's shape and compared by a
    test, so it moves here too. `--check` names what would move and
    writes nothing.
    """
    from livery.toolroom.store import Catalogue, export_schema

    wrote: list[str] = []
    unchanged: list[str] = []
    schema = _records_dir() / "record.schema.json"
    scratch = schema.with_name("record.schema.json.new")
    export_schema(scratch)
    fresh = scratch.read_text(encoding="utf-8")
    scratch.unlink()
    if schema.is_file() and schema.read_text(encoding="utf-8") == fresh:
        unchanged.append("record schema")
    else:
        if not check:
            schema.write_text(fresh, encoding="utf-8")
        wrote.append("record schema")
    catalogue = Catalogue.of_records(_GOLDENS / "records")
    for record in golden_records():
        for version in _surfaces.versions(record):
            path = golden_path(record, version)
            text = catalogue.stub(record.name, version)
            if path.is_file() and path.read_text(encoding="utf-8") == text:
                unchanged.append(f"{record.name} {version}")
                continue
            if not check:
                path.parent.mkdir(parents=True, exist_ok=True)
                path.write_text(text, encoding="utf-8")
            wrote.append(f"{record.name} {version}")
    verb = "would move" if check else "wrote"
    print(f"{verb} {len(wrote)}: {', '.join(wrote) or 'none'}")
    if unchanged:
        print(f"unchanged: {len(unchanged)}")
    return {"wrote": wrote, "unchanged": unchanged}

provision

provision(only: Annotated[str, doc('provision just this tool')] = '', prefix: Annotated[Path | None, doc("directory to materialise into; omitted = footman's data dir")] = None, sync_: Annotated[bool, doc('run `tools read` against the prefix afterwards')] = False, clean: Annotated[bool, doc('remove the prefix when done')] = False, strict: Annotated[bool, doc('fail if any tier could not be provisioned')] = False) -> None

Fetch the latest curated tools into an isolated prefix — no pollution.

The records are read from installed binaries, so syncing against the newest release means having it on PATH. This gathers the latest of every curated tool under one isolated prefix — uv tool install for the PyPI wheels (the Rust and C++ tools included), bun's own release then bun add for the node CLIs, a release asset for the Go ones — touching nothing outside it. Omitted, the prefix is the toolroom-bench room in footman's data directory, which every empty-prefix reading looks in first, so provisioning once serves every later sync/audit on this machine. --sync reads the records against the prefix; --clean deletes it, which is the whole undo.

--strict turns a failed tier into a failed run. Without it the table names what did not arrive and the run still succeeds, which is right for a person deciding what to do next and wrong for a job that will read the prefix and believe it.

Source code in packages/toolroom-bench/src/livery/toolroom/bench/_tasks.py
@tasks.task
def provision(
    only: Annotated[str, doc("provision just this tool")] = "",
    prefix: Annotated[
        Path | None,
        doc("directory to materialise into; omitted = footman's data dir"),
    ] = None,
    sync_: Annotated[
        bool, doc("run `tools read` against the prefix afterwards")
    ] = False,
    clean: Annotated[bool, doc("remove the prefix when done")] = False,
    strict: Annotated[bool, doc("fail if any tier could not be provisioned")] = False,
) -> None:
    """Fetch the latest curated tools into an isolated prefix — no pollution.

    The records are read from installed binaries, so syncing against the newest
    release means having it on PATH. This gathers the latest of every curated
    tool under one isolated prefix — `uv tool install` for the PyPI wheels
    (the Rust and C++ tools included), bun's own release then `bun add` for the
    node CLIs, a release asset for the Go ones — touching nothing outside it.
    Omitted, the prefix is the `toolroom-bench` room in footman's data directory,
    which every empty-prefix reading looks in first, so provisioning once
    serves every later `sync`/`audit` on this machine. `--sync` reads the
    records against the prefix; `--clean` deletes it, which is the whole undo.

    `--strict` turns a failed tier into a failed run. Without it the table
    names what did not arrive and the run still succeeds, which is right
    for a person deciding what to do next and wrong for a job that will
    read the prefix and believe it.
    """
    from livery.toolroom.bench import _provision

    # Absolute: bun errors `ReadOnlyFileSystem` on a relative BUN_INSTALL, and
    # an absolute prefix keeps every tier's launchers and env vars unambiguous.
    # Omitted, the binaries land in the default room — the same place every
    # empty-prefix reading looks first.
    prefix = (
        Path(prefix).expanduser().resolve() if prefix is not None else default_prefix()
    )
    outcomes = _provision.provision(_drivers.DRIVERS, prefix, only=only)
    _print_outcomes(outcomes)
    if strict:
        from livery.footman import fail

        # A person at a terminal reads the table and decides what to do
        # next, so a failed tier is named and the rest carries on. A step
        # whose whole purpose is to leave a prefix complete has no such
        # judgement: a run where bun hit a rate limit still said `ok`, and
        # a half-provisioned prefix went into the gather unremarked.
        failed = [out for out in outcomes if out.status == "fail"]
        if failed:
            fail(
                "could not provision "
                + ", ".join(f"{out.key} ({out.detail})" for out in failed),
                code=70,  # EX_SOFTWARE: the prefix is not what was asked for
            )
    if sync_:
        _sync_against(prefix, only)
    else:
        print(
            f'\nput them on PATH:\n  export PATH="{_provision.bin_dir(prefix)}:$PATH"'
        )
    if clean:
        import shutil

        shutil.rmtree(prefix, ignore_errors=True)
        print(f"removed {prefix}")

pages

pages(out: Annotated[Path, doc('directory to write the reference pages into')], nav: Annotated[Path | None, doc('the generated docs tree to emit the Tools nav block into')] = None, stubs: Annotated[Path | None, doc("directory to render the pages' stubs into")] = None) -> None

Write one reference page per tool, plus the index table.

Rendered from the records rather than from the installed tools, so the docs build needs nothing on PATH and says exactly what the records hold. The stubs the pages point at are written as the module toolroom_stubs under stubs, <out>/../stubs/toolroom_stubs by default, a search path the docs build adds for its renderer. Tools are ordered alphabetically. With nav, the Tools nav block is emitted into that generated tree too, so the sidebar can never fall behind the drivers again.

Source code in packages/toolroom-bench/src/livery/toolroom/bench/_tasks.py
@tasks.task
def pages(
    out: Annotated[Path, doc("directory to write the reference pages into")],
    nav: Annotated[
        Path | None, doc("the generated docs tree to emit the Tools nav block into")
    ] = None,
    stubs: Annotated[
        Path | None, doc("directory to render the pages' stubs into")
    ] = None,
) -> None:
    """Write one reference page per tool, plus the index table.

    Rendered from the records rather than from the installed tools, so
    the docs build needs nothing on PATH and says exactly what the records
    hold. The stubs the pages point at are written as the module
    `toolroom_stubs` under *stubs*, `<out>/../stubs/toolroom_stubs` by
    default, a search path the docs build adds for its renderer. Tools
    are ordered alphabetically. With *nav*, the Tools nav block is
    emitted into that generated tree too, so the sidebar can never fall
    behind the drivers again.
    """
    out.mkdir(parents=True, exist_ok=True)
    module = stubs if stubs is not None else out.parent / "stubs" / STUBS_MODULE
    module.mkdir(parents=True, exist_ok=True)
    (module / "__init__.pyi").write_text("", encoding="utf-8")
    stubbed: list[tuple[_drivers.Driver, Record]] = []
    for driver in sorted(_drivers.DRIVERS, key=lambda d: d.key):
        record = _surfaces.load(_record_path(driver.key))
        if record is None:
            continue
        if _surfaces.versions(record):
            (module / f"{driver.key}.pyi").write_text(
                _stub_from(driver, record), encoding="utf-8"
            )
        stubbed.append((driver, record))
    for stale in module.glob("*.pyi"):
        if stale.stem != "__init__" and stale.stem not in {d.key for d, _ in stubbed}:
            stale.unlink()
    rows = ["| Tool | Read from | In-process | Verbs |", "| --- | --- | --- | --- |"]
    for driver, record in stubbed:
        rows.append(_row(driver, record, module / f"{driver.key}.pyi"))
        (out / f"{driver.key}.md").write_text(_page(driver), encoding="utf-8")
    (out / "index.md").write_text(
        _INDEX.format(table="\n".join(rows)), encoding="utf-8"
    )
    if nav is not None:
        write_tools_nav(nav, [d.key for d, _ in stubbed])
    print(f"wrote {len(stubbed)} tool page(s) into {out}")

write_tools_nav

write_tools_nav(generated: Path, keys: list[str]) -> Path

Emit the Tools nav block's entries from keys into generated; the path.

The block is data the docs site reads: nav.tools.toml beside the pages, a nav list of one-key tables, each a label to its page. The bench writes the file itself, since only an extension that requires another imports it, and the bench is no extension.

Source code in packages/toolroom-bench/src/livery/toolroom/bench/_tasks.py
def write_tools_nav(generated: Path, keys: list[str]) -> Path:
    """Emit the Tools nav block's entries from *keys* into *generated*; the path.

    The block is data the docs site reads: `nav.tools.toml` beside the
    pages, a `nav` list of one-key tables, each a label to its page. The
    bench writes the file itself, since only an extension that requires
    another imports it, and the bench is no extension.
    """
    entries = "".join(f'{{ "{k}" = "_generated/tools/{k}.md" }},\n' for k in keys)
    generated.mkdir(parents=True, exist_ok=True)
    path = generated / "nav.tools.toml"
    path.write_text(
        "# The 'tools' nav block, emitted by toolroom.pages; the authored"
        f" nav.toml places it.\nnav = [\n{entries}]\n",
        encoding="utf-8",
    )
    return path

nav_keys

nav_keys(block: Path) -> list[str]

The tool keys the emitted Tools nav block at block lists, in order.

Source code in packages/toolroom-bench/src/livery/toolroom/bench/_tasks.py
def nav_keys(block: Path) -> list[str]:
    """The tool keys the emitted Tools nav block at *block* lists, in order."""
    if not block.is_file():
        return []
    return [m["key"] for m in _NAV_ENTRY.finditer(block.read_text(encoding="utf-8"))]