Skip to content

livery.toolroom.bench._provision

livery.toolroom.bench._provision

Fetch the latest curated tools into a throwaway prefix — the engine behind fm toolroom.provision.

The stubs are read from the installed binaries (fm toolroom.read), so telling an editor what the newest release accepts means having the newest release on PATH — across five ecosystems (PyPI, npm, bun, Go, C++), none of which should be allowed to touch the machine's own environment.

One isolated prefix answers all of it. A tool comes from its own release when it has one: a forge tier downloads the release asset for the host. PyPI is the tier for programs that are Python, where the wheel is the release; a wheel around a Rust or C++ binary is a wrapper with a platform gap wherever the wheel is missing, so a tool that is not Python never sits on the uv tier (a test pins the tier's members). What the forge tiers do not cover is node (its own release index), the npm packages and their runtimes, and the Python tools:

  • uv — uv tool install --upgrade <pkg>, tools and launchers under the prefix; nothing lands in ~/.local or the system site-packages. Python programs only.
  • bun — bun's GitHub release, unpacked into the prefix: the runtime a node-tier package names instead of node.
  • nodejs — node's build for this platform from nodejs.org, unpacked whole into the prefix with npm beside it, and node linked into the prefix's bin. Provisioned before the node tier, which runs on it.
  • node — each npm package through its runtime: npm install --global on the provisioned node, or bun add --global with BUN_INSTALL pointed at the prefix where the driver names bun.
  • github / gitlab — the latest release asset for this platform, matched from the release's own asset list (so Darwin/x86_64 vs darwin/x64 naming needn't be transcribed), unpacked whole, a launcher placed in the prefix.
  • system — git, docker, the uv running this: already on PATH, left be.
  • deferred — parked, with a reason (tea, until it stops hanging on --help).

Everything writes under one prefix and PATH="<prefix>/bin:$PATH" is all a sync needs to read the newest binaries; deleting the prefix undoes it. This is a maintainer tool: it shells out and downloads, and it is never on the completion hot path.

Classes:

Name Description
ProvisionError

A tool could not be fetched — reported per tool, never fatal.

Outcome

What became of one tool: the line provision prints.

Functions:

Name Description
bin_dir

The one directory to put on PATH; every tier lands its launchers here.

exe

name as this platform spells an executable.

provision

Materialise the latest of each curated tool under prefix.

provisioned_node

The node the nodejs tier unpacked under prefix, or None before it ran.

provisioned_dotnet

The dotnet the dotnet tier unpacked under prefix, or None before it ran.

assets_for

[(asset name, download url)] for one release — latest when tag is

place_binary

Unpack archive whole and put its tool in into; the file placed there.

write_launcher

A launcher in into that runs target with args first; its path.

Attributes:

Name Type Description
HOST_TOKENS

Each store host key as an OS and a CPU, in the spellings the alias tables fold.

HOST_TOKENS module-attribute

HOST_TOKENS = {host: (_SYSTEMS[host.split('-')[0]], _MACHINES[host.split('-')[1]]) for host in HOSTS}

Each store host key as an OS and a CPU, in the spellings the alias tables fold.

ProvisionError

Bases: Exception

A tool could not be fetched — reported per tool, never fatal.

Outcome dataclass

Outcome(key: str, kind: str, status: str, detail: str = '')

What became of one tool: the line provision prints.

Attributes:

Name Type Description
key str
kind str
status str
detail str

key instance-attribute

key: str

kind instance-attribute

kind: str

status instance-attribute

status: str

detail class-attribute instance-attribute

detail: str = ''

bin_dir

bin_dir(prefix: Path) -> Path

The one directory to put on PATH; every tier lands its launchers here.

Source code in packages/toolroom-bench/src/livery/toolroom/bench/_provision.py
def bin_dir(prefix: Path) -> Path:
    """The one directory to put on `PATH`; every tier lands its launchers here."""
    return prefix / "bin"

exe

exe(name: str, *, windows: bool | None = None) -> str

name as this platform spells an executable.

Windows resolves a command through PATHEXT, so an extensionless PE is invisible to shutil.which and to every reader that follows it. Every tier that names a binary needs the same suffix, and each one that grew its own copy of the conditional was a separate Windows bug — the placed file gained .exe while the tier still looked for the bare name.

Source code in packages/toolroom-bench/src/livery/toolroom/bench/_provision.py
def exe(name: str, *, windows: bool | None = None) -> str:
    """*name* as this platform spells an executable.

    Windows resolves a command through `PATHEXT`, so an extensionless PE is
    invisible to `shutil.which` and to every reader that follows it. Every
    tier that names a binary needs the same suffix, and each one that grew
    its own copy of the conditional was a separate Windows bug — the placed
    file gained `.exe` while the tier still looked for the bare name.
    """
    if windows is None:
        windows = os.name == "nt"
    return f"{name}.exe" if windows else name

provision

provision(drivers: tuple[Driver, ...], prefix: Path, *, only: str = '') -> list[Outcome]

Materialise the latest of each curated tool under prefix.

Tiers run in the one order that matters: the runtimes before the npm packages that run on them. Each tool's failure is its own line, never the run's — a missing binary should read as one skipped hint, not a broken provision.

Source code in packages/toolroom-bench/src/livery/toolroom/bench/_provision.py
def provision(
    drivers: tuple[Driver, ...], prefix: Path, *, only: str = ""
) -> list[Outcome]:
    """Materialise the latest of each curated tool under *prefix*.

    Tiers run in the one order that matters: the runtimes before the npm
    packages that run on them. Each tool's failure is its own line, never
    the run's — a missing binary should read as one skipped hint, not a
    broken provision.
    """
    prefix = Path(prefix)
    bin_dir(prefix).mkdir(parents=True, exist_ok=True)
    wanted = {name.strip() for name in only.split(",") if name.strip()}
    chosen = [d for d in drivers if not wanted or d.key in wanted]
    outcomes: list[Outcome] = []
    by_kind: dict[str, list[Driver]] = {}
    for driver in chosen:
        if driver.source == "manual":
            # A hand-written stub (the shells): its stub is curated, not read
            # from a binary, so there is nothing to fetch — skip it rather than
            # try `uv tool install bash` and print a spurious failure.
            outcomes.append(Outcome(driver.key, "manual", "skip", "hand-written"))
            continue
        by_kind.setdefault(driver.provision.kind, []).append(driver)

    for driver in by_kind.get("deferred", []):
        outcomes.append(
            Outcome(driver.key, "deferred", "deferred", driver.provision.note)
        )
    outcomes += _uv_tier(prefix, by_kind.get("uv", []))
    outcomes += _python_tier(prefix, by_kind.get("python", []))
    for driver in by_kind.get("bun", []):  # a runtime: before the node tier
        outcomes.append(_release(prefix, driver, host="github"))
    outcomes += _nodejs_tier(prefix, by_kind.get("nodejs", []))
    outcomes += _node_tier(prefix, by_kind.get("node", []))
    outcomes += _dotnet_tier(prefix, by_kind.get("dotnet", []))
    outcomes += _nuget_tier(prefix, by_kind.get("nuget", []))
    forges = ("github", "gitlab", "gitea")
    for driver in [d for kind in forges for d in by_kind.get(kind, [])]:
        outcomes.append(_release(prefix, driver, host=driver.provision.kind))
    outcomes += _docker_tier(prefix, by_kind.get("docker", []))
    outcomes += _man_tier(prefix, by_kind.get("man", []))
    return outcomes

provisioned_node

provisioned_node(prefix: Path) -> Path | None

The node the nodejs tier unpacked under prefix, or None before it ran.

Source code in packages/toolroom-bench/src/livery/toolroom/bench/_provision.py
def provisioned_node(prefix: Path) -> Path | None:
    """The node the nodejs tier unpacked under *prefix*, or None before it ran."""
    root = prefix / ".nodejs" / "node"
    if not root.is_dir():
        return None
    for tree in sorted(root.iterdir()):
        node = tree / exe("node") if os.name == "nt" else tree / "bin" / exe("node")
        if node.is_file():
            return node
    return None

provisioned_dotnet

provisioned_dotnet(prefix: Path) -> Path | None

The dotnet the dotnet tier unpacked under prefix, or None before it ran.

Source code in packages/toolroom-bench/src/livery/toolroom/bench/_provision.py
def provisioned_dotnet(prefix: Path) -> Path | None:
    """The dotnet the dotnet tier unpacked under *prefix*, or None before it ran."""
    root = prefix / ".dotnet" / "dotnet"
    if not root.is_dir():
        return None
    for tree in sorted(root.iterdir()):
        muxer = tree / exe("dotnet")
        if muxer.is_file():
            return muxer
    return None

assets_for

assets_for(host: str, repo: str, tag: str = '') -> list[tuple[str, str]]

[(asset name, download url)] for one release — latest when tag is empty, and a specific tag when priming a tool's history.

Source code in packages/toolroom-bench/src/livery/toolroom/bench/_provision.py
def assets_for(host: str, repo: str, tag: str = "") -> list[tuple[str, str]]:
    """`[(asset name, download url)]` for one release — latest when *tag* is
    empty, and a specific tag when priming a tool's history.
    """
    if not repo:
        raise ProvisionError("no repo to fetch from")
    if host == "github":
        where = f"tags/{tag}" if tag else "latest"
        data = _get_json(f"https://api.github.com/repos/{repo}/releases/{where}")
        assets = data.get("assets", [])
        return [(a["name"], a["browser_download_url"]) for a in assets]
    if host == "gitlab":
        quoted = urllib.parse.quote(repo, safe="")
        where = urllib.parse.quote(tag, safe="") if tag else "permalink/latest"
        data = _get_json(
            f"https://gitlab.com/api/v4/projects/{quoted}/releases/{where}"
        )
        links = data.get("assets", {}).get("links", [])
        return [(a["name"], a.get("direct_asset_url") or a["url"]) for a in links]
    if host == "gitea":
        # GitHub-shaped: same endpoints, same asset fields, gitea.com base.
        where = f"tags/{tag}" if tag else "latest"
        data = _get_json(f"https://gitea.com/api/v1/repos/{repo}/releases/{where}")
        assets = data.get("assets", [])
        return [(a["name"], a["browser_download_url"]) for a in assets]
    raise ProvisionError(f"unknown release host {host!r}")

place_binary

place_binary(archive: Path, tool: str, into: Path, *, windows: bool | None = None, launcher: bool = True) -> Path

Unpack archive whole and put its tool in into; the file placed there.

An archive is unpacked as it is, into a trees/<tool> directory beside into, and the binary runs where it lies, so an app that loads its runtime from beside itself (a PyInstaller directory app, conan's release) finds it. What lands in into is a launcher that runs the binary in its tree: a shell script, or a .cmd on Windows where shutil.which finds it through PATHEXT. With launcher off the binary itself is copied into into, for a place that must hold the real file (a docker plugin directory). A downloaded bare binary is placed as it is either way. The binary is the first file member named tool or tool.exe, so a directory of the same name (docker's docker/docker) never matches.

Raises:

Type Description
ProvisionError

when the archive will not unpack or holds no such file.

Source code in packages/toolroom-bench/src/livery/toolroom/bench/_provision.py
def place_binary(
    archive: Path,
    tool: str,
    into: Path,
    *,
    windows: bool | None = None,
    launcher: bool = True,
) -> Path:
    """Unpack *archive* whole and put its `tool` in *into*; the file placed there.

    An archive is unpacked as it is, into a `trees/<tool>` directory
    beside *into*, and the binary runs where it lies, so an app that
    loads its runtime from beside itself (a PyInstaller directory app,
    conan's release) finds it. What lands in *into* is a launcher that
    runs the binary in its tree: a shell script, or a `.cmd` on Windows
    where `shutil.which` finds it through `PATHEXT`. With *launcher*
    off the binary itself is copied into *into*, for a place that must
    hold the real file (a docker plugin directory). A downloaded bare
    binary is placed as it is either way. The binary is the first file
    member named `tool` or `tool.exe`, so a directory of the same name
    (docker's `docker/docker`) never matches.

    Raises:
        ProvisionError: when the archive will not unpack or holds no
            such file.
    """
    from livery.toolroom.store import ARCHIVE_SUFFIXES, UnpackError, unpack

    if windows is None:
        windows = os.name == "nt"
    into.mkdir(parents=True, exist_ok=True)
    if not archive.name.lower().endswith(
        ARCHIVE_SUFFIXES
    ):  # a bare binary, downloaded directly
        dest = into / exe(tool, windows=windows)
        dest.write_bytes(archive.read_bytes())
        dest.chmod(0o755)
        return dest
    tree = into.parent / "trees" / tool
    shutil.rmtree(tree, ignore_errors=True)
    try:
        unpack(archive, tree)
    except UnpackError as exc:
        raise ProvisionError(str(exc)) from exc
    found = _find_binary(tree, tool)
    if found is None:
        raise ProvisionError(f"{tool} not found inside {archive.name}")
    found.chmod(found.stat().st_mode | 0o755)
    if not launcher:
        dest = into / exe(tool, windows=windows)
        shutil.copy2(found, dest)
        return dest
    return write_launcher(into, tool, found, windows=windows)

write_launcher

write_launcher(into: Path, name: str, target: Path, *args: str, windows: bool | None = None) -> Path

A launcher in into that runs target with args first; its path.

A shell script that execs the target, so the process is the target itself and finds what sits beside it; on Windows a .cmd, which cmd resolves through PATHEXT and forwards %* to whole.

Source code in packages/toolroom-bench/src/livery/toolroom/bench/_provision.py
def write_launcher(
    into: Path, name: str, target: Path, *args: str, windows: bool | None = None
) -> Path:
    """A launcher in *into* that runs *target* with *args* first; its path.

    A shell script that `exec`s the target, so the process is the target
    itself and finds what sits beside it; on Windows a `.cmd`, which
    `cmd` resolves through `PATHEXT` and forwards `%*` to whole.
    """
    import stat

    if windows is None:
        windows = os.name == "nt"
    into.mkdir(parents=True, exist_ok=True)
    lead = "".join(f" {arg}" for arg in args)
    if windows:
        shim = into / f"{name}.cmd"
        shim.write_text(f'@echo off\r\n"{target}"{lead} %*\r\n', encoding="utf-8")
        return shim
    shim = into / name
    shim.write_text(f'#!/bin/sh\nexec "{target}"{lead} "$@"\n', encoding="utf-8")
    shim.chmod(shim.stat().st_mode | stat.S_IXUSR | stat.S_IXGRP | stat.S_IXOTH)
    return shim