Skip to content

toolroom store

The pinned tool store over strongroom. A record, one directory per tool, names the tool, every version tracked, per version and host the artifact's URL and its sha256 with the deployment resolved through the record's layers, and per version the surface its command line accepts; the store lands the artifact by that digest, extracts it, collects it as a tree, and names the tree under tools/<name>@<version> in one strongroom store per machine, shared by every checkout on it.

The store installs, links, emits and fetches:

from pathlib import Path

from livery.strongroom import FolderSource
from livery.toolroom.store import Home, Record, Store, records_in

store = Store(
    Home(Path.home() / ".local/share/toolroom"),
    sources=[FolderSource(Path("/mirrors/tools/store"))],
)
records = [Record.load(path) for path in records_in(Path("records"))]
ensured = [store.ensure(record, record.versions[-1]) for record in records]
store.link(ensured, Path(".workshop/bin"))
delta = store.delta(ensured, Path(".workshop/bin"))

The record

A record is one file of JSON lines, records/<tool>.jsonl, named by the tool. The first line is the tool axis: the name, the description, the kind, the version floor a system-check tool must reach, prime, the oldest version the reading history reaches, the hosts the tool has, the tool's layout and its override for one host each. A version line is one version's arrival: the version, its date, per host the artifact's URL and sha256, the version's override of the layout, for every host or for one, and read, the platforms and the extractor that read it and the tool's description when it changed. The statement lines under a version line are what its reading changed: one option per line with the option's fields whole, one line for a verb's own fields when they moved, one line for an option or a verb withdrawn (gone), and one line per absence, the platforms that read the version and did not find the option. Versions run in file order, a release appends lines, and the review diff is the options that moved. An option unchanged since the previous reading has no line.

A deployment resolves through four layers, most specific winning: the tool's layout, the tool's override for the host, the version's layout, the version's override for the host. The layout fields are root, the directory inside the archive hoisted to the install root; file, the name a bare download (one file, no archive) is saved under; format, what the artifact is (zip, tar or file) when its URL's suffix would sniff wrong; entry_points, the install-relative paths of the executables the deployment puts on PATH, annotated here and never discovered by scanning a directory; paths, the install-relative directories put on PATH; env, with $package standing for the install root, where a tool's own switches belong (an update check, a telemetry opt-out), since entering the environment exports them and every spawn of the tool inherits them; shims, a link name to an executable the install carries; and exclude, the archive members left out before import, fnmatch patterns over the install-relative path.

The surface

A delta may carry a surface: what the version's command line accepts, as it was read, and who read it. The reading is one verb at a time. verbs names the verbs this version changed, each whole, with help, wraps, positional, lead and its options, and an option carries flags, negation, help, type, default and choices; a verb set to null is withdrawn, and a verb not named is inherited from the nearest earlier version that has a surface. The tool's own options hang off the verb named "". help, the tool's description, is set when it changed and inherited otherwise; the record's first surface sets it. platforms names who read the version, from Linux, macOS and Windows, and extractor the generation of the reader; neither is inherited. absent records, per verb and option, the platforms that read the version and did not find that option, the option named "" standing for the verb itself; it names only platforms among platforms. A version whose delta carries no surface was never read, and surface_at answers None for it rather than inheriting a reading. A version with a surface and no artifact is tracked for its surface alone: it has no host and nothing installs it.

surface_at resolves one version whole, every verb in name order with its options in name order, and observations resolves every version that has a surface, in sequence.

The catalogue and the lock

A consumer resolves against the catalogue, never against a record. The catalogue lists every tool with its versions in order, per version the hosts it has an artifact for with the digest of each host's deployment, and per version read its surface, from which the store renders that version's stub. It is read on demand, the pointer first, a tool's tree when the tool is asked for and a version's hosts when a lock needs them, from the published index, by URL or from a directory holding one, through the machine's store, which keeps what it fetched so a second read is offline; the authoring site reads its records directly and gets the same catalogue, since a deployment's digest is the digest of its canonical JSON either way.

Every site that names a tool or an extension spells it in one grammar, name[option,...]?>=floor@scope, and Spec.parse is its one parser: [...] names options (a tool's profile, an extension's switches), ? marks it optional, >=floor is the lowest version that satisfies, and @scope limits it to some hosts. A scope is comma-separated platforms (windows, every host of it) or host keys (windows-x64), each excluded with a leading !; any other token refuses at parse. Scope.hosts(supported) resolves a scope against the hosts a workspace supports: no inclusion reaches every one of them, and an exclusion then removes its hosts, so tea@!windows-arm is every supported host but one. The parser enforces the grammar alone; what a site takes of it is the site's rule, refused there with the site named.

A tool requirement is a tool's name with a floor, ruff or ruff>=0.16, and may name the hosts it applies to after @, dotnet_coverage@windows or tea>=1.1@linux,macos-arm. The lock takes for each tool the newest version the catalogue lists that satisfies every floor and resolves on every host the tool is required on, the union of its requirements' scopes and the whole lock for one with no scope: a downloaded kind resolves on a host when it has that host's artifact, a delegated kind everywhere its installer does. A requirement that cannot be met refuses naming the tool, each floor with the site that declared it, and for a host no eligible version has, the first version that has it. The lock is toolroom.lock at the repository root, one version per tool with the deployment digest per host it is locked on; an entry locked on fewer hosts than the lock's names them in on, a scope no locked host matches locks nothing, and an entry stands while it still satisfies and resolves, moving only when asked. The graphs of the delegated tools, one resolved graph per tool, sit beside it in toolroom.graphs/.

The [toolroom] table

A directory's tool configuration is one table, [toolroom]: the catalogue (index), the requirements (requires), the mirrors (sources), the modes and the host allowances. A workspace the workshop manages keeps it in workshop.toml; a directory without one keeps it in toolroom.toml, under the same name, and livery.toolroom.store.read_table reads it from whichever the directory has, workshop.toml first:

[toolroom]
index = "https://example.org/tools/index"
requires = ["ruff>=0.16", "git"]

The table's schema ships with the store, livery.toolroom.store.SCHEMA_FRAGMENT, as the keys it declares in a workspace's root contract, a package's contract and an extension's extension.toml; the workshop composes it into every workspace's schema, so the editor completes the table and the workshop's judge refuses a key it does not declare.

What a load refuses

Loading a record resolves every host of every version and every version's surface, and refuses a record that does not resolve whole: an artifact without a sha256, a layer naming a host or a version the record does not carry, a layer restating the value it inherits, a version whose host resolves with no paths, a download that nothing reaches (no paths, no entry point, no env value under $package), a path directory with no entry point or an entry point with no paths, a bare download with no file or whose entry points are not that file alone, a version with neither an artifact nor a surface, a surface read on no platform, a surface restating the help or a verb it inherits, a withdrawn verb no earlier version has, an absence naming a verb or option the version lacks or a platform that did not read it, and a delta out of sequence. The schema of both documents is records/record.schema.json, exported by export_schema.

The store

ensure lands a version's artifact through the sources in order and through the origin URL unless the store is offline; a mismatch at any tier is refused naming the tier, and an offline miss names <name>@<version> and the origin. An archive is extracted and its root hoisted, a bare download saved as its file, the excluded members removed, the shims made, every declared entry point checked for in the tree, which refuses naming the tool, the version, the host and the path when one is absent, the directory collected as a tree with the entry points as its executables and nothing else, tools/<name>@<version> moved to it with the deployment that produced it recorded beside the ref, and the tree viewed at the home's tool directory. A later ensure of the same version compares the record's deployment with the one beside the ref: a changed layout is a new tree from the same artifact, the ref moves to it by compare-and-swap, the old view is dropped and the new tree viewed, and the old tree is swept as unreached. The artifact itself never changes under a version, since landing verifies it. One archive lands one tree digest on every platform by construction: the annotation decides the executable bit, not the modes the extractor happened to produce. A second ensure is a probe that answers offline. A directory the store did not make is never removed.

One process at a time supplies a version. The supply holds the version's lock, locks/<name>@<version>.lock under the home, from its probe to its view. A second process that asks for the same version waits, says so through the progress (wait), and then finds the version present. The operating system holds the lock for the open file and lets go when the holder ends, so a holder that died leaves nothing to clear and a waiter has no timeout.

link fills a bin directory with one link per declared entry point, a launcher where the platform refuses a link, and removes only links it made; delta is the one PATH prepend and the env with $package replaced by the tool directory. fetch lands every version's artifact for the hosts asked into a store at another root, a mirror by construction, for an offline install elsewhere.

supply is the primitive ensure stands on: a tool by name, kind, version and deployment, which is what a consumer holds after reading the catalogue. The delegated kinds go through their installer rather than the objects. A pypi tool is installed by uv at the locked version into a directory of its own under the home, the record's package naming what uv installs when it differs from the tool's name, and its launchers under bin are its entry points. An npm tool is installed the same way through the runtime its record names, node unless it says bun, whose executable the caller supplied first: npm run on node, or bun's own installer. Its launchers under bin are the entry points, they start with #!/usr/bin/env node, and the runtime's directory stays on PATH to answer that, bun through the node shim its record declares. A dotnet tool is installed through the dotnet SDK, the runtime its record names, with dotnet tool install --tool-path into a directory of its own under the home, the record's package naming the NuGet package; the shims the SDK writes under bin are its entry points, and they find the runtime through DOTNET_ROOT, which the SDK's own record sets. It has no graph: a .NET tool package carries its dependencies, and the version pins the whole. A launcher the runtime placed outside the tool's directory is refused naming where it points. A system-check tool is the machine's own, found on PATH and held to the record's min_version alone, since its locked version is the newest reading the stubs render for and not a version anyone installs, and the store installs nothing for it. python refuses naming the kind until it is supplied the same way. A delegated tool has no tree.

A record's mode says how a materialised tool reaches PATH: link puts its entry points in the checkout's bin directory, path its own directories on PATH, none neither, for a tool reached only through a typed handle; a download with paths takes path, one reached through its env alone and a system tool take none, unless the record says otherwise, and every installer's kind takes path.

A version or host the record does not carry refuses by name, naming what it does carry. The six host keys are macos-arm, macos-x64, linux-x64, linux-arm, windows-x64 and windows-arm.

A view is a materialisation of a tree the object store keeps, and the store keeps a record of every view it made: the tree, the directory and every path. Store.probe answers present from the ref and that record, without walking the files, so a sync pays no walk. Store.audit(name, version) walks one view and names the paths the disk lacks; Store.audit_views() does so for every view under the home; Store.mend_view(name, version) drops a damaged view's record and views its tree again, fetching nothing. fm doctor runs the store's doctor, registered under footman's footman.doctors group, which reports every damaged view and mends it under --fix.