Skip to content

livery.workshop._gate_record

livery.workshop._gate_record

The local gate record: the chain of trees this checkout's green gates proved.

Every green local gate writes one row: the tree it proved, and the proved tree it started from. A full gate proves a tree outright. A narrowed gate proves the packages a delta can influence, so its row names the tree the delta was taken from, and the tree is proved when that one is: a package the delta did not touch keeps the earlier proof, since affected is closed under dependents. A chain of such rows is a proof of its newest tree once it reaches a root, a full row or a tree CI's own workshop/verified record holds, which is what the merge base with the base branch is after a merge.

Two readers use the chain. The reflex, fm check on a machine, looks for the nearest proved tree in the checkout's history and gates the delta from it to the working tree, so the tenth commit of a branch pays for what the tenth commit touched; a working tree the chain already proves pays nothing. fm submit walks the chain from HEAD's tree and skips its gate when the chain reaches a root, saying so. The rows key the working tree's id, which a commit that takes everything shares, so a check before the commit proves the commit.

A CI check leg chooses where to measure from by the same rule (leg_plan), over CI's own record alone, read for the leg's own platform.

The record is a local series of the state store (livery.git): it lives in the checkout's git directory, shared by its worktrees, is written only by a green local gate, and never leaves the machine. CI never reads or writes it. The window keeps the newest KEEP rows, and a row older than MAX_AGE proves nothing, since the tree id covers the pins in uv.lock but not the machine's Python or tools, which drift under the same tree.

Classes:

Name Description
Row

One proved tree.

Chain

A proof of one tree: its steps back to a root.

Plan

What the reflex runs for the working tree.

Functions:

Name Description
rows

Every row, newest first, or ((), reason) when the record cannot be read.

remember

Record tree as proved green by the gate that just ran; the line to print.

chain

The chain proving tree, or None when the rows do not reach a root.

verified_roots

The trees CI's record holds that a chain may rest on, and why none when none.

covering

The chain proving HEAD's tree for the gate a submit would run, or why not.

plan

What the reflex runs for the working tree: nothing, a step, or everything.

leg_plan

Where a CI check leg measures its change from; the reflex's rule on CI's record.

Attributes:

Name Type Description
KEEP
MAX_AGE
REACH
NEAR
MERGES
SERIES
FULL
STEP
MERGE_BASE

KEEP module-attribute

KEEP = 200

MAX_AGE module-attribute

MAX_AGE = timedelta(days=7)

REACH module-attribute

REACH = 50

NEAR module-attribute

NEAR = 20

MERGES module-attribute

MERGES = 5

SERIES module-attribute

SERIES = git.Series('gate-record', window=KEEP, ci_only=False, local=True)

FULL module-attribute

FULL = 'full'

STEP module-attribute

STEP = 'step'

MERGE_BASE module-attribute

MERGE_BASE = 'merge-base'

Row dataclass

Row(tree: str, scope: str, packages: tuple[str, ...], base_tree: str, when: str, root: str)

One proved tree.

Attributes:

Name Type Description
tree str

The git tree id the gate proved.

scope str

full, or step for a gate of the delta from base_tree.

packages tuple[str, ...]

The packages a step gated; empty for full, and for a step whose delta reached no package.

base_tree str

The proved tree a step's delta was taken from; empty for full.

when str

When the store wrote the row, ISO 8601 in UTC.

root str

The checkout the gate ran in, for a reader.

tree instance-attribute

tree: str

scope instance-attribute

scope: str

packages instance-attribute

packages: tuple[str, ...]

base_tree instance-attribute

base_tree: str

when instance-attribute

when: str

root instance-attribute

root: str

Chain dataclass

Chain(tree: str, steps: tuple[Row, ...], root: str, root_tree: str)

A proof of one tree: its steps back to a root.

Attributes:

Name Type Description
tree str

The tree the chain proves.

steps tuple[Row, ...]

The rows walked, newest first; empty when CI's record holds the tree itself.

root str

full when the chain ends on a full row, verified when it rests on a tree CI's record holds.

root_tree str

The tree the chain rests on.

Methods:

Name Description
describe

One clause a line can carry: the steps and the root.

tree instance-attribute

tree: str

steps instance-attribute

steps: tuple[Row, ...]

root instance-attribute

root: str

root_tree instance-attribute

root_tree: str

describe

describe() -> str

One clause a line can carry: the steps and the root.

Source code in packages/workshop/src/livery/workshop/_gate_record.py
def describe(self) -> str:
    """One clause a line can carry: the steps and the root."""
    rooted = (
        f"a full gate of {self.root_tree[:12]}"
        if self.root == FULL
        else f"CI's record of {self.root_tree[:12]}"
    )
    count = len(self.steps)
    if self.root == FULL:
        count -= 1
    steps = f"{count} step(s) on " if count else ""
    return f"{steps}{rooted}"

Plan dataclass

Plan(mode: str, tree: str, base_tree: str = '', paths: tuple[str, ...] = (), why: str = '')

What the reflex runs for the working tree.

Attributes:

Name Type Description
mode str

proved when the chain already proves the working tree, step when a proved tree is in reach and the delta from it is what runs, full when none is. A CI leg's plan is merge-base (MERGE_BASE) instead of full while git has a merge base with the base branch.

tree str

The working tree's id, the row a green run records.

base_tree str

The tree a step builds on, proved but for a merge-base step; empty otherwise.

paths tuple[str, ...]

The paths a step's delta touches.

why str

The reason, for the line the gate prints.

mode instance-attribute

mode: str

tree instance-attribute

tree: str

base_tree class-attribute instance-attribute

base_tree: str = ''

paths class-attribute instance-attribute

paths: tuple[str, ...] = ()

why class-attribute instance-attribute

why: str = ''

rows

rows(root: Path) -> tuple[tuple[Row, ...], str]

Every row, newest first, or ((), reason) when the record cannot be read.

A row without a tree is not a proof and is left out; the store already skipped what does not parse or is of another schema.

Source code in packages/workshop/src/livery/workshop/_gate_record.py
def rows(root: Path) -> tuple[tuple[Row, ...], str]:
    """Every row, newest first, or ``((), reason)`` when the record cannot be read.

    A row without a tree is not a proof and is left out; the store
    already skipped what does not parse or is of another schema.
    """
    found = SERIES.rows(state_store(root))
    if found.failed:
        return (), found.reason
    kept: list[Row] = []
    for item in found.rows:
        tree = item.data.get("tree")
        if not isinstance(tree, str):
            continue
        kept.append(
            Row(
                tree=tree,
                scope=str(item.data.get("scope", "")),
                packages=tuple(str(p) for p in item.data.get("packages", []) or []),
                base_tree=str(item.data.get("base_tree", "")),
                when=item.when,
                root=str(item.data.get("root", "")),
            )
        )
    return tuple(kept), ""

remember

remember(root: Path, *, tree: str, packages: tuple[str, ...] | None, base_tree: str = '') -> str

Record tree as proved green by the gate that just ran; the line to print.

packages is what a step gated, with base_tree the proved tree its delta was taken from; None records a full gate. A row of the same tree, scope, and base replaces its predecessor; the window keeps the newest KEEP rows.

Never raises. The record only lets a later gate run less, so a row that is not written costs that gate time and changes no verdict: a write that fails, by its returned reason or by raising, comes back as the line that says so, and the gate that called it stays green.

Source code in packages/workshop/src/livery/workshop/_gate_record.py
def remember(
    root: Path,
    *,
    tree: str,
    packages: tuple[str, ...] | None,
    base_tree: str = "",
) -> str:
    """Record *tree* as proved green by the gate that just ran; the line to print.

    *packages* is what a step gated, with *base_tree* the proved tree
    its delta was taken from; ``None`` records a full gate. A row of
    the same tree, scope, and base replaces its predecessor; the
    window keeps the newest `KEEP` rows.

    Never raises. The record only lets a later gate run less, so a
    row that is not written costs that gate time and changes no
    verdict: a write that fails, by its returned reason or by raising,
    comes back as the line that says so, and the gate that called it
    stays green.
    """
    scope = FULL if packages is None else STEP
    base = base_tree if packages is not None else ""
    row = {
        "tree": tree,
        "scope": scope,
        "packages": list(packages or ()),
        "base_tree": base,
        "root": str(root),
    }
    try:
        why = SERIES.put(
            state_store(root),
            {_name(tree, scope, base): row},
            message=f"gate record: tree {tree[:12]} proved green ({scope})",
        )
    except Exception as error:
        why = str(error) or type(error).__name__
    if why:
        return (
            f"  gate record: not recorded ({why}); the next gate measures from"
            " the last recorded tree"
        )
    return f"  gate record: tree {tree[:12]} recorded as proved green ({scope})"

chain

chain(root: Path, tree: str, *, roots: set[str], now: datetime | None = None, found: tuple[Row, ...] | None = None) -> Chain | None

The chain proving tree, or None when the rows do not reach a root.

roots are the trees CI's record holds. Rows older than MAX_AGE do not count. A cycle or a base no row proves ends the walk empty. found are the rows already read, for a caller walking many trees; the record is read once here otherwise.

Source code in packages/workshop/src/livery/workshop/_gate_record.py
def chain(
    root: Path,
    tree: str,
    *,
    roots: set[str],
    now: datetime | None = None,
    found: tuple[Row, ...] | None = None,
) -> Chain | None:
    """The chain proving *tree*, or ``None`` when the rows do not reach a root.

    *roots* are the trees CI's record holds. Rows older than `MAX_AGE`
    do not count. A cycle or a base no row proves ends the walk empty. *found* are the
    rows already read, for a caller walking many trees; the record is
    read once here otherwise.
    """
    if found is None:
        found, why = rows(root)
        if why:
            return None
    moment = now or datetime.now(UTC)
    by_tree: dict[str, list[Row]] = {}
    for row in found:
        if _fresh(row, moment):
            by_tree.setdefault(row.tree, []).append(row)
    steps: list[Row] = []
    current = tree
    seen: set[str] = set()
    while current not in seen:
        seen.add(current)
        candidates = by_tree.get(current, [])
        full = [row for row in candidates if row.scope == FULL]
        if full:
            steps.append(max(full, key=lambda row: row.when))
            return Chain(tree, tuple(steps), FULL, current)
        if current in roots:
            return Chain(tree, tuple(steps), "verified", current)
        links = [row for row in candidates if row.scope == STEP and row.base_tree]
        if not links:
            return None
        step = max(links, key=lambda row: row.when)
        steps.append(step)
        current = step.base_tree
    return None

verified_roots

verified_roots(root: Path, base: str) -> tuple[set[str], str]

The trees CI's record holds that a chain may rest on, and why none when none.

The candidates are the merge base with base and HEAD's first-parent history, REACH commits deep. After a merge, main's run stamps its tree, so a branch cut from it starts proved; while that run is still red or unfinished, the parent's tree carries the chain and the merge's own changes ride in the first step. A record the forge could not answer leaves the chain rootless, and the reason is returned for the line.

Source code in packages/workshop/src/livery/workshop/_gate_record.py
def verified_roots(root: Path, base: str) -> tuple[set[str], str]:
    """The trees CI's record holds that a chain may rest on, and why none when none.

    The candidates are the merge base with *base* and HEAD's
    first-parent history, `REACH` commits deep. After a merge, main's
    run stamps its tree, so a branch cut from it starts proved; while
    that run is still red or unfinished, the parent's tree carries the
    chain and the merge's own changes ride in the first step. A record
    the forge could not answer leaves the chain rootless, and the
    reason is returned for the line.
    """
    from livery.git import remote_snapshot
    from livery.workshop import _verified

    try:
        base_tree = tree_id(root, git.merge_base(root, base))
    except git.GitError as error:
        return set(), f"no merge base with origin/{base} ({error})"
    history = git.first_parent_trees(root, REACH)
    with remote_snapshot(state_store(root), fetch=("verified",)):
        roots, why = _verified.held(root, [base_tree, *history])
    if roots:
        return roots, ""
    if why:
        return set(), f"CI's record could not be read ({why})"
    return set(), (
        f"CI's record holds neither the merge base's tree {base_tree[:12]}"
        f" nor one of HEAD's last {len(history)} first-parent trees"
    )

covering

covering(root: Path, *, base: str = 'main', now: datetime | None = None) -> tuple[Chain | None, str]

The chain proving HEAD's tree for the gate a submit would run, or why not.

Source code in packages/workshop/src/livery/workshop/_gate_record.py
def covering(
    root: Path, *, base: str = "main", now: datetime | None = None
) -> tuple[Chain | None, str]:
    """The chain proving HEAD's tree for the gate a submit would run, or why not."""
    try:
        tree = tree_id(root)
    except git.GitError as error:
        return None, f"no tree id ({error})"
    roots, why = verified_roots(root, base)
    proof = chain(root, tree, roots=roots, now=now)
    if proof is not None:
        return proof, ""
    reason = f"no chain of green gates reaches tree {tree[:12]}"
    return None, f"{reason} ({why})" if why else reason

plan

plan(root: Path, *, base: str = 'main', now: datetime | None = None) -> Plan

What the reflex runs for the working tree: nothing, a step, or everything.

The working tree's own chain settles it first. Else the proved tree with the fewest paths changed to the working tree becomes the step's base, chosen among HEAD's first-parent history, REACH commits deep, and the trees of the newest NEAR rows, so a green check of a dirty tree is the base of the next check after one more edit; a row's tree git no longer has is passed over. With none in reach the whole gate runs and roots a new chain.

Source code in packages/workshop/src/livery/workshop/_gate_record.py
def plan(root: Path, *, base: str = "main", now: datetime | None = None) -> Plan:
    """What the reflex runs for the working tree: nothing, a step, or everything.

    The working tree's own chain settles it first. Else the proved
    tree with the fewest paths changed to the working tree becomes the
    step's base, chosen among HEAD's first-parent history, `REACH`
    commits deep, and the trees of the newest `NEAR` rows, so a green
    check of a dirty tree is the base of the next check after one more
    edit; a row's tree git no longer has is passed over. With none in
    reach the whole gate runs and roots a new chain.
    """
    tree = git.working_tree_id(root)
    roots, why = verified_roots(root, base)
    found, unread = rows(root)
    moment = now or datetime.now(UTC)
    proof = chain(root, tree, roots=roots, now=now, found=found)
    if proof is not None:
        return Plan("proved", tree, why=proof.describe())
    near = list(dict.fromkeys(row.tree for row in found if _fresh(row, moment)))
    best = _fewest(
        root,
        tree,
        [*git.first_parent_trees(root, REACH), *near[:NEAR]],
        lambda candidate: (
            chain(root, candidate, roots=roots, now=now, found=found) is not None
        ),
    )
    if best is not None:
        return Plan("step", tree, base_tree=best[0], paths=best[1])
    return Plan("full", tree, why=why or unread or "no proved tree in reach")

leg_plan

leg_plan(root: Path, *, base: str, branch: str, leg: str = '', before: datetime | None = None) -> Plan

Where a CI check leg measures its change from; the reflex's rule on CI's record.

The candidates are the merge base with base, HEAD's first-parent history REACH commits deep, and, on a merge checkout, the merges branch's earlier runs proved. A pull request's run on GitHub tests the merge of its head onto the base branch, a commit no later clone holds, so its row names the two commits it merged; the merge is rebuilt from them, MERGES rows deep, and counts only when it yields the row's tree, which also puts that tree in this clone for every reader of the step's base. Of the candidates the record proves in full for leg's platform (any full row when leg is empty), the one with the fewest paths changed to HEAD's tree is the step's base: the earlier push's merge, when the base branch has not moved far, or the base branch's tip. With none, or a record that cannot be read, the step is taken from the merge base (MERGE_BASE), and why says so, naming the leg. Without a merge base the mode is full. With before, the record's rows count only when the store stamped them before it (livery.workshop._verified.written_before), so the shards of a leg, which share the bound, measure from one tree.

Source code in packages/workshop/src/livery/workshop/_gate_record.py
def leg_plan(
    root: Path,
    *,
    base: str,
    branch: str,
    leg: str = "",
    before: datetime | None = None,
) -> Plan:
    """Where a CI check leg measures its change from; the reflex's rule on CI's record.

    The candidates are the merge base with *base*, HEAD's first-parent
    history `REACH` commits deep, and, on a merge checkout, the merges
    *branch*'s earlier runs proved. A pull request's run on GitHub
    tests the merge of its head onto the base branch, a commit no later
    clone holds, so its row names the two commits it merged; the merge
    is rebuilt from them, `MERGES` rows deep, and counts only when it
    yields the row's tree, which also puts that tree in this clone for
    every reader of the step's base. Of the candidates the record
    proves in full for *leg*'s platform (any full row when *leg* is
    empty), the one with the fewest paths changed to HEAD's tree is
    the step's base: the earlier push's merge, when the base branch
    has not moved far, or the base branch's tip. With none, or a
    record that cannot be read, the step is taken from the merge base
    (`MERGE_BASE`), and *why* says so, naming the leg. Without a merge
    base the mode is ``full``. With *before*, the record's rows count
    only when the store stamped them before it
    ([livery.workshop._verified.written_before][]), so the shards of a
    leg, which share the bound, measure from one tree.
    """
    from livery.git import remote_snapshot
    from livery.workshop import _verified

    tree = tree_id(root)
    try:
        merge_base = tree_id(root, git.merge_base(root, base))
    except git.GitError as error:
        return Plan("full", tree, why=f"no merge base with origin/{base} ({error})")
    try:
        merged = git.merge_parents(root)
    except git.GitError:
        merged = None
    with remote_snapshot(state_store(root), fetch=("verified",)):
        rows, unread = _verified.proved(root, leg, before=before)
    candidates = [merge_base, *git.first_parent_trees(root, REACH)]
    rebuilt = _rebuilt(root, rows, branch) if merged is not None else []
    best = _fewest(
        root, tree, [*candidates, *rebuilt], lambda candidate: candidate in rows
    )
    if best is not None:
        base_tree, paths = best
        row = rows[base_tree]
        why = (
            f"run {row.run}'s merge of {row.head_sha[:12]} onto {row.base_sha[:12]}"
            if base_tree in rebuilt and base_tree not in candidates
            else f"proved by run {row.run}"
        )
        return Plan("step", tree, base_tree=base_tree, paths=paths, why=why)
    for_leg = f" for {leg}" if leg else ""
    why = (
        f"CI's record could not be read ({unread})"
        if unread
        else f"CI's record proves no tree in reach{for_leg}"
    )
    try:
        paths = tuple(git.tree_diff(root, merge_base, tree))
    except git.GitError as error:
        return Plan(
            "full", tree, why=f"{why}, and the merge base's diff failed ({error})"
        )
    return Plan(MERGE_BASE, tree, base_tree=merge_base, paths=paths, why=why)