Skip to content

livery.extensions.coverage._record

livery.extensions.coverage._record

The coverage record: main's and each branch's parts per leg, and the parts in flight.

Coverage stays global under the affected mode. A record holds, per check leg, one row per unit (a package's test suite, or the workspace's own tests): the unit's parts in the line format, one per measurer that measured the suite, each holding the files of the suite's closure it reached, and the identity of the closure it was measured at. Main has a record, and so does every branch with a pull request run, under the same family with the branch as the base part. A check leg skips a suite when its branch's record or main's holds the unit at the suite's current closure identity, and runs it otherwise. A leg that ran a suite leaves the suite's parts, with the scope its gate ran, for the leg's one write, which puts them on its per-run ref beside its timing half; the run's gate job unions the per-run refs with the rows it carries from the records, judges the floors on that union, and writes the union back: a pull request's run onto its branch's record, main's run onto main's, each replaced in place, the units that no longer exist removed. A row a write replaces at another closure, or removes, stays for a day under the moment it was kept and its closure's name: a run whose leg skipped a suite on it, while another run moved the record on, still unions it. At the merge main's run finds its tree on the verified record, which names the branch, and copies the branch's rows into main's without measuring; a squash of a stale branch has another tree, and main pays the full gate. A unit neither record holds is measured on the spot, never red.

The unit is the suite that ran, never the package covered: a suite executes lines across packages (a dependant's tests run its dependencies), so a unit's parts hold the files of the suite's closure alone, the files whose identity the key names. A row keeps its parts in the line format's stored form: what ran, each file's fixed facts named by their own digest. The facts are files of their own beside the rows (FACTS), each kept once however many rows, legs and writes name it, and a write drops the facts no row names any more. A python suite's row names the tests that reached each line, and its numbering of them lives in the row: each write numbers them as the row it replaces did, so a test keeps its index from run to run, a new test takes the next, and a test that is gone keeps its index, reached by no line (the line format's numbering). Two rows of a unit a run apart then keep a line's set of tests byte for byte wherever the same tests reached it. One run's writes append the same tests on every leg (run_contexts), so a test has one index in its unit's rows on every leg, and the legs' rows keep a line's set alike wherever the same tests reached it. Reach for recorded to read a record on a leg, leave_run to leave a leg's parts for its write, fold_leg to fold the files a platform's shards left into the platform leg's one, run_legs to read them in the gate job, and put_record to write the record. A read of a record parses its current rows and the facts they name alone, and names its kept rows, reading one only when the union asks for it.

Classes:

Name Description
Unit

One unit's measurement: the parts its suite's measurers made within its closure.

Leg

What one check leg put on its per-run ref.

Kept

A row a write replaced at another closure, or removed, kept for a day.

Record

A base's record on one leg, as read.

Functions:

Name Description
branches

The branches origin holds, by name; None if origin cannot be listed.

current_keys

The keys the world produces: main and every branch on origin, per check leg.

facts_file

The record's file for the fixed facts named name.

kept_name

The record's file for a kept row: the unit at path, measured at closure.

record_ref

The ref holding base's record on leg: main's, or a branch's.

record_bases

The records a leg reads, in order: the branch's own if any, then main's.

record_prefixes

The refs of the records a run reads, as prefixes: branch's if any, then main's.

series

Answer store.series: the coverage record and the coverage marks.

row_name

The record's file for the unit at path.

closure

package and every package it depends on, transitively, in path order.

closure_id

The identity of package's suite inputs at HEAD.

in_closure

Whether filename (workspace-relative) belongs to package's closure.

unit_fields

unit as a row and a leg's file hold it: its parts, without their facts.

units_facts

The fixed facts units' parts name, by name, each once.

leave_run

Leave leg's scope and measured units for its one write; "" or why not.

fold_leg

Answer ci.fold for RUN_FILE: one platform leg's file from its shards'.

folders

Answer ci.fold: the gate job folds RUN_FILE with fold_leg.

run_legs

Every check leg's coverage file on run's per-run refs; the legs, or the reason.

recorded

base's record on leg, main's by default: every recorded unit, in one read.

run_contexts

Every context one run's writes name, by unit path and measurer name.

put_record

Write the fresh units onto base's record on leg; "" or the reason.

Attributes:

Name Type Description
SCHEMA
FACTS
MAIN
RUN_FILE
ROOT_PINS
RECORD
KEPT_FOR
KEPT_STAMP

SCHEMA module-attribute

SCHEMA = 4

FACTS module-attribute

FACTS = 'facts/'

MAIN module-attribute

MAIN = 'main'

RUN_FILE module-attribute

RUN_FILE = 'coverage.json'

ROOT_PINS module-attribute

ROOT_PINS = ('pyproject.toml', 'uv.lock')

RECORD module-attribute

RECORD = git.Keyed('coverage', ('base', 'leg'), schema=SCHEMA, current=current_keys, linger=timedelta(days=1), blobs=FACTS)

KEPT_FOR module-attribute

KEPT_FOR = timedelta(days=1)

KEPT_STAMP module-attribute

KEPT_STAMP = '%Y%m%dT%H%M%SZ'

Unit dataclass

Unit(path: str, closure: str, run: str, sha: str, parts: tuple[dict[str, Any], ...] = (), facts: Mapping[str, Any] = dict())

One unit's measurement: the parts its suite's measurers made within its closure.

Attributes:

Name Type Description
path str

The unit's path (packages/forge, or tests).

closure str

The closure identity the measurement is keyed by.

run str

The run that measured it.

sha str

The commit that run checked out.

parts tuple[dict[str, Any], ...]

The unit's parts in the line format's stored form, as JSON holds them: one for each measurer that measured the suite, each holding the files of the suite's closure it reached. A suite that ran unmeasured has none.

facts Mapping[str, Any]

The fixed facts the parts name, by name; the units of one record share the record's. Two units with the same parts are equal whatever else their tables hold, since a part names its facts by their content.

path instance-attribute

path: str

closure instance-attribute

closure: str

run instance-attribute

run: str

sha instance-attribute

sha: str

parts class-attribute instance-attribute

parts: tuple[dict[str, Any], ...] = ()

facts class-attribute instance-attribute

facts: Mapping[str, Any] = field(default_factory=dict, compare=False, repr=False)

files property

files: tuple[str, ...]

The files the unit's parts hold, by their path from the root, sorted.

named property

named: dict[str, str]

The fixed facts the parts name, each with the digest of a file naming it.

Leg dataclass

Leg(key: str, label: str, scope: str, packages: tuple[str, ...], units: dict[str, Unit], why: str = '')

What one check leg put on its per-run ref.

Attributes:

Name Type Description
key str

The leg's part of the per-run key, as its ref spells it.

label str

The leg's label (check-ubuntu-latest-3.14), the key of its record.

scope str

The scope the leg's gate ran, as its marker named it.

packages tuple[str, ...]

The packages whose suites a narrowed leg ran.

units dict[str, Unit]

The suites the leg measured, by the unit's path.

why str

Why the ref carries no readable file, when it does not.

key instance-attribute

key: str

label instance-attribute

label: str

scope instance-attribute

scope: str

packages instance-attribute

packages: tuple[str, ...]

units instance-attribute

units: dict[str, Unit]

why class-attribute instance-attribute

why: str = ''

Kept dataclass

Kept(name: str, unit: Unit | None = None)

A row a write replaced at another closure, or removed, kept for a day.

Attributes:

Name Type Description
name str

The row's file on the ref: the moment the write kept it, the closure it was measured at, and its unit's slug (kept_name).

unit Unit | None

The measurement, once read. A read of a record names its kept rows and reads none; the union reads the one it asks for (Record.at).

name instance-attribute

name: str

unit class-attribute instance-attribute

unit: Unit | None = None

closure property

closure: str

The closure the kept row was measured at, from its name; empty for none.

slug property

slug: str

The slug of the kept row's unit, from its name; empty for none.

when property

when: datetime | None

When the write kept the row, from its name; None when it names none.

Record dataclass

Record(units: dict[str, Unit], skipped: tuple[Skipped, ...] = (), failed: bool = False, reason: str = '', kept: tuple[Kept, ...] = (), read_kept: Callable[[str], Unit | None] | None = None, facts: dict[str, Any] = dict(), held_facts: tuple[str, ...] = ())

A base's record on one leg, as read.

Attributes:

Name Type Description
units dict[str, Unit]

The recorded units by path: the current rows.

skipped tuple[Skipped, ...]

The rows that are not units, named; printing one gives the line.

failed bool

Whether the record could not be read at all.

reason str

Why it could not, in the store's wording.

kept tuple[Kept, ...]

The rows writes replaced or removed, by file name, kept for KEPT_FOR from the write that kept each.

read_kept Callable[[str], Unit | None] | None

Reads a kept row by its name, for at; None when the record was not read from the store.

facts dict[str, Any]

The fixed facts read, by name: the ones the current rows name, and a kept row's once it is read.

held_facts tuple[str, ...]

The name of every fixed facts file the record holds, read or not.

Methods:

Name Description
at

The measurement of path at closure: the current row's, else a kept one's.

expired

The names of the kept rows past their day at now.

stale

The row names to drop: units outside current, and rows that are no unit.

units instance-attribute

units: dict[str, Unit]

skipped class-attribute instance-attribute

skipped: tuple[Skipped, ...] = ()

failed class-attribute instance-attribute

failed: bool = False

reason class-attribute instance-attribute

reason: str = ''

kept class-attribute instance-attribute

kept: tuple[Kept, ...] = ()

read_kept class-attribute instance-attribute

read_kept: Callable[[str], Unit | None] | None = field(default=None, compare=False, repr=False)

facts class-attribute instance-attribute

facts: dict[str, Any] = field(default_factory=dict, compare=False, repr=False)

held_facts class-attribute instance-attribute

held_facts: tuple[str, ...] = ()

at

at(path: str, closure: str) -> Unit | None

The measurement of path at closure: the current row's, else a kept one's.

The union asks here for what a leg skipped: the leg read the current row, and a write since may have kept it instead. The kept rows of path at closure are read newest first, and the first that is the unit's answers.

Source code in packages/extensions/coverage/src/livery/extensions/coverage/_record.py
def at(self, path: str, closure: str) -> Unit | None:
    """The measurement of *path* at *closure*: the current row's, else a kept one's.

    The union asks here for what a leg skipped: the leg read the
    current row, and a write since may have kept it instead. The
    kept rows of *path* at *closure* are read newest first, and the
    first that is the unit's answers.
    """
    row = self.units.get(path)
    if row is not None and row.closure == closure:
        return row
    slug = git.slug(path)
    held = [
        kept for kept in self.kept if (kept.slug, kept.closure) == (slug, closure)
    ]
    # A kept row's name starts with its moment: newest first by name.
    for kept in sorted(held, key=attrgetter("name"), reverse=True):
        unit = kept.unit
        if unit is None and self.read_kept is not None:
            unit = self.read_kept(kept.name)
        if unit is not None and unit.path == path and unit.closure == closure:
            return unit
    return None

expired

expired(now: datetime) -> list[str]

The names of the kept rows past their day at now.

A row is kept for KEPT_FOR from the write that kept it, the moment its name carries. A row whose name carries none counts as expired: its age cannot be told.

Source code in packages/extensions/coverage/src/livery/extensions/coverage/_record.py
def expired(self, now: datetime) -> list[str]:
    """The names of the kept rows past their day at *now*.

    A row is kept for `KEPT_FOR` from the write that kept it, the
    moment its name carries. A row whose name carries none counts
    as expired: its age cannot be told.
    """
    names: list[str] = []
    for kept in self.kept:
        when = kept.when
        if when is None or now - when > KEPT_FOR:
            names.append(kept.name)
    return names

stale

stale(current: Iterable[str]) -> list[str]

The row names to drop: units outside current, and rows that are no unit.

A stale row is a unit that no longer exists, or a row the reader could not take as a unit at all.

Source code in packages/extensions/coverage/src/livery/extensions/coverage/_record.py
def stale(self, current: Iterable[str]) -> list[str]:
    """The row names to drop: units outside *current*, and rows that are no unit.

    A stale row is a unit that no longer exists, or a row the
    reader could not take as a unit at all.
    """
    keep = set(current)
    names = [row_name(path) for path in self.units if path not in keep]
    return names + [item.name for item in self.skipped]

branches

branches(root: Path) -> list[str] | None

The branches origin holds, by name; None if origin cannot be listed.

Source code in packages/extensions/coverage/src/livery/extensions/coverage/_record.py
def branches(root: Path) -> list[str] | None:
    """The branches origin holds, by name; ``None`` if origin cannot be listed."""
    heads = git.list_refs(state_store(root), "refs/heads/")
    if heads is None:
        return None
    return sorted(name[len("refs/heads/") :] for name in heads)

current_keys

current_keys(root: Path) -> set[tuple[str, ...]] | None

The keys the world produces: main and every branch on origin, per check leg.

The janitor drops a ref outside this set: a branch's record once the branch is gone from origin, and a leg's once the matrix no longer produces it. None when the contract or the remote cannot be read, and then nothing is dropped.

Source code in packages/extensions/coverage/src/livery/extensions/coverage/_record.py
def current_keys(root: Path) -> set[tuple[str, ...]] | None:
    """The keys the world produces: main and every branch on origin, per check leg.

    The janitor drops a ref outside this set: a branch's record once
    the branch is gone from origin, and a leg's once the matrix no
    longer produces it. ``None`` when the contract or the remote
    cannot be read, and then nothing is dropped.
    """
    from livery.workshop import check_legs

    try:
        legs = check_legs(root)
    except (Exception, SystemExit):
        return None
    heads = branches(root)
    if heads is None:
        return None
    bases = {MAIN, *(git.slug(branch) for branch in heads)}
    return {(base, git.slug(leg)) for base in bases for leg in legs}

facts_file

facts_file(name: str) -> str

The record's file for the fixed facts named name.

Source code in packages/extensions/coverage/src/livery/extensions/coverage/_record.py
def facts_file(name: str) -> str:
    """The record's file for the fixed facts named *name*."""
    return f"{FACTS}{name}.json"

kept_name

kept_name(path: str, closure: str, when: datetime) -> str

The record's file for a kept row: the unit at path, measured at closure.

The name carries the moment when the write kept it, so a reader tells a kept row's age from its name and reads no kept row for it. It ends in the unit's own row name (row_name), the moment and the closure before it: git chooses which files to store as the difference from another by the last sixteen characters of their names, so a kept row can be stored against its unit's other rows, of every leg and every run, where the stable numbering of the tests keeps that difference small.

Source code in packages/extensions/coverage/src/livery/extensions/coverage/_record.py
def kept_name(path: str, closure: str, when: datetime) -> str:
    """The record's file for a kept row: the unit at *path*, measured at *closure*.

    The name carries the moment *when* the write kept it, so a reader
    tells a kept row's age from its name and reads no kept row for it.
    It ends in the unit's own row name (`row_name`), the moment and the
    closure before it: git chooses which files to store as the
    difference from another by the last sixteen characters of their
    names, so a kept row can be stored against its unit's other rows,
    of every leg and every run, where the stable numbering of the tests
    keeps that difference small.
    """
    moment = f"{when.astimezone(UTC):{KEPT_STAMP}}"
    return f"{moment}.{closure}.{row_name(path)}"

record_ref

record_ref(leg: str, base: str = MAIN) -> str

The ref holding base's record on leg: main's, or a branch's.

Source code in packages/extensions/coverage/src/livery/extensions/coverage/_record.py
def record_ref(leg: str, base: str = MAIN) -> str:
    """The ref holding *base*'s record on *leg*: main's, or a branch's."""
    return RECORD.series(base, leg).ref

record_bases

record_bases(branch: str) -> tuple[str, ...]

The records a leg reads, in order: the branch's own if any, then main's.

Source code in packages/extensions/coverage/src/livery/extensions/coverage/_record.py
def record_bases(branch: str) -> tuple[str, ...]:
    """The records a leg reads, in order: the branch's own if any, then main's."""
    return (branch, MAIN) if branch and branch != MAIN else (MAIN,)

record_prefixes

record_prefixes(branch: str) -> tuple[str, ...]

The refs of the records a run reads, as prefixes: branch's if any, then main's.

A check leg and the gate job hand them to livery.git.remote_snapshot as the refs to bring when the block opens: one prefix per base record_bases names, spelled in full as that base's refs are (refs/workshop/coverage/<branch>/), so a run fetches the two records it reads and no other branch's. A record outside them, the one of the branch a verified row names, is brought on its first read, every leg of it, in one more fetch.

Source code in packages/extensions/coverage/src/livery/extensions/coverage/_record.py
def record_prefixes(branch: str) -> tuple[str, ...]:
    """The refs of the records a run reads, as prefixes: *branch*'s if any, then main's.

    A check leg and the gate job hand them to
    [livery.git.remote_snapshot][] as the refs to bring when the block
    opens: one prefix per base `record_bases` names, spelled in full
    as that base's refs are (``refs/workshop/coverage/<branch>/``), so
    a run fetches the two records it reads and no other branch's. A
    record outside them, the one of the branch a verified row names,
    is brought on its first read, every leg of it, in one more fetch.
    """
    return tuple(f"{RECORD.prefix}{git.slug(base)}/" for base in record_bases(branch))

series

series(root: Path) -> tuple[Series | Keyed, ...]

Answer store.series: the coverage record and the coverage marks.

The janitor bounds them and fm store.ls lists them while the workspace lists the coverage extension. The answer is the same in every workspace, so root is not read.

Source code in packages/extensions/coverage/src/livery/extensions/coverage/_record.py
def series(root: Path) -> tuple[git.Series | git.Keyed, ...]:
    """Answer ``store.series``: the coverage record and the coverage marks.

    The janitor bounds them and ``fm store.ls`` lists them while the
    workspace lists the coverage extension. The answer is the same in
    every workspace, so *root* is not read.
    """
    from livery.extensions.coverage._marks import SERIES

    del root
    return (RECORD, SERIES)

row_name

row_name(path: str) -> str

The record's file for the unit at path.

Source code in packages/extensions/coverage/src/livery/extensions/coverage/_record.py
def row_name(path: str) -> str:
    """The record's file for the unit at *path*."""
    return f"{git.slug(path)}.json"

closure

closure(packages: tuple[Package, ...], package: Package) -> tuple[Package, ...]

package and every package it depends on, transitively, in path order.

An edge naming a path no package has is skipped: the workspace lint refuses such an edge before any gate runs, and the identity here is of the packages that exist.

Source code in packages/extensions/coverage/src/livery/extensions/coverage/_record.py
def closure(packages: tuple[Package, ...], package: Package) -> tuple[Package, ...]:
    """*package* and every package it depends on, transitively, in path order.

    An edge naming a path no package has is skipped: the workspace
    lint refuses such an edge before any gate runs, and the identity
    here is of the packages that exist.
    """
    by_path = {item.path: item for item in packages}
    seen: dict[str, Package] = {}
    pending = [package]
    while pending:
        current = pending.pop()
        if current.path in seen:
            continue
        seen[current.path] = current
        pending.extend(
            by_path[edge.path] for edge in current.depends if edge.path in by_path
        )
    return tuple(seen[path] for path in sorted(seen))

closure_id

closure_id(root: Path, packages: tuple[Package, ...], package: Package) -> str

The identity of package's suite inputs at HEAD.

A digest over the tree ids of the package's closure directories and the blob ids of ROOT_PINS, so an identical identity means identical code and identical dependencies: the argument that lets the suite's run be skipped at all. A root pin the tree lacks contributes nothing.

The workspace's own tests reach any package, so their identity is every package's tree, their own directory's tree, and the root pins: prose leaves it untouched, and a root configuration change forces a full run anyway, which measures the unit afresh.

Raises:

Type Description
GitError

When a closure directory is not in HEAD.

Source code in packages/extensions/coverage/src/livery/extensions/coverage/_record.py
def closure_id(root: Path, packages: tuple[Package, ...], package: Package) -> str:
    """The identity of *package*'s suite inputs at ``HEAD``.

    A digest over the tree ids of the package's closure directories
    and the blob ids of `ROOT_PINS`, so an identical identity means
    identical code and identical dependencies: the argument that lets
    the suite's run be skipped at all. A root pin the tree lacks
    contributes nothing.

    The workspace's own tests reach any package, so their identity is
    every package's tree, their own directory's tree, and the root
    pins: prose leaves it untouched, and a root configuration change
    forces a full run anyway, which measures the unit afresh.

    Raises:
        GitError: When a closure directory is not in ``HEAD``.
    """
    if package.path == WORKSPACE_TESTS:
        required = [member.path for member in sorted(packages, key=attrgetter("path"))]
        required.append(WORKSPACE_TESTS)
    else:
        required = [member.path for member in closure(packages, package)]
    # One process whatever the closure holds: the ids of every required
    # directory and every pin come back together, and a pin the tree
    # lacks is simply absent.
    ids = git.object_ids(root, "HEAD", [*required, *ROOT_PINS])
    parts: list[str] = []
    for path in required:
        if path not in ids:
            raise git.GitError(f"HEAD holds no {path}")
        parts.append(f"{path}={ids[path]}")
    parts.extend(f"{pin}={ids[pin]}" for pin in ROOT_PINS if pin in ids)
    return hashlib.sha256("\n".join(parts).encode("utf-8")).hexdigest()

in_closure

in_closure(packages: tuple[Package, ...], package: Package, filename: str) -> bool

Whether filename (workspace-relative) belongs to package's closure.

Every file belongs to the workspace tests' closure.

Source code in packages/extensions/coverage/src/livery/extensions/coverage/_record.py
def in_closure(packages: tuple[Package, ...], package: Package, filename: str) -> bool:
    """Whether *filename* (workspace-relative) belongs to *package*'s closure.

    Every file belongs to the workspace tests' closure.
    """
    if package.path == WORKSPACE_TESTS:
        return True
    roots = tuple(member.path + "/" for member in closure(packages, package))
    return filename.replace("\\", "/").startswith(roots)

unit_fields

unit_fields(unit: Unit) -> dict[str, Any]

unit as a row and a leg's file hold it: its parts, without their facts.

Source code in packages/extensions/coverage/src/livery/extensions/coverage/_record.py
def unit_fields(unit: Unit) -> dict[str, Any]:
    """*unit* as a row and a leg's file hold it: its parts, without their facts."""
    return {
        "unit": unit.path,
        "closure": unit.closure,
        "run": unit.run,
        "sha": unit.sha,
        "parts": list(unit.parts),
    }

units_facts

units_facts(units: Iterable[Unit]) -> dict[str, Any]

The fixed facts units' parts name, by name, each once.

Source code in packages/extensions/coverage/src/livery/extensions/coverage/_record.py
def units_facts(units: Iterable[Unit]) -> dict[str, Any]:
    """The fixed facts *units*' parts name, by name, each once."""
    found: dict[str, Any] = {}
    for unit in units:
        for name in unit.named:
            if name not in found and name in unit.facts:
                found[name] = unit.facts[name]
    return found

leave_run

leave_run(root: Path, run: RunContext, *, leg: str, scope: str, packages: tuple[str, ...], units: Mapping[str, Unit]) -> str

Leave leg's scope and measured units for its one write; "" or why not.

The one file carries the scope the leg's gate ran, the packages a narrowed gate named, every unit's parts, and the fixed facts they name, each once, so the gate job reads a leg's whole fact in one read and the parts go with the ref when the run's metrics are collected. It lands in the leg's spool (livery.workshop.leave_leg_file), and the leg's last entry, ci.leg, puts it on the leg's per-run ref beside the timing row, so a leg writes its ref once.

Source code in packages/extensions/coverage/src/livery/extensions/coverage/_record.py
def leave_run(
    root: Path,
    run: RunContext,
    *,
    leg: str,
    scope: str,
    packages: tuple[str, ...],
    units: Mapping[str, Unit],
) -> str:
    """Leave *leg*'s scope and measured *units* for its one write; ``""`` or why not.

    The one file carries the scope the leg's gate ran, the packages a
    narrowed gate named, every unit's parts, and the fixed facts they
    name, each once, so the gate job reads a leg's whole fact in one
    read and the parts go with the ref when the run's metrics are
    collected. It lands in the leg's spool
    ([livery.workshop.leave_leg_file][]), and the leg's last entry,
    ``ci.leg``, puts it on the leg's per-run ref beside the timing row,
    so a leg writes its ref once.
    """
    from livery.workshop import leave_leg_file

    if not leg:
        return "refusing: the leg has no label, so its parts have no key"
    row = {
        "leg": leg,
        "scope": scope,
        "packages": list(packages),
        "run": run.run_id,
        "forge": run.forge,
        "units": {path: unit_fields(unit) for path, unit in sorted(units.items())},
        "facts": units_facts(units.values()),
    }
    leave_leg_file(root, RUN_FILE, row)
    return ""

fold_leg

fold_leg(shards: Sequence[Mapping[str, Any]]) -> dict[str, Any]

Answer ci.fold for RUN_FILE: one platform leg's file from its shards'.

Each shard of a platform ran a slice of the test files and left the parts of the suites it reached; together they are one leg's measurement. The folded file names the platform leg (livery.workshop.platform_leg), the shards' one scope and packages, and every unit any shard measured, its parts every shard's concatenated in shard order: a unit one shard measured and another did not is measured. The fixed facts are every shard's, each name once, since a name is its content's digest.

Raises:

Type Description
ValueError

When the shards' files are not one leg's, naming what differs: legs of two platforms, two scopes, two package lists, or a unit at two closures.

Source code in packages/extensions/coverage/src/livery/extensions/coverage/_record.py
def fold_leg(shards: Sequence[Mapping[str, Any]]) -> dict[str, Any]:
    """Answer ``ci.fold`` for `RUN_FILE`: one platform leg's file from its *shards*'.

    Each shard of a platform ran a slice of the test files and left
    the parts of the suites it reached; together they are one leg's
    measurement. The folded file names the platform leg
    ([livery.workshop.platform_leg][]), the shards' one scope and
    packages, and every unit any shard measured, its parts every
    shard's concatenated in shard order: a unit one shard measured and
    another did not is measured. The fixed facts are every shard's,
    each name once, since a name is its content's digest.

    Raises:
        ValueError: When the shards' files are not one leg's, naming
            what differs: legs of two platforms, two scopes, two
            package lists, or a unit at two closures.
    """
    if not shards:
        raise ValueError("no shard left its coverage file; nothing to fold")
    legs = {str(shard.get("leg", "") or "") for shard in shards}
    platforms = {platform_leg(leg) for leg in legs}
    if len(platforms) > 1:
        raise ValueError(
            f"the files belong to {len(platforms)} platforms"
            f" ({', '.join(sorted(platforms))}); a fold is one platform's"
        )
    scopes = {str(shard.get("scope", "") or "") for shard in shards}
    if len(scopes) > 1:
        raise ValueError(
            f"the shards measured different scopes ({', '.join(sorted(scopes))});"
            " a platform's shards run one gate"
        )
    packages = {
        tuple(
            sorted(str(item) for item in cast("list[Any]", shard.get("packages") or []))
        )
        for shard in shards
    }
    if len(packages) > 1:
        shown = "; ".join(", ".join(held) or "none" for held in sorted(packages))
        raise ValueError(
            f"the shards named different packages ({shown}); a platform's shards"
            " run one gate"
        )
    units: dict[str, dict[str, Any]] = {}
    facts: dict[str, Any] = {}
    for shard in shards:
        raw_units: Any = shard.get("units")
        if isinstance(raw_units, dict):
            for path, raw in sorted(cast("dict[str, Any]", raw_units).items()):
                if not isinstance(raw, dict):
                    continue
                entry = cast("dict[str, Any]", raw)
                parts: Any = entry.get("parts")
                held = units.get(path)
                if held is None:
                    units[path] = {**entry, "unit": path, "parts": []}
                    held = units[path]
                elif str(held.get("closure", "")) != str(entry.get("closure", "")):
                    raise ValueError(
                        f"{path} is measured at closure"
                        f" {str(held.get('closure', ''))[:12]} by one shard and"
                        f" {str(entry.get('closure', ''))[:12]} by another; a"
                        " platform's shards measure one closure"
                    )
                if isinstance(parts, list):
                    cast("list[Any]", held["parts"]).extend(parts)
        raw_facts: Any = shard.get("facts")
        if isinstance(raw_facts, dict):
            for name, content in cast("dict[str, Any]", raw_facts).items():
                facts.setdefault(name, content)
    first = shards[0]
    return {
        "leg": next(iter(platforms)) if platforms else "",
        "scope": next(iter(scopes)) if scopes else "",
        "packages": list(next(iter(packages))) if packages else [],
        "run": first.get("run", ""),
        "forge": first.get("forge", ""),
        "units": units,
        "facts": facts,
    }

folders

folders(root: Path) -> dict[str, Callable[[Sequence[Mapping[str, Any]]], dict[str, Any]]]

Answer ci.fold: the gate job folds RUN_FILE with fold_leg.

The answer is the same in every workspace, so root is not read.

Source code in packages/extensions/coverage/src/livery/extensions/coverage/_record.py
def folders(
    root: Path,
) -> dict[str, Callable[[Sequence[Mapping[str, Any]]], dict[str, Any]]]:
    """Answer ``ci.fold``: the gate job folds `RUN_FILE` with `fold_leg`.

    The answer is the same in every workspace, so *root* is not read.
    """
    del root
    return {RUN_FILE: fold_leg}

run_legs

run_legs(root: Path, run: RunContext) -> tuple[list[Leg], str]

Every check leg's coverage file on run's per-run refs; the legs, or the reason.

The reason is set when the refs could not be listed. A ref that carries no readable coverage file is a leg whose why names it: a leg that died before its parts, or a file that is not a row; the caller decides, and a missing put is red there.

Source code in packages/extensions/coverage/src/livery/extensions/coverage/_record.py
def run_legs(root: Path, run: RunContext) -> tuple[list[Leg], str]:
    """Every check leg's coverage file on *run*'s per-run refs; the legs, or the reason.

    The reason is set when the refs could not be listed. A ref that
    carries no readable coverage file is a leg whose ``why`` names
    it: a leg that died before its parts, or a file that is not a
    row; the caller decides, and a missing put is red there.
    """
    from livery.workshop import RUNS

    keys = RUNS.listed(state_store(root), run.run_id)
    if keys is None:
        return [], f"{RUNS.prefix}{run.run_id}/*: the remote could not be listed"
    legs: list[Leg] = []
    for key in keys:
        series = RUNS.at(*key)
        found, why = series.row(state_store(root), RUN_FILE)
        if why:
            legs.append(Leg(key[-1], "", "", (), {}, why=f"{series.ref}: {why}"))
            continue
        if found is None:
            legs.append(
                Leg(
                    key[-1],
                    "",
                    "",
                    (),
                    {},
                    why=f"{series.ref} carries no {RUN_FILE}",
                )
            )
            continue
        raw_units = found.data.get("units")
        raw_facts = found.data.get("facts")
        facts = raw_facts if isinstance(raw_facts, dict) else {}
        units: dict[str, Unit] = {}
        if isinstance(raw_units, dict):
            for path, raw in raw_units.items():
                unit, _why = _parse_unit(raw, path=str(path), facts=facts)
                if unit is not None:
                    units[unit.path] = unit
        legs.append(
            Leg(
                key[-1],
                str(found.data.get("leg", "") or ""),
                str(found.data.get("scope", "") or ""),
                tuple(str(item) for item in found.data.get("packages", []) or []),
                units,
            )
        )
    return legs, ""

recorded

recorded(root: Path, *, leg: str, base: str = MAIN) -> Record

base's record on leg, main's by default: every recorded unit, in one read.

An absent record is no units and no failure; a record the store could not read is a failure with its reason; a row that is not a unit is skipped and named, and the rest stand. The current rows alone are read, then the fixed facts they name: a row naming facts the record does not hold is skipped, naming the facts and the content, so its unit is a miss and is measured afresh. A row named by its unit, closure and moment (kept_name) is a kept one, named in the record and read, with its facts, only when the union asks for it (Record.at). The record is the platform leg's (livery.workshop.platform_leg), so a shard's label reads its platform's. A leg without a label, or an empty base, has no record to read.

Source code in packages/extensions/coverage/src/livery/extensions/coverage/_record.py
def recorded(root: Path, *, leg: str, base: str = MAIN) -> Record:
    """*base*'s record on *leg*, main's by default: every recorded unit, in one read.

    An absent record is no units and no failure; a record the store
    could not read is a failure with its reason; a row that is not a
    unit is skipped and named, and the rest stand. The current rows
    alone are read, then the fixed facts they name: a row naming facts
    the record does not hold is skipped, naming the facts and the
    content, so its unit is a miss and is measured afresh. A row named
    by its unit, closure and moment (`kept_name`) is a kept one, named
    in the record and read, with its facts, only when the union asks
    for it (`Record.at`). The record is the platform leg's
    ([livery.workshop.platform_leg][]), so a shard's label reads its
    platform's. A leg without a label, or an empty base, has no record
    to read.
    """
    if not leg:
        return Record({}, failed=True, reason="this leg has no label")
    if not base:
        return Record({}, failed=True, reason="no branch to read a record for")
    series = RECORD.series(base, platform_leg(leg))
    store = state_store(root)
    found = series.rows(store, only=_current_row)
    if found.failed:
        return Record({}, failed=True, reason=found.reason)
    facts: dict[str, Any] = {}
    units: dict[str, Unit] = {}
    skipped = list(found.skipped)
    for row in found.rows:
        unit, why = _parse_unit(row.data, facts=facts)
        if unit is None:
            skipped.append(git.Skipped(row.name, why))
            continue
        units[unit.path] = unit
    why = _read_facts(series, store, facts, units.values())
    if why:
        return Record({}, failed=True, reason=why)
    for path, unit in sorted(units.items()):
        missing = sorted(name for name in unit.named if name not in facts)
        if missing:
            skipped.append(git.Skipped(row_name(path), _no_facts(unit, missing[0])))
            del units[path]

    def read_kept(name: str) -> Unit | None:
        row, _why = series.row(store, name)
        unit = None if row is None else _parse_unit(row.data, facts=facts)[0]
        if unit is None or _read_facts(series, store, facts, [unit]):
            return None
        return unit if all(name in facts for name in unit.named) else None

    held = tuple(name for name in found.unread if series.blob(name))
    kept = tuple(Kept(name) for name in found.unread if not series.blob(name))
    return Record(
        units,
        tuple(skipped),
        kept=kept,
        read_kept=read_kept,
        facts=facts,
        held_facts=held,
    )

run_contexts

run_contexts(writes: Iterable[Mapping[str, Unit]]) -> dict[str, dict[str, set[str]]]

Every context one run's writes name, by unit path and measurer name.

writes holds each leg's rows, by unit path. A context that is not text names nothing here; the write that reads its row refuses it.

Source code in packages/extensions/coverage/src/livery/extensions/coverage/_record.py
def run_contexts(
    writes: Iterable[Mapping[str, Unit]],
) -> dict[str, dict[str, set[str]]]:
    """Every context one run's *writes* name, by unit path and measurer name.

    *writes* holds each leg's rows, by unit path. A context that is not
    text names nothing here; the write that reads its row refuses it.
    """
    found: dict[str, dict[str, set[str]]] = {}
    for rows in writes:
        for path, unit in rows.items():
            for raw in unit.parts:
                named: object = raw.get("contexts")
                if not isinstance(named, list):
                    continue
                held = found.setdefault(path, {}).setdefault(_measurer_of(raw), set())
                held.update(
                    item
                    for item in cast("list[object]", named)
                    if isinstance(item, str)
                )
    return found

put_record

put_record(root: Path, run: RunContext, *, leg: str, fresh: Mapping[str, Unit], remove: Iterable[str] = (), base: str = MAIN, held: Record | None = None, now: datetime | None = None, numbered_like: Mapping[str, Unit] | None = None, run_contexts: Mapping[str, Mapping[str, Collection[str]]] | None = None) -> str

Write the fresh units onto base's record on leg; "" or the reason.

Main's record is written by main's run alone, the run of a push; a branch's record by that branch's own pull request run alone. Either replaces its record in place: a fresh row replaces the unit's, a row named in remove goes, every other row stands as the carried measurement it is. Any other run is refused by name, so a pull request never writes what main's next run will skip on, and no branch writes another's record.

A current row that a fresh one replaces at another closure, or that remove names, stays as a kept row under its closure's name and the moment now (kept_name), and every kept row older than KEPT_FOR at now goes. held is the record as the caller read it; it is read here when absent, and a record that cannot be read refuses the write, since what it would keep is unknown.

The fixed facts a fresh row names are written beside the rows when the record does not hold them yet, from the fresh units' own facts, and the facts no row of the written record names go. The kept rows that stay are read for the facts they name, and a write whose rows name facts no unit holds is refused by name.

Each fresh row numbers the tests that reached its lines as the row it replaces numbers them, part by part of one measurer: every context that row numbers keeps its index, and the new ones follow, sorted: the ones run_contexts names for the unit and measurer (run_contexts, every test the run's rows of the unit name on any leg) and the part's own. Rows of every leg start alike and extend alike, so a test has one index in a unit's rows on every leg. A unit the record does not hold yet is numbered as its row in numbered_like, main's for a branch's first write, and without one its numbering starts afresh. A fresh part off the line format refuses the write, naming the unit and the key.

Source code in packages/extensions/coverage/src/livery/extensions/coverage/_record.py
def put_record(
    root: Path,
    run: RunContext,
    *,
    leg: str,
    fresh: Mapping[str, Unit],
    remove: Iterable[str] = (),
    base: str = MAIN,
    held: Record | None = None,
    now: datetime | None = None,
    numbered_like: Mapping[str, Unit] | None = None,
    run_contexts: Mapping[str, Mapping[str, Collection[str]]] | None = None,
) -> str:
    """Write the *fresh* units onto *base*'s record on *leg*; ``""`` or the reason.

    Main's record is written by main's run alone, the run of a push;
    a branch's record by that branch's own pull request run alone.
    Either replaces its record in place: a fresh row replaces the
    unit's, a row named in *remove* goes, every other row stands as
    the carried measurement it is. Any other run is refused by name,
    so a pull request never writes what main's next run will skip on,
    and no branch writes another's record.

    A current row that a fresh one replaces at another closure, or
    that *remove* names, stays as a kept row under its closure's name
    and the moment *now* (`kept_name`), and every kept row older than
    `KEPT_FOR` at *now* goes. *held* is the record as the caller read
    it; it is read here when absent, and a record that cannot be read
    refuses the write, since what it would keep is unknown.

    The fixed facts a fresh row names are written beside the rows when
    the record does not hold them yet, from the fresh units' own
    facts, and the facts no row of the written record names go. The
    kept rows that stay are read for the facts they name, and a write
    whose rows name facts no unit holds is refused by name.

    Each fresh row numbers the tests that reached its lines as the row
    it replaces numbers them, part by part of one measurer: every
    context that row numbers keeps its index, and the new ones follow,
    sorted: the ones *run_contexts* names for the unit and measurer
    (`run_contexts`, every test the run's rows of the unit name on any
    leg) and the part's own. Rows of every leg start alike and extend
    alike, so a test has one index in a unit's rows on every leg. A
    unit the record does not hold yet is numbered as its row in
    *numbered_like*, main's for a branch's first write, and without
    one its numbering starts afresh. A fresh part off the line format
    refuses the write, naming the unit and the key.
    """
    if not leg:
        return "refusing: the leg has no label, so the record has no key"
    if not base:
        return "refusing: no branch to write a record for"
    if base == MAIN and run.event != "push":
        return (
            f"refusing: a {run.event or 'local'} run reads {MAIN}'s record"
            " and never writes it"
        )
    if base != MAIN and (run.event != "pull_request" or run.head_ref != base):
        return (
            f"refusing: {base}'s record is written by that branch's own pull"
            f" request run, not a {run.event or 'local'} run of"
            f" {run.head_ref or 'no branch'}"
        )
    current = held if held is not None else recorded(root, leg=leg, base=base)
    if current.failed:
        return (
            f"refusing: {base}'s record on {leg} could not be read"
            f" ({current.reason}), so the rows it keeps are unknown"
        )
    like = numbered_like or {}
    appended = run_contexts or {}
    try:
        fresh = {
            path: _numbered(
                unit, current.units.get(path) or like.get(path), appended.get(path, {})
            )
            for path, unit in fresh.items()
        }
    except PartError as error:
        return f"refusing: {error}"
    moment = now or datetime.now(UTC)
    removed = list(remove)
    rows = {row_name(path): unit_fields(unit) for path, unit in sorted(fresh.items())}
    named: dict[str, str] = {}
    for unit in fresh.values():
        named.update(unit.named)
    for path, unit in sorted(current.units.items()):
        replaced = path in fresh and fresh[path].closure != unit.closure
        dropped = path not in fresh and row_name(path) in removed
        if replaced or dropped:
            rows[kept_name(path, unit.closure, moment)] = unit_fields(unit)
        if replaced or path not in fresh:
            # The row stands, or stays as a kept row: its facts stay.
            named.update(unit.named)
    removed += current.expired(moment)
    series = RECORD.series(base, leg)
    store = state_store(root)
    staying = {kept.name for kept in current.kept} - set(removed) - set(rows)
    if staying:
        found = series.rows(store, only=staying.__contains__)
        if found.failed:
            return (
                f"refusing: {base}'s kept rows on {leg} could not be read"
                f" ({found.reason}), so the fixed facts they name are unknown"
            )
        for row in found.rows:
            staying_unit, _why = _parse_unit(row.data)
            if staying_unit is not None:
                named.update(staying_unit.named)
    on_record = set(current.held_facts)
    table: dict[str, Any] = dict(current.facts)
    for unit in fresh.values():
        table.update(unit.facts)
    absent = sorted(
        name
        for name in named
        if facts_file(name) not in on_record and name not in table
    )
    if absent:
        return (
            f"refusing: the rows name the fixed facts {absent[0]} of content"
            f" {named[absent[0]][:12] or 'that was not read'}, which no unit holds"
        )
    blobs = {
        facts_file(name): facts_text(table[name])
        for name in sorted(named)
        if facts_file(name) not in on_record
    }
    removed += sorted(file for file in on_record if _facts_named_by(file) not in named)
    return series.put(
        store,
        rows,
        message=f"coverage: {base}'s record on {leg} by run {run.run_id}",
        remove=removed,
        blobs=blobs,
    )