Skip to content

livery.workshop._speed_tasks

livery.workshop._speed_tasks

fm speed.judge and fm speed.accept: the test speed marks' verbs.

speed.judge runs in the gate job after the timing rows are collected: it judges every check leg's summed test time per package against the marks (livery.workshop._speed), records the first marks and the ratchets, warns on a run over the mark, and is red on the second run over in a row on a reference leg. speed.accept is the person's raise of a mark, with the reason on the record. Both verbs do nothing until the contract declares [ci] speed-marks = true, and speed.judge runs on a CI leg alone.

Functions:

Name Description
judge_flow

Judge the run's legs, record what moves, and return the red lines.

speed_judge

Judge the run's test times against the speed marks; red on the second run over.

speed_accept

Raise a package's speed mark deliberately, with the reason on the record.

Attributes:

Name Type Description
speed

speed module-attribute

speed = group('speed', help="The test suites' speed marks")

judge_flow

judge_flow(root: Path, run: RunContext) -> list[str]

Judge the run's legs, record what moves, and return the red lines.

Prints every verdict and every write. Fails open: a store that cannot be read prints its reason and reddens nothing.

Source code in packages/workshop/src/livery/workshop/_speed_tasks.py
def judge_flow(root: Path, run: RunContext) -> list[str]:
    """Judge the run's legs, record what moves, and return the red lines.

    Prints every verdict and every write. Fails open: a store that
    cannot be read prints its reason and reddens nothing.
    """
    from livery.git import remote_snapshot

    with remote_snapshot(state_store(root), fetch=("metrics", "speed/marks")):
        verdicts, why = _speed.judge_run(root, run)
        for line in why:
            print(line)
        for verdict in verdicts:
            for line in _speed.render(verdict):
                print(line)
        for line in _speed.apply(root, verdicts, by=f"run {run.run_id}"):
            print(line)
    return [
        f"{v.sample.package} on {v.sample.leg}: {v.sample.seconds:.1f}s over the"
        f" limit {v.limit:.1f}s for the second run in a row"
        for v in verdicts
        if v.red and v.limit is not None
    ]

speed_judge

speed_judge() -> None

Judge the run's test times against the speed marks; red on the second run over.

Runs in the gate job after the timing rows are collected. Off until the contract declares [ci] speed-marks = true: it says so, and drops any marks left from before the switch, so a later opt-in starts from the timings alone. A CI verb: outside CI it is listed as unavailable with the reason and refused, since a machine's clock is not a leg the marks are taken on.

Source code in packages/workshop/src/livery/workshop/_speed_tasks.py
@speed.task(name="judge", hidden=True)
@requires(
    lambda: run_context() is not None,
    reason="a CI run: the speed marks are judged on the CI legs",
)
def speed_judge() -> None:
    """Judge the run's test times against the speed marks; red on the second run over.

    Runs in the gate job after the timing rows are collected. Off
    until the contract declares ``[ci] speed-marks = true``: it says
    so, and drops any marks left from before the switch, so a later
    opt-in starts from the timings alone. A CI verb: outside CI it is
    listed as unavailable with the reason and refused, since a
    machine's clock is not a leg the marks are taken on.
    """
    # The CI question first: outside a run the verb refuses with that
    # reason wherever it stands, a workspace or not.
    run = run_context()
    if run is None:
        fail("not a CI run: the speed marks are judged on the CI legs")
    root = _root()
    if not _speed.enabled(root):
        print(
            f"  speed marks are off: declare [ci] {_speed.ENABLED_KEY} = true"
            " in workshop.toml to judge test time"
        )
        for line in _speed.drop_marks(root):
            print(line)
        return
    red = judge_flow(root, run)
    if red:
        fail(
            "test speed over the mark for two runs in a row:\n  "
            + "\n  ".join(red)
            + "\n  make the suite faster, or accept the cost with"
            f" `{footman.prog()} speed.accept <package> <seconds> --reason=<why>`"
        )

speed_accept

speed_accept(package: Annotated[str, doc('the package path, as packages/forge')], seconds: Annotated[float, doc("the new mark: the suite's summed test time")], reason: Annotated[str, doc('why the suite may take longer; goes on the record')] = '', leg: Annotated[str, doc('the check leg; the reference leg when empty')] = '') -> None

Raise a package's speed mark deliberately, with the reason on the record.

The mark comes down on its own when the runs beat it; raising it is a person's act, so this writes a dated row naming who and why, and the next gated run judges from it. Refuses while the marks are off, without a reason, for an unknown package, at or below the current mark, or when the marks cannot be read.

Source code in packages/workshop/src/livery/workshop/_speed_tasks.py
@speed.task(name="accept")
def speed_accept(
    package: Annotated[str, doc("the package path, as packages/forge")],
    seconds: Annotated[float, doc("the new mark: the suite's summed test time")],
    reason: Annotated[
        str, doc("why the suite may take longer; goes on the record")
    ] = "",
    leg: Annotated[str, doc("the check leg; the reference leg when empty")] = "",
) -> None:
    """Raise a package's speed mark deliberately, with the reason on the record.

    The mark comes down on its own when the runs beat it; raising it
    is a person's act, so this writes a dated row naming who and why,
    and the next gated run judges from it. Refuses while the marks
    are off, without a reason, for an unknown package, at or below
    the current mark, or when the marks cannot be read.
    """
    from livery.workshop._points import check_legs
    from livery.workshop._quality import _git_identity

    root = _root()
    if not _speed.enabled(root):
        fail(
            f"speed marks are off: declare [ci] {_speed.ENABLED_KEY} = true"
            " in workshop.toml before accepting a mark"
        )
    packages = tuple(item.path for item in discover_packages(root))
    legs = check_legs(root)
    chosen = leg or next((item for item in legs if _speed.is_reference(item)), "")
    if not chosen:
        chosen = legs[0] if legs else ""
    if not chosen:
        fail("no check leg: the contract names no runner to judge on")
    for line in _speed.accept(
        root,
        package=package,
        leg=chosen,
        seconds=seconds,
        reason=reason,
        by=_git_identity(root),
        packages=packages,
    ):
        print(line)