livery.workshop._speed¶
livery.workshop._speed ¶
The test speed marks: a package's suite time per check leg, and how it moves.
The gate job judges each check leg's summed test time per package, the
packages field of the timing rows (livery.workshop._metrics),
against a mark on the speed/marks series. The mark is the median of
the leg's last five green runs for the package, recorded once five are
seen; it ratchets down when that median beats it by more than five per
cent, so a suite cannot regress slowly, and a person raises it with
fm speed.accept, since a new heavy test is a cost someone chose.
A run over the mark by more than fifteen per cent and twenty seconds warns, naming the leg's slowest tests and the ones that grew most against their own medians; the second such run in a row on a reference leg (ubuntu) is red, and the other legs warn only until their variance is measured. A store that cannot be read falls open with its reason and writes nothing. The margins are constants here, chosen before the rows could set them; a reading of a month of rows may move them.
Reach for judge_run to judge the run the gate job is in, accept for
the person's raise, render_marks for what fm ci.timings adds, and
livery.workshop.speed_marks for the current marks. A test's
package, which the speed plugin sums by and the timing rows key by, is
what livery.workshop.package_of names.
Classes:
| Name | Description |
|---|---|
SpeedMark |
A package's mark on one check leg and the row that set it. |
Sample |
One package's time on one leg in this run, with the leg's history. |
Verdict |
One package on one leg, judged against its mark. |
Functions:
| Name | Description |
|---|---|
package_of |
Name the workspace package a test belongs to, from its pytest node id. |
enabled |
The contract's |
drop_marks |
Drop the marks series while the marks are off; the lines to print. |
leg_label |
The leg label of a matrix job's display name; a plain name stands. |
display_shard |
The shard a job's display name spells; |
platform_display |
The display name of a shard's platform: display without its shard. |
forge_shard_row |
Whether job's row is the forge's own row of a shard: no scope, a shard named. |
leg_of |
The leg a run's job row measured: the row's own label, or the name's. |
is_reference |
Whether leg runs on the reference runner, the one whose red counts. |
speed_marks |
Read each package's current test-time mark on each check leg. |
write_mark |
Append one row setting package's mark on leg; |
samples |
This run's package times per check leg, with each leg's green history. |
judge |
Each sample against its mark; one verdict each. |
render |
The lines for verdict: the time, the mark, the state, the culprits. |
judge_run |
Judge the run's check legs; the verdicts and the lines for what could not be. |
apply |
Write the first marks and the ratchets among verdicts; the lines. |
accept |
Raise package's mark on leg to seconds deliberately; the lines. |
render_marks |
The marks beside the newest run's times, for |
Attributes:
| Name | Type | Description |
|---|---|---|
SPEED_FILE_VARIABLE |
|
|
PACKAGE_TREES |
|
|
SERIES |
|
|
SAMPLE |
|
|
MARGIN |
|
|
FLOOR_SECONDS |
|
|
RATCHET |
|
|
REFERENCE |
|
|
ENABLED_KEY |
|
|
SLOWEST |
|
|
GROWN |
|
SpeedMark
dataclass
¶
A package's mark on one check leg and the row that set it.
The mark is the suite's summed test time the gate job judges the leg's runs against; livery.workshop.speed_marks reads the current mark of each package on each leg.
Attributes:
| Name | Type | Description |
|---|---|---|
package |
str
|
The package path ( |
leg |
str
|
The check leg ( |
seconds |
float
|
The mark: the suite's summed test time. |
kind |
str
|
|
by |
str
|
The run or the person that wrote the row. |
reason |
str
|
The reason an accept gave; empty otherwise. |
when |
str
|
When the row was written, ISO 8601. |
Sample
dataclass
¶
Sample(package: str, leg: str, seconds: float, tests: int, previous: tuple[float, ...], slowest: tuple[tuple[str, float], ...] = (), grown: tuple[tuple[str, float, float], ...] = ())
One package's time on one leg in this run, with the leg's history.
Attributes:
| Name | Type | Description |
|---|---|---|
package |
str
|
The package path. |
leg |
str
|
The check leg. |
seconds |
float
|
This run's summed test time for the package. |
tests |
int
|
How many tests ran. |
previous |
tuple[float, ...]
|
The leg's earlier green runs' times for the package,
newest first, at most one fewer than |
slowest |
tuple[tuple[str, float], ...]
|
This run's slowest tests, name and seconds, slowest first. |
grown |
tuple[tuple[str, float, float], ...]
|
The tests that grew most against their median over the earlier runs: name, the median, this run's seconds. |
Verdict
dataclass
¶
One package on one leg, judged against its mark.
Attributes:
| Name | Type | Description |
|---|---|---|
sample |
Sample
|
The run's time and the leg's history. |
mark |
SpeedMark | None
|
The current mark, or |
package_of ¶
Name the workspace package a test belongs to, from its pytest node id.
The package is the path before the first directory a package keeps
its tests or sources in, tests, src or docs:
packages/forge/tests/test_x.py::test_y belongs to
packages/forge. A package in a group directory is one level
deeper, and its own name may be one of those three: in
packages/extensions/docs/tests/test_x.py::test_y the tests
beneath docs says docs is the package, so the answer is
packages/extensions/docs. Nothing is read from the disk.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
nodeid
|
str
|
A test's node id, relative to the workspace root. |
required |
Returns:
| Type | Description |
|---|---|
str
|
The package's path relative to the workspace root, or |
str
|
a node outside |
str
|
|
Source code in packages/workshop/src/livery/workshop/_speed.py
enabled ¶
The contract's [ci] speed-marks; false when undeclared.
Off by default: a hosted runner's test time varies by tens of seconds between two runs of one tree, and a mark taken from such runs says nothing about the suite. A workspace whose runners keep a steady clock declares the key, which the contract's judge holds to a boolean.
Source code in packages/workshop/src/livery/workshop/_speed.py
drop_marks ¶
Drop the marks series while the marks are off; the lines to print.
Rows written before the switch went off would be the marks a later opt-in judges from, so the gate job drops the whole series when it finds any. Fails open: a store that cannot be read, or a delete the remote refuses, prints its reason and drops nothing. Nothing to print when the series is already gone.
Source code in packages/workshop/src/livery/workshop/_speed.py
leg_label ¶
The leg label of a matrix job's display name; a plain name stands.
check (ubuntu-latest, 3.14) is the leg check-ubuntu-latest-3.14
and check (windows-latest, 3.14, 2of4) the shard leg
check-windows-latest-3.14-2of4, the spelling the job runner gives
every entry it spawns.
Source code in packages/workshop/src/livery/workshop/_speed.py
display_shard ¶
The shard a job's display name spells; ONE when it spells none.
check (windows-latest, 3.14, 2of4) on GitHub and Gitea,
check: [2of4] on GitLab. A tail that is not spelled
<i>of<n> names no shard, so a matrix with another third value
reads as one shard of one.
Source code in packages/workshop/src/livery/workshop/_speed.py
platform_display ¶
The display name of a shard's platform: display without its shard.
check (windows-latest, 3.14, 2of4) gives check (windows-latest,
3.14) and check: [2of4] gives check; a name that spells no
shard stands. The gate job's fold names a platform's folded row by
it, and the collect joins that row with the forge's jobs.
Source code in packages/workshop/src/livery/workshop/_speed.py
forge_shard_row ¶
Whether job's row is the forge's own row of a shard: no scope, a shard named.
The gate job folds a platform's shards into one row under the
platform's display name, and the collect keeps the forge's rows of
the shards beside it for fm ci.timings; those carry the forge's
times and no scope. The stamp and the speed judge skip them, since
the folded row carries what they read.
Source code in packages/workshop/src/livery/workshop/_speed.py
leg_of ¶
The leg a run's job row measured: the row's own label, or the name's.
A check leg writes its label (check-ubuntu-latest-3.14) into
its row under scope.leg, and that spelling holds on every
forge; the job's display name carries the matrix on GitHub and
Gitea alone, and on GitLab is the bare check, which would name
no runner and never count as the reference.
Source code in packages/workshop/src/livery/workshop/_speed.py
is_reference ¶
speed_marks ¶
Read each package's current test-time mark on each check leg.
The marks live on the checkout's state store (livery.workshop.state_store). The newest row per package and leg is its mark. A row that does not parse, is of another schema, or lacks a mark's fields is skipped, so one bad row disables nothing. A store that cannot be read falls open: the caller prints the reason and judges nothing.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
root
|
Path
|
The checkout whose state store holds the marks. |
required |
Returns:
| Type | Description |
|---|---|
dict[tuple[str, str], SpeedMark] | None
|
Each mark by its package's path and its leg, with an empty |
str
|
reason; an empty store gives |
tuple[dict[tuple[str, str], SpeedMark] | None, str]
|
cannot be read, None and the reason. |
Source code in packages/workshop/src/livery/workshop/_speed.py
write_mark ¶
write_mark(root: Path, *, package: str, leg: str, seconds: float, kind: str, by: str, reason: str = '') -> str
Append one row setting package's mark on leg; "" or why not.
Source code in packages/workshop/src/livery/workshop/_speed.py
samples ¶
This run's package times per check leg, with each leg's green history.
entries are the timing rows newest first. The run's own entry
is the one whose run is run_id; a check leg of it yields one
sample per package whose tests ran, and the forge's own row of a
shard (forge_shard_row) yields none. The history is the same leg's
earlier entries that concluded green and ran the package's tests,
newest first, up to one fewer than SAMPLE. Empty when the run
has no entry.
Source code in packages/workshop/src/livery/workshop/_speed.py
judge ¶
Each sample against its mark; one verdict each.
Source code in packages/workshop/src/livery/workshop/_speed.py
render ¶
The lines for verdict: the time, the mark, the state, the culprits.
Source code in packages/workshop/src/livery/workshop/_speed.py
judge_run ¶
Judge the run's check legs; the verdicts and the lines for what could not be.
Reads the timing rows and the marks; either unreadable falls open with its reason and no verdict, so a store fault never reddens a run and never writes.
Source code in packages/workshop/src/livery/workshop/_speed.py
apply ¶
Write the first marks and the ratchets among verdicts; the lines.
Source code in packages/workshop/src/livery/workshop/_speed.py
accept ¶
accept(root: Path, *, package: str, leg: str, seconds: float, reason: str, by: str, packages: tuple[str, ...]) -> list[str]
Raise package's mark on leg to seconds deliberately; the lines.
Refuses without a reason, for a path not among packages, a time that is not positive, an unreadable store, and a value at or below the current mark: lowering is the ratchet's own move.
Source code in packages/workshop/src/livery/workshop/_speed.py
render_marks ¶
The marks beside the newest run's times, for fm ci.timings.