1"""The step registry: what the quality gate runs, one record per step.
3A step is one tool's judgment over the workspace or over one
4package: a formatter's pass, the render drift comparison, ctest
5over a native member. Each is a [livery.workshop._steps.StepRecord][]
6registered through [livery.workshop._steps.register_step][], and
7the gate is the walk over the registry: every applicable step
8green, the exit code the verdict. The workshop registers the
9builtin records at import; an extension's records register at
10mount, after them and in composition order, each extension after
11the ones it requires, ties alphabetical, and re-registering a name
12replaces the record.
14A step names the extensions whose packages it judges, and a package
15holding none of them skips it by name: a C++ member is formatted over
16its recipe by a python formatter and over its sources by clang-format
17because each step names an extension it holds. The phases the
18registry knows are the ones its steps name, so they are what the
19listed extensions register.
21A step is named by its phase and its name in the phase,
22``test.pytest``. The phase is a declared one
23([livery.workshop._phases.declared_phases][]), and a step of a phase
24nobody declares refuses naming the declared ones. A phase that
25declares a verb is a verb and every step a sub-task of it, made by
26[livery.workshop._steps.generate_verbs][] from the registry: ``fm
27test`` runs the test phase's steps and ``fm test.pytest`` the one;
28the gate runs the steps of the phases that declare themselves gated.
30The gate runs the registered steps through the one walk
31([livery.workshop._phases.walk_steps][]), each a footman step: the
32rewriters one at a time before any judge reads the tree
33([livery.workshop._steps.rewriters][]), then every judge in one
34block ([livery.workshop._steps.judges][]), each with its own record.
35A step that must follow another on the same package, a build after
36its configure, names it in ``after``: the walk starts it once the
37earlier one ran, and the gate brings the earlier one in
38([livery.workshop._steps.with_earlier][]).
39"""
56 from livery.workshop._categories import CategoryRule
57 from livery.workshop._influence import Changes, Inputs, Selection
58 from livery.workshop._packages import Package
59 from livery.workshop._phases import PhaseContext
60 from livery.workshop._words import Words
62#: A step's scope: the whole workspace in one run, or one package at a time.
66#: How a workspace step narrows under a package-scoped gate.
71#: The extension the builtin records belong to.
77 """One option a package may set on a step, in the step's own table.
79 The table is ``[phases.<phase>.<step>]``, the step's address; a
80 ``[phases.<phase>]`` table sets an option on every step of the
81 phase.
83 Attributes:
84 name: The option's key, kebab-case.
85 kind: ``bool``, ``str``, ``int`` or ``strs``, a list of
86 strings; a value of another type refuses naming the option.
87 default: What the step reads when the package says nothing.
88 doc: One line saying what the option changes.
89 """
97#: Every step carries this option: a package that turns a step off
98#: is skipped by name in the gate's output.
104 """One category a step judges, and the rules it withholds there.
106 Attributes:
107 category: The category, ``source``, ``test``, ``configuration``.
108 ignore: Rule codes the step does not apply to that category;
109 a tool with per-file ignores renders them as those.
110 suffixes: The file suffixes the step reads in the category,
111 ``(".py", ".pyi")`` for a python tool; empty reads every
112 file. A category names what a file is for, not its
113 language: a native package's ``source`` is C++ and its
114 ``conanfile.py`` is configuration, and the suffix keeps a
115 python tool's claim to the files it reads.
116 """
125 """One step, completely.
127 A step's name is its phase and its name in the phase,
128 ``test.pytest``: the verb ``fm test.pytest``, the
129 ``[phases.test.pytest]`` table of its options, the gate's lines,
130 and every lookup use it. An extension declares the step in its
131 ``[phases.<phase>.<step>]`` table.
133 Attributes:
134 step: The step's name in its phase, as its sub-task names it
135 under the phase's verb: ``pytest`` in ``fm test.pytest``.
136 phase: The phase the step belongs to: ``format``, ``lint``,
137 ``typecheck``, ``typecomplete``, ``test``, ``build``,
138 ``drift``, ``provenance`` or ``layering``, a declared
139 phase a verb is generated from.
140 run: The judging callable, called with the run's
141 [livery.workshop.PhaseContext][]: the engine running the
142 step's words or a reference to its code; a refusal is its
143 verdict.
144 scope: ``workspace`` for a step that judges the whole in one
145 run, ``package`` for one that judges one package at a
146 time.
147 narrowing: How a workspace step narrows under a scoped gate:
148 ``paths`` (the subset's directories), ``packages`` (the
149 subset's members) or ``none`` (the whole is always
150 judged). A package step narrows by its packages. No
151 declaration names it: the reader derives it, ``paths``
152 for a template with ``{paths}``, ``packages`` for a
153 workspace step in code with no ``inputs``, ``none``
154 otherwise ([livery.workshop._declaration.read][]).
155 fix: The rewriting callable when the tool can rewrite, run
156 serially before any judge under ``--fix``; None for a
157 step that only judges.
158 extensions: The extensions whose packages the step judges: a
159 package whose set holds one of them
160 ([livery.workshop.extension_set][]). A package step
161 runs per such member; a workspace step names them so its
162 tools and claims reach those packages, or leaves them empty.
163 tests_only: Whether a package step still runs when the
164 package's change is confined to its tests. A build and
165 the tests do; a formatter does not.
166 inputs: The files a workspace step reads, so a run judges it
167 only when one of them, or what widens it, changed, and then
168 only the changed ones when it judges per file
169 ([livery.workshop._influence.Inputs][]). None for a step
170 that narrows by its packages alone.
171 after: The steps this one runs after, by name: a build
172 after its configure, a ctest after its build. The walk
173 starts this one once each of them ran for the same
174 package, and the gate's walk brings them in
175 ([livery.workshop._steps.with_earlier][]).
176 extension: The extension that registered the step, named when a
177 narrowing is printed.
178 tools: The tools the step runs, as the tool profile names
179 them, ``("clang_format",)``; each reaches the profile while a
180 package the step judges exists, naming ``step <name>`` as its
181 site, and leaves it when the step is unregistered.
182 options: The options a package may set under
183 ``[phases.<phase>.<step>]``; ``enabled`` is every step's.
184 fragments: The configuration the render manages for the
185 step, one per rendered file it has something to say in;
186 composed in step-name order, judged by the drift gate,
187 gone from the next render with the record.
188 claims: The categories the step judges, each with the rules
189 it withholds there and the suffixes it reads; a step
190 judges every file its claims reach and no other, and a
191 tool with per-file ignores renders its ignores from them.
192 flags: The command-line flags the step reads, from `FLAGS`:
193 ``point`` for a step that selects tests by CI point,
194 ``shard`` for a step that runs its share of a sharded
195 check leg, and so runs on every shard; a step without it
196 runs on the first shard alone
197 ([livery.workshop._steps.on_this_shard][]). A generated
198 verb offers the flags its steps declare, and ``--fix``
199 with ``--safe-fix`` where a step has a fix mode.
200 arguments: Whether the step hands the tool it wraps the words
201 after ``--`` on its own verb (``fm test.pytest -- -k name``),
202 which it reads from [livery.workshop.PhaseContext][]'s
203 ``arguments``. A step that wraps no tool leaves it off, and
204 its verb refuses them. No declaration names it: the reader
205 derives it, on for a template with ``{arguments}`` and for
206 a step in code that names ``tools``.
207 words: The step's words when it is declared in words, which
208 ``fm explain`` prints and the conformance kit judges; None
209 for a step whose ``main`` is code.
210 provides: The context keys the step writes inside its phase,
211 each to its type name; a step of one phase reads what
212 another provides.
213 reads: The context keys the step reads inside its phase.
214 isolated: Whether the step serves the phase's isolated side
215 (True), from a copy of the sources in a fresh environment,
216 or its tree side (False); None serves both. The walk runs
217 the step on the side the engine gives the phase
218 ([livery.workshop._phases.serves][]), and the gate gives
219 the tree side.
220 enabled: Whether the step registers while its extension is
221 listed. An extension declares False for a step a workspace
222 turns on at the step's address in the root contract,
223 ``[phases.<phase>.<step>] enabled = true``.
224 replaces: The steps of the base or of an extension this one
225 requires that it replaces, each ``<owner>:<phase>.<step>``
226 with the reason; naming the step registered at this one's
227 address is what lets a second extension take the address.
228 deletes: The steps of the base or of an extension this one
229 requires that it deletes, each ``<owner>:<phase>.<step>``
230 with the reason.
231 outputs: The paths the step writes, each relative to the root
232 or ``package/<path>`` for the path in each package. When
233 [livery.workshop.run_phase][] runs the step's phase, an
234 output no running step declares is removed first, so a step
235 that stops running leaves nothing behind wherever it wrote.
237 """
267 """The step's name: its phase and its name in the phase, ``test.pytest``."""
268 return f"{self.phase}.{self.step}"
274#: The command-line flags a step may read, beyond the fix modes:
275#: ``point``, the CI point whose tests the step selects, and ``shard``,
276#: the shard of a sharded check leg the step runs its share of.
281 """Register *record*; a name already registered is replaced.
283 The mount registers each step a listed extension declares.
284 Replacing is how an extension swaps the step at an address, and
285 how a test injects a fake; a package step names at least one
286 extension, since a step that judges packages and applies to none
287 never runs. The step's address, ``<phase>.<step>``, is a task's
288 address, so a dot inside the phase or the step's name refuses, and
289 a step named ``default`` refuses (that address is the phase verb's
290 own).
291 """
292 from livery.workshop import _phases
294 for part in (record.phase, record.step):
295 if not part or "." in part:
296 fail(
297 f"step {record.name!r}: a phase and a step's name are one word"
298 f" each, not {part!r}; the step answers to <phase>.<step>"
299 )
300 declared = _phases.declared_phases()
301 if record.phase not in declared:
302 fail(
303 f"step {record.name!r}: {record.phase} is not a declared phase; the"
304 f" phases are {', '.join(declared)}"
305 )
306 if record.step == "default":
307 fail(
308 f"step {record.name!r}: {record.name} is the address of the"
309 f" {record.phase} verb itself; name the step by its tool"
310 )
311 held = _STEPS.get(record.name)
312 if held is not None and held.extension != record.extension:
313 # A second extension takes an address by saying so: the step it
314 # replaces or deletes, named with its owner, and why.
315 named = {address for address, _reason in (*record.replaces, *record.deletes)}
316 if f"{held.extension}:{held.name}" not in named:
317 fail(
318 f"step {record.name} is declared by both {held.extension} and"
319 f" {record.extension}; two extensions declare one step unless"
320 f" {record.extension} names {held.extension}:{held.name} in"
321 " replaces or deletes, with a reason"
322 )
323 if record.scope not in (WORKSPACE, PACKAGE):
324 fail(
325 f"step {record.name!r}: scope is {WORKSPACE!r} or {PACKAGE!r},"
326 f" not {record.scope!r}"
327 )
328 unknown = [flag for flag in record.flags if flag not in FLAGS]
329 if unknown:
330 fail(
331 f"step {record.name!r} reads {', '.join(unknown)}, which no verb"
332 f" offers; the flags are {', '.join(FLAGS)}"
333 )
334 if record.narrowing not in (PATHS, PACKAGES, NONE):
335 fail(
336 f"step {record.name!r}: narrowing is {PATHS!r}, {PACKAGES!r} or"
337 f" {NONE!r}, not {record.narrowing!r}"
338 )
339 if record.scope == PACKAGE and not record.extensions:
340 fail(
341 f"step {record.name!r} judges packages and names no extension: a"
342 " package step judges the packages whose set holds one it names"
343 )
344 try:
345 _fragments.verify(record.fragments, record.name)
346 except ValueError as error:
347 fail(str(error))
348 from livery.workshop._categories import known_categories
350 known = known_categories()
351 for claim in record.claims:
352 if claim.category not in known:
353 fail(
354 f"step {record.name!r} claims {claim.category!r}, which no category"
355 f" table knows; the categories are {', '.join(sorted(known))}"
356 )
357 _STEPS[record.name] = record
361 if option.kind == "strs":
362 return isinstance(value, list) and all(
363 isinstance(item, str) for item in cast("list[object]", value)
364 )
365 expected = {"bool": bool, "str": str, "int": int}[option.kind]
366 if option.kind == "int" and isinstance(value, bool):366 ↛ 367line 366 didn't jump to line 367 because the condition on line 366 was never true
367 return False
368 return isinstance(value, expected)
372 """The tables that reach *record*'s options, shallowest first.
374 ``phases.<phase>``, the phase's table, which reaches every step of
375 the phase, then ``phases.<phase>.<step>``, the step's own: the
376 later table wins key by key, since it names fewer steps.
377 """
378 return (f"phases.{record.phase}", f"phases.{record.phase}.{record.step}")
382 """What *package* sets for *record*'s option *name*, or the option's default.
384 The deepest table that sets it wins
385 ([livery.workshop._steps.option_addresses][]). Refuses an option
386 the record does not declare and a value of the wrong type, naming
387 the record's options.
388 """
389 declared = {option.name: option for option in (*record.options, ENABLED)}
390 if name not in declared:
391 fail(
392 f"step {record.name!r} declares no option {name!r}; its options are"
393 f" {', '.join(declared)}"
394 )
395 option = declared[name]
396 tables = dict(package.step_options)
397 found: object = option.default
398 for address in option_addresses(record):
399 for key, value in tables.get(address, ()):
400 if key != name:
401 continue
402 if not _typed(option, value):
403 fail(
404 f"{package.path}/workshop.toml: [{address}] {name} is"
405 f" {option.kind}, not {value!r}"
406 )
407 found = value
408 return found
412 """The registered steps a table *address* reaches.
414 ``phases.<phase>`` reaches every step of the phase, and
415 ``phases.<phase>.<step>`` the one step.
416 """
417 _kind, _, rest = address.partition(".")
418 phase, _, step = rest.partition(".")
419 return tuple(
420 r for r in _STEPS.values() if r.phase == phase and (not step or r.step == step)
421 )
425 """Every option table or key no registered step takes, one line each.
427 Read by ``layering.graph``, so a package that addresses a phase or
428 a step that does not exist, or sets an option none of the steps it
429 reaches declares, is refused with the vocabulary named.
430 """
431 problems: list[str] = []
432 known = phases()
433 for package in packages:
434 where = f"{package.path}/workshop.toml"
435 for address, options in package.step_options:
436 reached = _reached(address)
437 if not reached:
438 _kind, _, rest = address.partition(".")
439 if "." in rest:
440 problems.append(
441 f"{where}: [{address}] reaches no registered step; the"
442 f" steps are {', '.join(sorted(_STEPS))}"
443 )
444 else:
445 problems.append(
446 f"{where}: [{address}] names no phase a registered step"
447 f" has; the phases are {', '.join(known)}"
448 )
449 continue
450 declared: dict[str, Option] = {ENABLED.name: ENABLED}
451 for record in reached:
452 declared.update({option.name: option for option in record.options})
453 for key, value in options:
454 option = declared.get(key)
455 if option is None:
456 problems.append(
457 f"{where}: [{address}] sets {key!r}, which no step it"
458 f" reaches declares; their options are"
459 f" {', '.join(sorted(declared))}"
460 )
461 elif not _typed(option, value):
462 problems.append(
463 f"{where}: [{address}] {key} is {option.kind}, not {value!r}"
464 )
465 return problems
469 """*packages* minus those that turned the step *name* off, each skip printed."""
470 record = step_for(name)
471 kept: list[Package] = []
472 for package in packages:
473 if option_value(record, package, "enabled"):
474 kept.append(package)
475 else:
476 print(
477 f" {name}: {package.path} skips (turned off in"
478 f" {package.path}/workshop.toml)"
479 )
480 return tuple(kept)
484 """The tools the steps judging packages that hold *extensions* run.
486 Each as a ``(tool, step)`` pair. A step judges such a package
487 when it names one of *extensions*; a workspace step names the
488 extensions its bodies judge the same way.
489 """
490 held = set(extensions)
491 found: list[tuple[str, str]] = []
492 for record in _STEPS.values():
493 if held & set(record.extensions):
494 found.extend((tool, record.name) for tool in record.tools)
495 return tuple(found)
499 """Drop the step *name*; unknown names refuse naming the registry."""
500 if name not in _STEPS:
501 fail(f"{name!r} is not a registered step; steps: {', '.join(_STEPS)}")
502 del _STEPS[name]
506 """The gate's lines naming each step an extension registered.
508 A step the base did not register names its extension. Empty for the
509 base alone.
510 """
511 return tuple(
512 f" {record.name}: registered by {record.extension}"
513 for record in _STEPS.values()
514 if record.extension != BASE_EXTENSION
515 )
519 """Every registered step, in registration order."""
520 return tuple(_STEPS)
524 """The registry's state, for [livery.workshop._steps.restore][] to put back.
526 With *initial*, the state importing this module leaves instead: the
527 builtin steps alone, as a process starts with them.
528 """
529 return dict(_BUILTIN_STATE if initial else _STEPS)
533 """Put the registry back to *state*."""
534 _STEPS.clear()
535 _STEPS.update(state)
539 """Whether the claim admits the file's suffix; no suffixes admits every file."""
540 return not claim.suffixes or PurePosixPath(relative).suffix in claim.suffixes
544 """Whether a category pattern can name a file the claim reads.
546 A pattern ending in a literal suffix outside the claim's, cpp's
547 ``tests/**/*.cpp`` against a python claim, renders no ignore; a
548 directory pattern renders as it is, since the tool reads only its
549 own files under it.
550 """
551 return not PurePosixPath(pattern).suffix or _reaches(claim, pattern)
555 """The files git holds under the package: tracked, or untracked and not ignored.
557 Sorted, relative to the package. A build tree or a cache under the
558 package is ignored, so no claim reaches it; a tracked file deleted
559 from the tree is left out.
560 """
561 names = git.ls_files(package.directory, others=True)
562 return tuple(name for name in names if (package.directory / name).is_file())
566 """The files of *package* the step's claims reach, relative to it, sorted.
568 Every file git holds under the package directory is categorised
569 and kept when a claim names its category and admits its suffix; a
570 record without claims judges nothing by this measure, which is
571 what a step that runs a tool over directories looks like from
572 here.
573 """
574 from livery.workshop._categories import category_of
576 if not record.claims or not package.directory.is_dir():
577 return ()
578 found: list[str] = []
579 for relative in _package_files(package):
580 category = category_of(package, relative).name
581 if any(c.category == category and _reaches(c, relative) for c in record.claims):
582 found.append(relative)
583 return tuple(found)
587 """The steps whose claims reach *path* in *package*, by name, sorted."""
588 from livery.workshop._categories import category_of
590 category = category_of(package, path).name
591 names = []
592 for record in _STEPS.values():
593 if not any(c.category == category and _reaches(c, path) for c in record.claims):
594 continue
595 if record.extensions and not _applies(record, package):
596 continue
597 names.append(record.name)
598 return tuple(sorted(names))
602 held: Iterable[Iterable[str]],
603) -> list[tuple[str, tuple[str, ...]]]:
604 """The per-file ignores the claims render, pattern to rule codes, sorted.
606 *held* is each set of extensions a present package holds. For
607 each set, every claim of a step that judges a package
608 holding it and withholds rules on a category maps to that
609 category's patterns in the tables that answer for such a package,
610 each under ``packages/**/``, which reaches a member in a group
611 directory too. For the workspace's own unit, every step's claims map
612 to its table's patterns as they are. Two steps claiming one category
613 under different rules land in one entry with both sets.
614 """
615 from livery.workshop._categories import category_rules, workspace_rules
617 table: dict[str, set[str]] = {}
618 units: list[tuple[frozenset[str] | None, tuple[CategoryRule, ...], str]] = []
619 for listed in held:
620 names = tuple(listed)
621 units.append((frozenset(names), category_rules(names), "packages/**/"))
622 units.append((None, workspace_rules(), ""))
623 for holding, rules, prefix in units:
624 for record in _STEPS.values():
625 if holding is not None and not judges_extensions(record, holding):
626 continue
627 for claim in record.claims:
628 if not claim.ignore:
629 continue
630 for rule in rules:
631 if rule.category != claim.category or rule.pattern == "**":
632 continue
633 if not _admits(claim, rule.pattern):
634 continue
635 table.setdefault(prefix + rule.pattern, set()).update(claim.ignore)
636 return [(pattern, tuple(sorted(codes))) for pattern, codes in sorted(table.items())]
640 """Every registered record by name, in registration order."""
641 return dict(_STEPS)
645 """The step answering to *address*, its name; None when none is registered."""
646 return address if address in _STEPS else None
650 """The step named *name*, ``test.pytest``.
652 Refusal names the registry.
653 """
654 if name not in _STEPS:
655 fail(f"{name!r} is not a registered step; steps: {', '.join(_STEPS)}")
656 return _STEPS[name]
660 """The phases the registered steps belong to, sorted."""
661 return tuple(sorted({r.phase for r in _STEPS.values()}))
665 """Whether *record* judges a package holding *extensions*.
667 A step that names no extensions judges every package; one that
668 names some judges a package holding one of them.
669 """
670 return not record.extensions or bool(set(record.extensions) & set(extensions))
674 """Whether *record* judges *unit*, by the extensions the unit holds.
676 A member holds its set, and the workspace's own tests hold the
677 root's set (`extension_set`). A step that names no extension judges
678 every unit; one that names some judges a unit holding one of them.
679 """
680 from livery.workshop._composition import extension_set
682 return judges_extensions(record, extension_set(unit))
686 record: StepRecord, packages: tuple[Package, ...], *, quiet: bool = False
687) -> tuple[Package, ...]:
688 """The *packages* *record* judges, by the extensions they hold; each other skips.
690 A package holding none of the step's extensions prints a skip
691 naming the step, the package and the extensions, so a narrowed gate
692 is visible in the output and never passes silently. *quiet* counts
693 without printing.
694 """
695 kept: list[Package] = []
696 for package in packages:
697 if judges_unit(record, package):
698 kept.append(package)
699 elif not quiet:
700 wanted = " or ".join(record.extensions)
701 print(f" {record.name}: {package.path} skips (not a {wanted} package)")
702 return tuple(kept)
706 """Whether a package step judges *package*, by the extensions it holds."""
707 from livery.workshop._composition import extension_set
709 return bool(set(record.extensions) & set(extension_set(package)))
713 """Whether the package step *record* runs over *package* in this run.
715 The package holds an extension the step names; a change confined
716 to the package's tests leaves the step out unless it is tests-only
717 ([livery.workshop._steps.StepRecord][]).
718 """
719 tests_only = ctx.scoped and package.path in ctx.tests
720 return (
721 record.scope == PACKAGE
722 and _applies(record, package)
723 and (record.tests_only or not tests_only)
724 )
728 """The packages under judgment that are members: every one but the tests unit."""
729 from livery.workshop._packages import WORKSPACE_TESTS
731 return tuple(package for package in ctx.judged if package.path != WORKSPACE_TESTS)
735 """Whether *phase* gets a verb: it declares one."""
736 from livery.workshop import _phases
738 spec = _phases.declared_phases().get(phase)
739 return spec is None or spec.verb
743 """Each phase's steps by name, sorted: the verbs the registry generates.
745 A phase that declares no verb is left out.
746 """
747 tree: dict[str, dict[str, StepRecord]] = {}
748 for record in _STEPS.values():
749 if _makes_verb(record.phase):
750 tree.setdefault(record.phase, {})[record.step] = record
751 return {phase: dict(sorted(steps.items())) for phase, steps in sorted(tree.items())}
754#: The verb functions the generator made. Footman shares a task's
755#: function when it copies a tree into a project, so the function is
756#: how a later generation knows a verb as its own: its own it remakes
757#: or removes, and any other task at an address serves that address.
762 """A verb's positional words as its paths, then the words after ``--``.
764 footman hands a task's variadic parameter the words after ``--``
765 too, last, and names them through ``passthrough()``.
766 """
767 from livery.footman import passthrough
769 after = tuple(passthrough())
770 if after and paths[-len(after) :] == after:
771 return paths[: -len(after)], after
772 return paths, after
776 names: tuple[str, ...],
777 words: tuple[str, ...],
778 *,
779 refusal: str,
780 fix: bool = False,
781 safe_fix: bool = False,
782 point: str = "",
783 shard: str = "",
784) -> None:
785 from livery.workshop import _quality
787 paths, arguments = split_arguments(words)
788 if arguments and refusal:
789 fail(refusal)
790 _quality.run_steps(
791 names,
792 paths,
793 fix=fix,
794 safe_fix=safe_fix,
795 point=point,
796 shard=shard,
797 arguments=arguments,
798 )
801#: A generated verb's paths and flags, each with its help text. They live at
802#: module level because footman reads a task's annotations in its
803#: module's namespace: a name local to the function making the verb
804#: does not resolve there, and the flag's value would arrive as text.
806 str, doc("the files or directories to check; the workspace when none")
807]
811 str, doc("select this CI point's tests: gate, merge, nightly, release")
812]
814 str, doc("run this shard's share of a sharded check leg: <i>of<n>, such as 2of4")
815]
819 select: Callable[[], tuple[str, ...]],
820 *,
821 fixes: bool,
822 flags: frozenset[str],
823 what: str,
824 refusal: str = "",
825) -> Callable[..., None]:
826 """A verb over the steps *select* names, offering exactly the flags they read.
828 ``--fix`` and ``--safe-fix`` where a step can fix, ``--point`` and
829 ``--shard`` where one reads them (*flags*, from `FLAGS`); the
830 paths narrow the run to those files. The words after ``--`` reach
831 the step's tool, unless *refusal* says why this verb takes none.
832 One body per set of flags, because footman reads a verb's flags
833 from its signature, and a signature is written, not computed.
834 """
836 def run(
837 paths: tuple[str, ...],
838 *,
839 fix: bool = False,
840 safe_fix: bool = False,
841 point: str = "",
842 shard: str = "",
843 ) -> None:
844 _run_named(
845 select(),
846 paths,
847 refusal=refusal,
848 fix=fix,
849 safe_fix=safe_fix,
850 point=point,
851 shard=shard,
852 )
854 def bare(*paths: _Paths) -> None:
855 run(paths)
857 def with_point(*paths: _Paths, point: _Point = "") -> None:
858 run(paths, point=point)
860 def with_shard(*paths: _Paths, shard: _Shard = "") -> None:
861 run(paths, shard=shard)
863 def with_point_and_shard(
864 *paths: _Paths, point: _Point = "", shard: _Shard = ""
865 ) -> None:
866 run(paths, point=point, shard=shard)
868 def with_fix(*paths: _Paths, fix: _Fix = False, safe_fix: _SafeFix = False) -> None:
869 run(paths, fix=fix, safe_fix=safe_fix)
871 def with_fix_and_point(
872 *paths: _Paths,
873 fix: _Fix = False,
874 safe_fix: _SafeFix = False,
875 point: _Point = "",
876 ) -> None:
877 run(paths, fix=fix, safe_fix=safe_fix, point=point)
879 def with_fix_and_shard(
880 *paths: _Paths,
881 fix: _Fix = False,
882 safe_fix: _SafeFix = False,
883 shard: _Shard = "",
884 ) -> None:
885 run(paths, fix=fix, safe_fix=safe_fix, shard=shard)
887 def with_fix_point_and_shard(
888 *paths: _Paths,
889 fix: _Fix = False,
890 safe_fix: _SafeFix = False,
891 point: _Point = "",
892 shard: _Shard = "",
893 ) -> None:
894 run(paths, fix=fix, safe_fix=safe_fix, point=point, shard=shard)
896 bodies: dict[tuple[bool, bool, bool], Callable[..., None]] = {
897 (False, False, False): bare,
898 (False, True, False): with_point,
899 (False, False, True): with_shard,
900 (False, True, True): with_point_and_shard,
901 (True, False, False): with_fix,
902 (True, True, False): with_fix_and_point,
903 (True, False, True): with_fix_and_shard,
904 (True, True, True): with_fix_point_and_shard,
905 }
906 chosen = bodies[(fixes, "point" in flags, "shard" in flags)]
907 chosen.__doc__ = what
908 _MADE.add(chosen)
909 return chosen
913 """Every registered step of *phase*, read when the verb runs."""
915 def select() -> tuple[str, ...]:
916 return tuple(record.name for record in _STEPS.values() if record.phase == phase)
918 return select
922 def select() -> tuple[str, ...]:
923 return (name,)
925 return select
929 """Whether the generator made *task*, through the function footman wraps."""
930 return inspect.unwrap(task) in _MADE
934 """The flags a verb offers: its keyword parameters."""
935 return frozenset(
936 name
937 for name, parameter in inspect.signature(task).parameters.items()
938 if parameter.kind is inspect.Parameter.KEYWORD_ONLY
939 )
943 """Put *verb* at *key* in *parent*, unless a verb that serves it is there.
945 A verb the generator made is remade when its flags changed, so a
946 step registered later that fixes or reads the point reaches its
947 phase's verb; any other task at the address is left alone.
948 """
949 existing = parent.tasks.get(key)
950 if existing is not None:
951 if not _is_made(existing) or _flags_of(existing) == _flags_of(verb):
952 return
953 del parent.tasks[key]
954 if key == "default":
955 parent.default(verb)
956 else:
957 parent.task(name=key)(verb)
961 """Remove the verbs the generator made whose steps are gone.
963 A phase with no step left loses its verb, and its group when that
964 leaves the group empty; a task another extension put there stays.
965 """
966 for phase, parent in list(target.groups.items()):
967 steps = tree.get(phase, {})
968 removed = False
969 for key, task in list(parent.tasks.items()):
970 kept = bool(steps) if key == "default" else key in steps
971 if _is_made(task) and not kept:
972 del parent.tasks[key]
973 removed = True
974 if removed and not parent.tasks and not parent.groups:
975 del target.groups[phase]
979 """Make a verb per phase and a sub-task per step, from the registry.
981 ``fm test`` runs every step of the test phase and ``fm test.pytest``
982 the one, each offering exactly the flags its steps read. A phase
983 or a sub-task whose address a verb already holds is served by that
984 verb, ``fm provenance``, and nothing is
985 made there. Run again after extensions registered or withdrew steps,
986 it makes what is new, remakes a verb whose flags changed, and
987 removes a verb whose step is gone.
988 """
989 target = into if into is not None else root_group
990 tree = verb_tree()
991 _prune(target, tree)
992 for phase, steps in tree.items():
993 if phase in target.tasks:
994 continue
995 parent = target.groups.get(phase)
996 default = None if parent is None else parent.default_task
997 # The generator owns the phase's verb in a group it made; a
998 # group of another verb's gets no default.
999 owned = parent is None or (default is not None and _is_made(default))
1000 if parent is None:
1001 parent = target.group(phase, help=f"The {phase} steps, one task each")
1002 if owned:
1003 _place(
1004 parent,
1005 "default",
1006 _verb(
1007 _steps_of(phase),
1008 fixes=any(r.fix is not None for r in steps.values()),
1009 flags=frozenset(f for r in steps.values() for f in r.flags),
1010 what=f"Run every {phase} step, over the named paths or the"
1011 " workspace.",
1012 refusal=arguments_refusal(f"{prog()} {phase}", steps.values()),
1013 ),
1014 )
1015 for step, record in steps.items():
1016 if step in parent.groups:1016 ↛ 1017line 1016 didn't jump to line 1017 because the condition on line 1016 was never true
1017 continue
1018 _place(
1019 parent,
1020 step,
1021 _verb(
1022 _one_step(record.name),
1023 fixes=record.fix is not None,
1024 flags=frozenset(record.flags),
1025 what=f"Run the {record.name} step alone, over the named paths"
1026 " or the workspace"
1027 + (
1028 "; the words after -- go to its tool."
1029 if record.arguments
1030 else "."
1031 ),
1032 refusal=""
1033 if record.arguments
1034 else f"{record.name} wraps no tool, so it takes no arguments"
1035 " after --",
1036 ),
1037 )
1041 """Why *verb*, which runs several steps, takes no arguments after ``--``.
1043 The words after ``--`` are one tool's, so they go to a step's own
1044 verb; the refusal names those of *records* that take them.
1045 """
1046 takers = [f"{prog()} {r.phase}.{r.step}" for r in records if r.arguments]
1047 named = f": {', '.join(takers)}" if takers else ""
1048 return (
1049 f"`{verb}` runs several steps, and the words after -- are one tool's;"
1050 f" give them to the step's own verb{named}"
1051 )
1055 """Run the step *name* over *ctx* through the walk: its judge, or its fix mode.
1057 One step alone ([livery.workshop._phases.walk_steps][]): a workspace
1058 step runs once, a package step once per package it judges, and
1059 a step without a fix mode runs nothing in fix mode. The steps
1060 named in ``after`` are not run first: a caller running several
1061 steps by hand orders them itself, and the gate's walk brings them
1062 in ([livery.workshop._steps.with_earlier][]).
1063 """
1064 from livery.workshop._phases import walk_steps
1066 walk_steps(ctx, (step_for(name),), fixing=fix)
1070 """The records of *names*, and of every step one of them runs after.
1072 Followed through ``after`` transitively, in registration order,
1073 so a build's configure joins the walk that judges the build. An
1074 ``after`` naming a step an installed extension declares and the
1075 mount did not register orders nothing: clang-tidy's lint runs after
1076 cmake's configure in a workspace whose packages hold cmake, and
1077 alone in one whose packages do not.
1079 Raises:
1080 Failed: when an ``after`` names a step that neither the base nor
1081 an installed extension declares, naming the registry.
1082 """
1083 wanted = set(names)
1084 queue = list(names)
1085 while queue:
1086 for earlier in step_for(queue.pop()).after:
1087 if earlier in wanted:
1088 continue
1089 if earlier not in _STEPS and installed_step(earlier):
1090 continue
1091 # A name nothing declares joins the queue, and its lookup
1092 # refuses naming the registry.
1093 wanted.add(earlier)
1094 queue.append(earlier)
1095 return tuple(record for record in _STEPS.values() if record.name in wanted)
1099 """Whether an installed extension or the base declares the step *name*.
1101 *name* is ``<phase>.<step>``. Read from the declarations, so a step
1102 the mount did not register, one of an extension the workspace does
1103 not mount, is declared all the same.
1104 """
1105 from livery.workshop._declaration import installed_steps
1107 return any(address.partition(":")[2] == name for address in installed_steps())
1111 if record.scope == WORKSPACE:
1112 if record.inputs is not None:
1113 # Listed, so the walk says when no file it reads changed.
1114 return True
1115 # A step without declared inputs narrows by its packages, so a
1116 # scope that holds none leaves it nothing to judge. Running it
1117 # would read the empty scope as the whole: the test step would
1118 # start the whole suite. Named files are a scope of their own,
1119 # which the step's claims judge.
1120 return bool(ctx.files) or not (ctx.scoped and not ctx.subset)
1121 return any(judges_package(record, ctx, p) for p in members(ctx))
1125 """The workspace steps with declared inputs that this run's changes select."""
1126 return tuple(
1127 record.name
1128 for record in _STEPS.values()
1129 if record.scope == WORKSPACE
1130 and record.inputs is not None
1131 and selected(record, ctx).runs
1132 )
1136 """The files the running step judges in this run; None for every file it reads.
1138 A workspace step that declares ``inputs`` judges, in a run that
1139 knows what changed, the changed files among its inputs, and what
1140 its ``widen`` reference adds; in a run that knows nothing of the
1141 changes, it judges every file it reads, and this is None. The
1142 context names the running step, so a step asks with the context
1143 its run was handed.
1145 Raises:
1146 Failed: when *ctx* names no step: one built by hand, outside
1147 a step's run.
1148 """
1149 if not ctx.step:
1150 fail(
1151 "selected_files() answers for the running step, and this"
1152 " PhaseContext names none: call it with the context a step's"
1153 " run is handed, which sets PhaseContext.step"
1154 )
1155 return _selected_files(step_for(ctx.step), ctx)
1159 """The files *record* judges in this run; None for every file it reads."""
1160 selection = selected(record, ctx)
1161 return None if selection.whole else frozenset(selection.files)
1164#: What never counts as a manifest or a composed file's input, though it
1165#: sits beside them: prose, and the licence.
1170 """The files directly in each package's directory, as patterns.
1172 The contract and the native manifests are among them, whatever the
1173 package's kind calls its manifest, so the base names none of them.
1174 """
1175 from livery.workshop._packages import package_directories
1177 return tuple(
1178 f"{directory.relative_to(root).as_posix()}/*"
1179 for directory in package_directories(root)
1180 )
1184 """What the graph is made of: the contracts, the native manifests, a root src/.
1186 The layering steps over the sources widen on them: a change there
1187 moves what the rules read, so every source is judged again.
1188 """
1189 return ("workshop.toml", "src/**", *package_files(root))
1193 """Tell whether a contract, a native manifest or a root ``src/`` changed.
1195 A step over a language's sources widens on it through its
1196 ``widen`` reference, a function of its own extension that calls
1197 this one: the members, their module roots and what they declare
1198 come from these files, so a change to one judges every source
1199 again, as the base's own layering steps do. Prose and the licence
1200 beside a manifest never count.
1202 Args:
1203 changes: What the run changed, as a step's ``widen`` reference
1204 receives it.
1206 Returns:
1207 True when a changed path is the root contract, a file directly
1208 in a member's directory, or a file under ``src/`` at the root.
1209 """
1210 from livery.workshop._categories import matches
1212 patterns = graph_files(changes.root)
1213 return any(
1214 matches(pattern, path)
1215 for path in changes.paths
1216 if not any(matches(ignored, path) for ignored in NOT_INPUTS)
1217 for pattern in patterns
1218 )
1221#: The files the generated outputs land in, for every forge: the CI
1222#: files, the entry script and the code-owners file.
1224 ".github/workflows/*.yml",
1225 ".gitea/workflows/*.yml",
1226 ".gitlab-ci.yml",
1227 "setup.sh",
1228 ".github/CODEOWNERS",
1229 ".gitlab/CODEOWNERS",
1230 ".gitea/CODEOWNERS",
1231 "CODEOWNERS",
1232 "docs/CODEOWNERS",
1233)
1237 """The tracked outputs drift judges: the receipted composed ones and the generated.
1239 Read from the render's receipts, never by rendering: listing what
1240 is composed must cost less than composing it.
1241 """
1242 from livery.workshop._fragment_engine import read_rendered
1243 from livery.workshop._packages import package_directories
1245 found = list(GENERATED)
1246 found += list(read_rendered(root))
1247 for directory in package_directories(root):
1248 home = directory.relative_to(root).as_posix()
1249 found += [f"{home}/{name}" for name in read_rendered(directory)]
1250 return tuple(found)
1254 """What every tracked output is made from: contracts, manifests, locks, extensions.
1256 A listed extension whose sources live in a member package composes
1257 from them, so a change under them may move any output.
1258 """
1259 from livery.workshop._extensions import extension_names, extension_provider
1260 from livery.workshop._packages import discover_packages
1262 packages = discover_packages(root)
1263 providers = {extension_provider(name, packages) for name in extension_names(root)}
1264 return (
1265 "workshop.toml",
1266 *package_files(root),
1267 "uv.lock",
1268 "toolroom.lock",
1269 *(f"{provider}/src/**" for provider in sorted(providers) if provider),
1270 )
1274 """What *record* judges in this run, by its declared inputs.
1276 Everything for a step that declares none. The run's changes
1277 select, or the named files of `fm check <paths>` when there are no
1278 changes; a run with neither judges everything.
1279 """
1280 from livery.workshop._extensions import extension_provider
1281 from livery.workshop._influence import WHOLE, Changes, select
1283 if record.inputs is None:1283 ↛ 1284line 1283 didn't jump to line 1284 because the condition on line 1283 was never true
1284 return WHOLE
1285 changes = ctx.changes
1286 if changes is None and ctx.files:
1287 changes = Changes(ctx.root, ctx.files)
1288 return select(
1289 record.inputs,
1290 changes,
1291 provider=extension_provider(record.extension, ctx.packages),
1292 )
1295#: The unit of every file under no package: the root's own files,
1296#: ``tasks.py`` and the workspace's tests among them.
1301 """Every file git holds under the root, by unit, each with its category.
1303 Listed once per walk: tracked files and untracked ones git does not
1304 ignore, as the tools see them, or the run's named files alone. A
1305 package's files are relative to it and categorised by the tables of
1306 the extensions it holds; a file under no package belongs to
1307 [livery.workshop._steps.ROOT_UNIT], relative to the root and
1308 categorised by the workspace's table. None when the root is not a
1309 git checkout, and then every step that applies runs.
1310 """
1311 from livery.workshop._categories import category_of
1312 from livery.workshop._packages import WORKSPACE_TESTS
1313 from livery.workshop._provenance import unit_of
1315 if ctx.files:
1316 names: set[str] = set(ctx.files)
1317 else:
1318 try:
1319 names = set(git.ls_files(ctx.root, others=True))
1320 except git.GitError:
1321 return None
1322 packages = tuple(p for p in ctx.packages if p.path != WORKSPACE_TESTS)
1323 found: dict[str, list[tuple[str, str]]] = {}
1324 for name in sorted(names):
1325 if not (ctx.root / name).is_file():1325 ↛ 1326line 1325 didn't jump to line 1326 because the condition on line 1325 was never true
1326 continue
1327 unit, inside = unit_of(ctx.root, packages, name)
1328 if unit is None:1328 ↛ 1329line 1328 didn't jump to line 1329 because the condition on line 1328 was never true
1329 continue
1330 key = unit.path if unit in packages else ROOT_UNIT
1331 found.setdefault(key, []).append((inside, category_of(unit, inside).name))
1332 return {key: tuple(files) for key, files in found.items()}
1336 """The units *record* reads in this run.
1338 A package step reads the packages it judges. A workspace step
1339 reads every unit in a whole gate; in a scoped one, the subset's
1340 packages and the root's own files, since the workspace's tests
1341 ride every scoped run.
1342 """
1343 if record.scope == PACKAGE:
1344 return tuple(
1345 package.path
1346 for package in members(ctx)
1347 if judges_package(record, ctx, package)
1348 )
1349 if ctx.subset is None or ctx.catalogue is None:
1350 return tuple(ctx.catalogue or ())
1351 return (*(package.path for package in members(ctx)), ROOT_UNIT)
1355 record: StepRecord, ctx: PhaseContext, units: tuple[str, ...] | None = None
1356) -> bool:
1357 """Whether *record*'s claims reach a file of *units*, or of its scope in this run.
1359 A step without claims reads what its own body decides, and a run
1360 without a catalogue has no listing to judge by: both answer yes,
1361 except over named files, which a step without claims cannot say
1362 it reads.
1363 """
1364 if record.inputs is not None:
1365 return selected(record, ctx).runs
1366 if not record.claims:
1367 return not ctx.files
1368 if ctx.catalogue is None:
1369 return True
1370 for unit in units if units is not None else _scope_units(record, ctx):
1371 for relative, category in ctx.catalogue.get(unit, ()):
1372 if any(
1373 c.category == category and _reaches(c, relative) for c in record.claims
1374 ):
1375 return True
1376 return False
1380 record: StepRecord, ctx: PhaseContext, unit: str | None = None
1381) -> tuple[str, ...]:
1382 """The files of the run's catalogue *record*'s claims reach, as absolute paths.
1384 For a run over named files: the ones a step takes. *unit* keeps
1385 one package's alone, for a package step.
1386 """
1387 found: list[str] = []
1388 for key, files in (ctx.catalogue or {}).items():
1389 if unit is not None and key != unit:
1390 continue
1391 base = ctx.root if key == ROOT_UNIT else ctx.root / key
1392 for relative, category in files:
1393 if any(
1394 c.category == category and _reaches(c, relative) for c in record.claims
1395 ):
1396 found.append(str(base / relative))
1397 return tuple(sorted(found))
1401 """*names* less the steps with no file to read in this run, each skip said.
1403 No process starts for a step whose claims reach nothing in scope.
1404 """
1405 kept: list[str] = []
1406 where = (
1407 "the named files"
1408 if ctx.files
1409 else "the affected packages"
1410 if ctx.scoped
1411 else "the workspace"
1412 )
1413 for name in names:
1414 record = _STEPS[name]
1415 if reads_files(record, ctx):
1416 kept.append(name)
1417 continue
1418 said = (
1419 "the changed files"
1420 if record.inputs is not None and ctx.changes is not None
1421 else where
1422 )
1423 print(f" {name}: no file it reads in {said}; not run")
1424 return tuple(kept)
1428 """*names* less the steps this shard leaves to the first shard, each said.
1430 A step that declares ``shard`` among its flags runs on every shard
1431 of a sharded check leg and runs its share, the shard in its
1432 context's ``shard``. A step that does not runs whole, so it runs
1433 on the first shard alone: on any other shard it is left out, and a
1434 line says so. On the first shard, and on a run that is one shard
1435 of one, every step stays and nothing is said.
1436 """
1437 if ctx.shard.first:
1438 return names
1439 kept: list[str] = []
1440 for name in names:
1441 if "shard" in _STEPS[name].flags:
1442 kept.append(name)
1443 continue
1444 print(f" {name}: left to the first shard (this leg is {ctx.shard})")
1445 return tuple(kept)
1449 """Whether the gate runs *record*: its phase declares itself gated."""
1450 from livery.workshop import _phases
1452 spec = _phases.declared_phases().get(record.phase)
1453 return spec is None or spec.gated
1457 """The steps whose fix mode runs, serially and in order, under --fix.
1459 Those of the gated phases; empty unless the gate runs in its fix
1460 mode. They run in registration order, the base's first and then
1461 each extension's in composition order, with each step after the
1462 ones it names in ``after`` and the providers of the keys it reads.
1464 Raises:
1465 OrderCycle: when the rewriters' order has no start, naming the
1466 members of one cycle.
1467 """
1468 if not ctx.fix:
1469 return ()
1470 from livery.workshop._phases import phase_order
1472 records = [
1473 record
1474 for record in _STEPS.values()
1475 if record.fix is not None and _gated(record) and _applies_now(record, ctx)
1476 ]
1477 return tuple(record.name for record in phase_order(records))
1481 """The steps the gate judges together, in registration order.
1483 Those of the gated phases. A step that rewrote in this run is not
1484 judged again: re-judging the style it just wrote would only spend
1485 time agreeing.
1486 """
1487 names = []
1488 for record in _STEPS.values():
1489 if not _gated(record) or not _applies_now(record, ctx):
1490 continue
1491 if record.fix is not None and ctx.fix:
1492 continue
1493 names.append(record.name)
1494 return tuple(names)
1497#: The whole repository, as a step that walks it from the root reads it.
1498#: [livery.workshop._steps.scoped_paths][] answers it when a step
1499#: judges its configured whole; a tool that reads ``.`` as more than
1500#: its configuration names takes no paths then.
1505 """The workspace's own tests, when this run judges them as a unit."""
1506 from livery.workshop._packages import WORKSPACE_TESTS
1508 return tuple(p for p in ctx.judged if p.path == WORKSPACE_TESTS)
1512 """The files of the native members the step *name* claims, by path.
1514 A native member's directories are not a python step's to walk, so
1515 its files come by name: its ``conanfile.py``, say.
1516 """
1517 record = step_for(name)
1518 return tuple(f"{p.path}/{f}" for p in natives for f in judged_files(record, p))
1522 """Whether a scoped run over *chosen* units reaches every unit *record* judges.
1524 Every member the step judges, the workspace's tests unit among
1525 them while the step judges it, with none turned off anywhere: the
1526 whole would reach the same, so the step runs over its configured
1527 whole instead of a list naming every unit.
1528 """
1529 from livery.workshop._packages import WORKSPACE_TESTS
1531 everyone = tuple(p for p in ctx.packages if p.path != WORKSPACE_TESTS)
1532 gated_all = judged_by(record, everyone, quiet=True)
1533 # The workspace's tests unit rides in the scope, not always among
1534 # the packages: counted once from either.
1535 suites = tuple(p for p in (*ctx.packages, *ctx.judged) if p.path == WORKSPACE_TESTS)
1536 tests_unit = {p.path for p in judged_by(record, suites, quiet=True)}
1537 units = len(gated_all) + len(tests_unit)
1538 on = all(option_value(record, p, "enabled") for p in gated_all)
1539 return on and 0 < units <= chosen
1543 """The paths the path-narrowing step *name* judges in this run.
1545 The files the run names, when it names any. Otherwise the whole
1546 tree, ``.``, when the run reaches every member the step judges;
1547 else the ``src`` and ``tests`` directories of the reached members
1548 holding the runtime's language, the workspace's own tests when the
1549 run and the step judge them, and the files the step's claims give
1550 it in the other reached members.
1551 """
1552 from livery.workshop._packages import holds_runtime_language, package_paths
1554 record = step_for(name)
1555 if ctx.files:
1556 return claimed_files(record, ctx)
1557 gated_members = judged_by(record, members(ctx))
1558 chosen_members = enabled(name, gated_members)
1559 pythons = tuple(p for p in chosen_members if holds_runtime_language(p))
1560 natives = tuple(p for p in chosen_members if not holds_runtime_language(p))
1561 unit = judged_by(record, _workspace_unit(ctx), quiet=True)
1563 def holding(packages: tuple[Package, ...]) -> tuple[str, ...]:
1564 # A directory with no file the claims reach has nothing to
1565 # judge, and a tool such as mypy refuses one it is handed.
1566 return tuple(
1567 path
1568 for path in package_paths(packages)
1569 if _holds_claimed(record, ctx.root / path)
1570 )
1572 if ctx.subset is None:
1573 if len(chosen_members) == len(gated_members):1573 ↛ 1575line 1573 didn't jump to line 1575 because the condition on line 1573 was always true
1574 return WHOLE
1575 chosen = holding(pythons + unit) + _claimed(name, natives)
1576 return chosen or WHOLE
1577 judged = tuple(p for p in ctx.subset if p in pythons or p in unit)
1578 present = tuple(p for p in natives if p in ctx.subset)
1579 if _every_unit_reached(ctx, record, len(judged) + len(present)):
1580 return WHOLE
1581 return holding(judged) + _claimed(name, present)
1585 """Whether *directory* holds a file with a suffix *record*'s claims read.
1587 A step with no claim, or a claim that names no suffix, reads every
1588 file, so any directory holds one.
1589 """
1590 if not record.claims or any(not claim.suffixes for claim in record.claims):1590 ↛ 1591line 1590 didn't jump to line 1591 because the condition on line 1590 was never true
1591 return True
1592 suffixes = sorted({suffix for claim in record.claims for suffix in claim.suffixes})
1593 return any(next(directory.rglob(f"*{suffix}"), None) for suffix in suffixes)
1597 ctx: PhaseContext, name: str, package: Package | None = None
1598) -> tuple[Path, ...]:
1599 """The files of *package* the step *name* judges in this run, absolute.
1601 *package* is the run's own package unless named: a package step
1602 names none, and a step that narrows by packages names each one
1603 [livery.workshop.scoped_packages][] answers. The files the run
1604 names, when it names any; else every file the step's claims reach
1605 in the package, an untracked new one among them.
1607 Raises:
1608 Failed: when neither *package* nor the run names a package.
1609 """
1610 record = step_for(name)
1611 package = package or ctx.package
1612 if package is None:1612 ↛ 1613line 1612 didn't jump to line 1613 because the condition on line 1612 was never true
1613 fail(f"{name} judges one package at a time, and this run names none")
1614 if ctx.files:
1615 return tuple(Path(path) for path in claimed_files(record, ctx, package.path))
1616 return tuple(package.directory / path for path in judged_files(record, package))
1620 """The members the package-narrowing step *name* judges in this run.
1622 The members holding an extension it names, in a scoped run those
1623 of the subset, less each one that turned the step off in its
1624 contract. Each skip is printed, so a narrowed gate names what it
1625 left out. A run over named files keeps the members holding a named
1626 file the step claims. The workspace's own tests are not a member.
1627 """
1628 record = step_for(name)
1629 chosen = enabled(name, judged_by(record, members(ctx)))
1630 if not ctx.files:
1631 return chosen
1632 return tuple(p for p in chosen if claimed_files(record, ctx, p.path))
1636 """The workspace's own tests, when the step *name* judges them.
1638 The unit ([livery.workshop.workspace_suite][]) holds the root's
1639 set, so the step judges it while that set holds an extension the
1640 step names. Its tests reach any package, so every run of the step
1641 runs them then. None when the workspace has no ``tests``
1642 directory. None as well when the step does not judge the unit, and
1643 then a line says why.
1644 """
1645 from livery.workshop._packages import workspace_suite
1647 record = step_for(name)
1648 unit = workspace_suite(ctx.root)
1649 if unit is None or judges_unit(record, unit):
1650 return unit
1651 wanted = " or ".join(record.extensions)
1652 which = "it" if len(record.extensions) == 1 else "one"
1653 print(
1654 f" {name}: {unit.path} skips (the root's set holds no {wanted};"
1655 f" list {which} in [workspace] extensions to run them)"
1656 )
1657 return None
1661 """The step *name*: its registered record, else the installed declaration's.
1663 A process that mounted nothing holds the base's own steps alone,
1664 so an extension's step is read from the installed extension that
1665 declares it, the first by name (`installed_record` in the
1666 extensions module).
1668 Raises:
1669 Failed: when no installed extension declares the step, naming it.
1670 """
1671 record = _STEPS.get(name)
1672 if record is not None:
1673 return record
1674 from livery.workshop._extensions import installed_record
1676 found = installed_record(name)
1677 if found is None:
1678 fail(
1679 f"no installed extension declares the step {name!r}; install the"
1680 f" extension that declares it (`{prog()} sync` installs the ones the"
1681 " workspace lists), or name a step an installed extension declares"
1682 )
1683 return found
1687 """What *package* sets for the step *name*'s *option*, or the option's default.
1689 The deepest of the package's tables that sets it wins. A step the
1690 mount registered answers from its registration. A step it did not
1691 register, as in a process that mounted nothing, answers from the
1692 installed extension that declares it.
1694 Raises:
1695 Failed: when no installed extension declares the step, naming
1696 it; when the step declares no *option*, naming its options;
1697 or when the package sets a value of the wrong type, naming
1698 the table.
1699 """
1700 return option_value(declared_step(name), package, option)
1704 """The extensions the step *name* names in its ``extensions`` list.
1706 The step judges a package whose set holds one of them
1707 ([livery.workshop.extension_set][]), and every package when it
1708 names none. The step is read as [livery.workshop.step_option][]
1709 reads it: registered, else installed.
1711 Raises:
1712 Failed: when no installed extension declares the step, naming it.
1713 """
1714 return declared_step(name).extensions
1718 """Whether the step *name* is on in the workspace at *root*.
1720 A step its extension declares on is always on. A step it declares
1721 ``enabled = false`` is on while the root contract turns it on, with
1722 ``enabled = true`` in the step's ``[phases.<phase>.<step>]`` table,
1723 and registers only then. A package turns a step off for itself
1724 with the step's ``enabled`` option, which
1725 [livery.workshop.step_option][] reads.
1727 Raises:
1728 Failed: when no installed extension declares the step, naming it.
1729 """
1730 if declared_step(name).enabled:
1731 return True
1732 from livery.workshop._extensions import enabled_steps
1734 return name in enabled_steps(root)
1738 """Keep the *units* whose suites a registered step of the test phase runs.
1740 A step runs a unit's suite while the unit's set holds an extension
1741 the step names, and every unit's when it names none: a member holds
1742 its own set, and the workspace's own tests
1743 ([livery.workshop.WORKSPACE_TESTS][]) hold the root's set. A step
1744 of package scope runs over one member at a time and never over the
1745 workspace's own tests.
1747 Returns:
1748 The units of *units* some test step runs, in *units* order.
1749 """
1750 from livery.workshop._packages import WORKSPACE_TESTS
1752 tests = [record for record in _STEPS.values() if record.phase == "test"]
1753 return tuple(
1754 unit
1755 for unit in units
1756 if any(
1757 judges_unit(record, unit)
1758 and not (unit.path == WORKSPACE_TESTS and record.scope == PACKAGE)
1759 for record in tests
1760 )
1761 )
1765 """Register the workshop's own steps, in the order the rewriters run.
1767 Each judges the whole tree: the render's drift, the provenance of
1768 the shipped content, the graph and the base's rules over the
1769 sources. A step that judges one package at a time is an
1770 extension's, and registers at mount.
1771 """
1776 from livery.workshop import _quality
1778 _quality.drift_check(files=_selected_files(step_for("drift.check"), ctx))
1781 # A rewriter is not judged again under --fix, so the fix judges
1782 # what it leaves, as the provenance fix does.
1783 from livery.workshop import _quality
1784 from livery.workshop._shipped_files import deliver, relocate
1785 from livery.workshop._templates import apply_generated
1787 for line in relocate(ctx.root) + deliver(ctx.root):
1788 print(line)
1789 for path in apply_generated(ctx.root):
1790 print(f" generated: {path}")
1791 _quality.drift_check()
1794 from livery.workshop import _provenance
1796 _provenance.check_content(
1797 files=_selected_files(step_for("provenance.check"), ctx)
1798 )
1801 from livery.workshop import _provenance
1803 _provenance.check_content(
1804 fix=True, files=_selected_files(step_for("provenance.check"), ctx)
1805 )
1808 from livery.workshop._packages import verify_graph
1810 verify_graph(ctx.root)
1813 """Print each line a fix *written*, and lock again when one moved a manifest."""
1814 from livery.workshop._uv import run_uv
1816 for line in written:
1817 print(line)
1818 if any("pyproject.toml" in line for line in written):
1819 # Each line names the files written. A requirement written
1820 # into a uv member's manifest moves the uv workspace's lock,
1821 # so the fix locks again and leaves the tree consistent, as
1822 # a person would after editing by hand.
1823 run_uv("lock", root=root)
1826 from livery.workshop._extensions import write_extensions
1827 from livery.workshop._packages import verify_graph, write_test_requirements
1829 for line in write_extensions(ctx.root):1829 ↛ 1830line 1829 didn't jump to line 1830 because the loop on line 1829 never started
1830 print(line)
1831 relocked(ctx.root, write_test_requirements(ctx.root))
1832 verify_graph(ctx.root)
1835 """The judge of the base's layering step *step*, over the files it reaches."""
1837 def run(ctx: PhaseContext) -> None:
1838 from livery.workshop._packages import rule_problems
1840 files = _selected_files(step_for(f"layering.{step}"), ctx)
1841 problems = rule_problems(step, ctx.root, ctx.packages, files)
1842 if problems:
1843 fail("the workspace breaks its layering:\n " + "\n ".join(problems))
1845 return run
1848 from dataclasses import replace
1850 from livery.workshop._packages import discover_packages, write_edges
1852 written = write_edges(ctx.root)
1853 relocked(ctx.root, written)
1854 if written:
1855 # The run read the packages before the fix wrote their
1856 # contracts, and the judge reads them as the fix left them.
1857 ctx = replace(ctx, packages=discover_packages(ctx.root))
1858 # A rewriter is not judged again under --fix, so the fix judges
1859 # what it leaves.
1860 rule_run("sibling-references")(ctx)
1863 StepRecord(
1864 "check",
1865 "drift",
1866 render_run,
1867 fix=render_fix,
1868 inputs=Inputs(reads=_drift_reads, widens=_drift_widens, ignores=NOT_INPUTS),
1869 ),
1870 StepRecord(
1871 "check",
1872 "provenance",
1873 provenance_run,
1874 fix=provenance_fix,
1875 inputs=Inputs(reads=("packages/**/src/**/content/**",)),
1876 ),
1877 StepRecord(
1878 "graph",
1879 "layering",
1880 graph_run,
1881 fix=graph_fix,
1882 inputs=Inputs(reads=graph_files, per_file=False, ignores=NOT_INPUTS),
1883 ),
1884 # The base's rules over the sources, one step each; the sibling
1885 # step's fix is the one rewrite among them.
1886 *(
1887 StepRecord(
1888 step,
1889 "layering",
1890 rule_run(step),
1891 fix=sibling_fix if step == "sibling-references" else None,
1892 inputs=Inputs(
1893 reads=("tasks.py", "packages/**/*.py"),
1894 widens=graph_files,
1895 ignores=NOT_INPUTS,
1896 ),
1897 )
1898 for step in RULES
1899 ),
1900 ):
1906#: The registry as the import leaves it, for `snapshot(initial=True)`.