1"""The declared phases, the context a step receives, and the walk that runs steps.
3A phase is declared, by the base in the ``[phases]`` section of its
4``contract.toml`` and by an extension under ``[phases.<phase>]`` in its
5``extension.toml``, in one grammar ([livery.workshop._phases.PhaseSpec][]):
6the keys the engine running it gives its steps, typed; its level,
7``package`` for a phase whose steps run once per package and
8``workspace`` for one whose steps run once; whether ``fm check`` runs
9it; the phases it follows in the train's order; and whether a verb is
10generated for it. [livery.workshop._phases.declared_phases][] is every
11declared phase by name, the base's and then each installed
12extension's; a phase nobody declares refuses naming the declared ones,
13and two extensions declaring one phase refuse naming both.
15Every phase's steps run through the one walk,
16[livery.workshop._phases.walk_steps][]: each step a footman step inside
17one ``parallel()`` block, a workspace step once and a package step
18once per package in scope, independent steps concurrently, a step
19after the steps it names in ``after`` and after the providers of the
20keys it ``reads``. The gate runs its phases through it with its own
21engine around it ([livery.workshop._quality.walk][]); the release
22train and ``fm run`` run a package's lifecycle phases through it
23([livery.workshop._lifecycle][]), the steps those of the package's set
24([livery.workshop._phases.phase_steps][]); ``fm sync`` runs the sync
25phase's registered steps through it over the workspace, and
26[livery.workshop.run_phase][] those of any phase the gate never runs,
27first removing what a step that stopped running declared it writes.
28Each step receives the one context, [livery.workshop.PhaseContext][].
30A step table's ``deletes`` names steps of other extensions, each
31``<owner>:<phase>.<step>`` with its reason, that never run for a
32package whose set holds the deleting extension: the set's steps leave
33them out, and the walk skips such a package for a registered step it
34deletes, saying why. A package whose set does not hold the deleting
35extension runs them.
37The engine running a walk gives each phase its keys, by phase. The
38train builds on the isolated side in the ``release`` configuration and
39tests on the isolated side once per resolution it proves; the gate
40builds on the tree side in the ``instrumented`` configuration, and
41``fm run`` on the tree side in ``debug``. A phase whose declaration
42holds ``isolated`` has two sides: the tree, and the isolated one, from
43a copy of the sources in a fresh environment. A step that declares
44``isolated`` serves the side it names alone, and one that declares
45nothing serves both ([livery.workshop._phases.serves][]).
47Two steps providing one key, a key nobody provides, a step declaring a
48side of a phase that has one side, and an order with no start refuse
49before any step runs ([livery.workshop._phases.phase_problems][]);
50``layering.graph`` names them for each package.
51"""
66 from livery.workshop._composition import Lookup
67 from livery.workshop._influence import Changes
68 from livery.workshop._packages import Package
69 from livery.workshop._steps import StepRecord
71#: The levels a phase runs at: over one package at a time, or over the
72#: workspace once.
77#: The types an engine key or a context key may take.
80#: The extension that declares the base's phases.
86 """A phase as its declarer declares it: one ``[phases.<phase>]`` table.
88 Attributes:
89 name: The phase's name, ``build``.
90 keys: The keys the engine running the phase gives its steps,
91 each to its type; a step reads one like any key, and no
92 step provides one.
93 level: ``package`` for a phase whose steps run once per
94 package in scope; ``workspace`` for one whose steps run
95 once over the workspace.
96 gated: Whether ``fm check`` runs the phase.
97 after: The phases this one follows in the train's order.
98 verb: Whether a verb is generated for the phase, ``fm <phase>``
99 with a sub-task per step.
100 extension: The extension that declares the phase; the base for
101 its own.
102 """
114 """A phase whose steps cannot run as declared, or a step's misuse of the context.
116 Raised before any step runs for the phase's own problems, and from
117 [livery.workshop.PhaseContext.provide][] or
118 [livery.workshop.PhaseContext.read][] for a key the running step's
119 table does not declare, or a value of another type.
120 """
125 """The base's phases, read from the ``[phases]`` section of its contract file.
127 In the file's order, each spec naming the base as its declarer.
129 Raises:
130 DeclarationError: when a section is off the shape a phase's
131 declaration has.
132 """
141 """Every declared phase by name: the base's, then each installed extension's.
143 An installed extension's declaration is read whether the workspace
144 lists it or not, so a step into its phase reads the same anywhere;
145 the mount refuses the step while the declarer is not listed. Each
146 phase's ``after`` names declared phases, and the order they give
147 has a start.
149 Raises:
150 DeclarationError: when two extensions declare one phase, a
151 phase follows one nobody declares, or the order has no
152 start.
153 """
154 from livery.footman import installed_entry_points
155 from livery.workshop._declaration import DeclarationError, phase_declarations
157 found = dict(base_phases())
158 for entry in installed_entry_points("workshop.extensions"):
159 for spec in phase_declarations(entry.name, entry.value.partition(":")[0]):
160 held = found.get(spec.name)
161 if held is not None and held.extension != spec.extension:
162 both = " and ".join(sorted((held.extension, spec.extension)))
163 raise DeclarationError(
164 f"the {spec.name} phase is declared by both {both}; one extension"
165 " declares a phase"
166 )
167 found[spec.name] = spec
168 _judge_after(found)
169 return found
173 """Refuse an ``after`` naming no declared phase, and an order with no start."""
174 from livery.workshop._declaration import DeclarationError
176 for spec in specs.values():
177 for name in spec.after:
178 if name not in specs:
179 raise DeclarationError(
180 f"{spec.extension} declares the {spec.name} phase after {name},"
181 f" which no extension declares; the phases are"
182 f" {', '.join(specs)}"
183 )
184 done: set[str] = set()
185 path: list[str] = []
187 def visit(name: str) -> None:
188 if name in done:
189 return
190 if name in path:
191 cycle = path[path.index(name) :]
192 raise DeclarationError(
193 "the phases' order has no start: "
194 + " after ".join((name, *reversed(cycle)))
195 + "; remove an after among them"
196 )
197 path.append(name)
198 for earlier in specs[name].after:
199 visit(earlier)
200 path.pop()
201 done.add(name)
203 for name in specs:
204 visit(name)
208 """The declared phase *name*; refuses naming the declared ones.
210 Raises:
211 PhaseError: when no extension declares *name*.
212 """
213 specs = declared_phases()
214 found = specs.get(name)
215 if found is None:
216 raise PhaseError(
217 f"{name} is not a declared phase; the phases are {', '.join(specs)}"
218 )
219 return found
223 """The package-level phases, in declaration order: a package's lifecycle."""
224 return tuple(
225 name for name, spec in declared_phases().items() if spec.level == PACKAGE_LEVEL
226 )
230 """Whether *value* is of the context type *kind*."""
231 if kind == "str":
232 return isinstance(value, str)
233 if kind == "int":
234 return isinstance(value, int) and not isinstance(value, bool)
235 if kind == "bool":
236 return isinstance(value, bool)
237 if kind == "path":
238 return isinstance(value, Path)
239 if kind in ("strs", "paths"):
240 member = str if kind == "strs" else Path
241 return isinstance(value, (list, tuple)) and all(
242 isinstance(item, member) for item in value
243 )
244 return isinstance(value, dict)
249 """What every view of one walk's context writes and reads.
251 Attributes:
252 values: The keys the steps provided, by the package they ran
253 over (an empty path for a workspace step), each to its
254 value.
255 exit_code: The exit code the ``run`` phase's step recorded.
256 lock: Held around a write, since the steps run concurrently.
257 """
260 default_factory=dict[str, dict[str, object]]
261 )
268 """What a run hands a step: the workspace, the run's scope, and the phase's keys.
270 One context serves one walk, and every step receives its own view
271 of it, with the step named in ``step`` and, for a package step,
272 the package it runs over in ``package``. A step writes the keys its
273 table ``provides`` with [livery.workshop.PhaseContext.provide][]
274 and reads the ones it ``reads`` with
275 [livery.workshop.PhaseContext.read][]; the views share what is
276 written, so the engine reads a provided key back from the context
277 the walk returns ([livery.workshop.PhaseContext.result][]). The
278 ``run`` phase's step that starts the executable records its exit
279 code with [livery.workshop.PhaseContext.exit][]. A view is frozen:
280 a step that assigns an attribute changes a copy nobody reads, so
281 the keys and the exit code go through the methods.
283 Attributes:
284 root: The workspace root.
285 packages: Every package the workspace carries.
286 subset: The packages a scoped gate judges, or None for the
287 whole gate.
288 tests: Per package path, the test files that stand for the
289 package's suite in this run.
290 examples: The package paths whose changed files are examples
291 and nothing else: their examples run and no suite.
292 fix: Whether the rewriters run in their fix mode.
293 files: The files the run is limited to, root-relative, from
294 `fm check <paths>`; empty when the scope decides.
295 safe: Whether the fixers run in their in-flight mode, which
296 removes no code: ``--safe-fix``.
297 point: The CI point whose tests the test phase selects; empty
298 for the gate's own.
299 shard: The shard of a sharded check leg this run is, as
300 ``fm check --shard`` or the leg's label names it
301 ([livery.workshop.Shard][]); one shard of one for a run
302 that is not split. A step that declares ``shard`` among
303 its flags runs its share of it, and the walk runs every
304 other step on the first shard alone
305 ([livery.workshop._steps.on_this_shard][]).
306 catalogue: Every file the run may read, by unit, each with its
307 category, listed once per walk by
308 [livery.workshop._steps.catalogue][]; None when there is
309 no listing, and then every step that applies runs.
310 package: The package a package step runs over, else None.
311 selection: The test files selected for that package, relative
312 to it, else empty.
313 changes: What changed since the tree the run measures from,
314 which selects what each workspace step with declared
315 inputs judges ([livery.workshop._influence.select][]); None
316 when the run knows nothing of it, and then each judges
317 everything.
318 arguments: The words after ``--``, the same for every step of
319 the run: on a step's own verb, which a step that takes
320 them hands its tool (``StepRecord.arguments``, a
321 template's ``{arguments}``), and on ``fm run``, which the
322 step starting the executable passes it; empty in every
323 other run.
324 step: The running step's name, ``<phase>.<step>``, which the
325 walk sets as it hands the context to the step; empty
326 outside a step's run.
327 phase: The phase the engine runs, ``build``, whose given keys
328 [livery.workshop.PhaseContext.result][] reads back; on a
329 step's view, the step's own phase, whose given keys the
330 step reads. Empty for a walk over several phases, the
331 gate's.
332 """
338 default_factory=dict[str, tuple[str, ...]]
339 )
354 default_factory=dict[str, "StepRecord"], repr=False
355 )
357 default_factory=dict[str, Mapping[str, object]], repr=False
358 )
363 """Whether this is a package-scoped gate."""
364 return self.subset is not None
368 """The packages under judgment: the subset, or every package."""
369 return self.packages if self.subset is None else self.subset
373 """The package the step runs over, for a step that runs once per package.
375 Raises:
376 Failed: when the run names none: a workspace step, or a
377 context built outside a package step's run.
378 """
379 if self.package is None:
380 fail(
381 f"{self.step or 'this step'} runs over one package at a time, and"
382 " this run names none"
383 )
384 return self.package
388 """The exit code the ``run`` phase's step recorded; 0 until one does."""
389 return self._shared.exit_code
392 """Record *code* as the exit code ``fm run`` exits with.
394 The ``run`` phase's step that starts the executable calls it
395 with the executable's; the walk hands the code to the verb
396 through the context it returns.
397 """
398 self._shared.exit_code = code
401 """This context narrowed to one package and its selected tests."""
402 files = self.tests.get(package.path, ())
403 relative = tuple(path[len(package.path) + 1 :] for path in files)
404 return replace(self, package=package, selection=relative)
407 """Write *value* under *key*, a key the running step's table provides.
409 Raises:
410 PhaseError: when the step's table does not provide *key*,
411 or *value* is not of the type it declares.
412 """
413 record = self._running()
414 provided = dict(record.provides)
415 if key not in provided:
416 raise PhaseError(
417 f"{self.step} provides {key}, and its {self._table()} table does"
418 " not name it in provides"
419 )
420 if not _holds(provided[key], value):
421 raise PhaseError(
422 f"{self.step} provides {key} as {type(value).__name__}, and its"
423 f" {self._table()} table declares it {provided[key]}"
424 )
425 with self._shared.lock:
426 self._shared.values.setdefault(self._unit(), {})[key] = value
429 """The value under *key*, a key the running step's table reads.
431 Its type is the one its provider declares, or the one the
432 phase's declaration gives an engine key.
434 Raises:
435 PhaseError: when the step's table does not read *key*, the
436 engine did not give it, or its provider wrote nothing
437 under it.
438 """
439 record = self._running()
440 if key not in record.reads:
441 raise PhaseError(
442 f"{self.step} reads {key}, and its {self._table()} table does not"
443 " name it in reads"
444 )
445 if key in self._engine():
446 given = self._given.get(self.phase, {})
447 if key not in given:
448 raise PhaseError(
449 f"{self.step} reads {key}, which the engine running the"
450 f" {self.phase} phase did not give it"
451 )
452 return given[key]
453 values = self._shared.values.get(self._unit(), {})
454 if key not in values:
455 provider = next(
456 (
457 name
458 for name, held in self._steps.items()
459 if key in dict(held.provides)
460 ),
461 "its provider",
462 )
463 raise PhaseError(
464 f"{self.step} reads {key}, which {provider} has not provided in"
465 f" the {self.phase} phase"
466 )
467 return values[key]
470 """The value under *key* once the phase ran, for the engine that ran it.
472 A key a step provided for the context's package, else one the
473 engine gave the context's phase; None when neither wrote one.
474 """
475 values = self._shared.values.get(self._unit(), {})
476 return values.get(key, self._given.get(self.phase, {}).get(key))
479 """The running step's record; refuses outside a walk's step."""
480 record = self._steps.get(self.step)
481 if record is None:
482 raise PhaseError(
483 f"{self.step or 'this step'} is not a step of the walk that built"
484 " this context, so the context has no table to judge its keys by"
485 )
486 return record
489 """The running step's table, as the declaration spells it."""
490 phase, _dot, name = self.step.partition(".")
491 return f"[phases.{phase}.{name}]"
494 """The keys the engine gives the phase, when the context names one."""
495 return phase_spec(self.phase).keys if self.phase else {}
498 """Where the keys this view writes and reads live: its package, or the root."""
499 return self.package.path if self.package is not None else ""
503 phase: str, members: tuple[str, ...], lookup: Lookup
504) -> tuple[StepRecord, ...]:
505 """The steps the extensions *members* add to *phase*, in their order.
507 Each member's steps in declaration order, the members in the order
508 given, which for a package's set is its composition order. A step
509 one of *members* deletes is left out, so it never runs for the set
510 (`deletions`).
511 """
512 deleted = deletions(members, lookup)
513 found: list[StepRecord] = []
514 for name in members:
515 declared = lookup(name)
516 if declared is not None:
517 found += [
518 record
519 for record in declared.phases.get(phase, ())
520 if step_address(record) not in deleted
521 ]
522 return tuple(found)
526 """*record*'s address among every extension's steps: ``<owner>:<phase>.<step>``."""
527 return f"{record.extension}:{record.name}"
531 """The steps the extensions *members* delete, by address, to the deleter and why.
533 An address is ``<owner>:<phase>.<step>``, as a step table's
534 ``deletes`` names it; the first of *members* to delete a step is
535 its deleter.
536 """
537 found: dict[str, tuple[str, str]] = {}
538 for name in members:
539 declared = lookup(name)
540 if declared is None:
541 continue
542 for records in declared.phases.values():
543 for record in records:
544 for address, reason in record.deletes:
545 found.setdefault(address, (name, reason))
546 return found
550 """The steps *package*'s set deletes, by address, to the deleter and why."""
551 from livery.workshop._composition import extension_set
552 from livery.workshop._extensions import installed_declaration
554 return deletions(extension_set(package), installed_declaration)
557#: The engine key that names a phase's side: true for the isolated one.
562 """Whether *record* runs on the side of its phase the engine's *given* keys name.
564 *given* are the keys the engine gives the step's phase. A step
565 declaring ``isolated = true`` runs when the engine gives
566 ``isolated`` true; one declaring false runs when the engine gives
567 false or nothing; one declaring neither runs on both sides.
568 """
569 if record.isolated is None:
570 return True
571 return record.isolated == (given.get(ISOLATED) is True)
575 """Each key the steps provide, to the steps that provide it."""
576 found: dict[str, list[str]] = {}
577 for record in steps:
578 for key, _kind in record.provides:
579 found.setdefault(key, []).append(record.name)
580 return found
584 record: StepRecord,
585 by_name: Mapping[str, StepRecord],
586 providers: Mapping[str, list[str]],
587 *,
588 after: bool,
589) -> set[str]:
590 """The steps among *by_name* that *record* runs after.
592 The providers of the keys it reads, in its own phase, and, with
593 *after*, the steps it names in ``after``; a name outside
594 *by_name* orders nothing.
595 """
596 found = {
597 provider
598 for key in record.reads
599 for provider in providers.get(key, ())
600 if by_name[provider].phase == record.phase
601 }
602 if after:
603 found.update(other for other in record.after if other in by_name)
604 found.discard(record.name)
605 return found
609 steps: Iterable[StepRecord], *, after: bool = True
610) -> tuple[StepRecord, ...]:
611 """*steps* in the order the walk starts them.
613 Each step after the providers of the keys it reads, in its own
614 phase, and, with *after*, after the steps it names in ``after``;
615 otherwise in the given order. An ``after`` naming a step outside
616 *steps* orders nothing.
618 Raises:
619 OrderCycle: when the order has no start, naming the members
620 of one cycle.
621 """
622 from livery.workshop._composition import OrderCycle, cycle_of
624 by_name = {record.name: record for record in steps}
625 providers = _providers(by_name.values())
626 waiting = {
627 name: _earlier(record, by_name, providers, after=after)
628 for name, record in by_name.items()
629 }
630 done: list[str] = []
631 while len(done) < len(by_name):
632 ready = next(
633 (
634 name
635 for name in by_name
636 if name not in done and not waiting[name].difference(done)
637 ),
638 None,
639 )
640 if ready is None:
641 left = {name: waiting[name] for name in by_name if name not in done}
642 raise OrderCycle(cycle_of(left))
643 done.append(ready)
644 return tuple(by_name[name] for name in done)
648 """Why *steps* cannot run as *phase*; empty when they can.
650 Two extensions declaring one step, two steps providing one key, a
651 step providing a key the phase's engine gives, a step declaring a
652 side of a phase whose declaration gives no ``isolated``, a key a
653 step reads and no step provides, and an order with no start.
655 Raises:
656 PhaseError: when no extension declares *phase*.
657 """
658 from livery.workshop._composition import OrderCycle
660 records = tuple(steps)
661 problems: list[str] = []
662 engine = phase_spec(phase).keys
663 owners: dict[str, list[str]] = {}
664 for record in records:
665 owners.setdefault(record.name, []).append(record.extension)
666 for name, extensions in owners.items():
667 if len(extensions) > 1:
668 problems.append(
669 f"the {phase} phase: {' and '.join(sorted(extensions))} both declare"
670 f" {name}; one extension of a package declares a step"
671 )
672 providers = _providers(records)
673 for key, names in sorted(providers.items()):
674 if key in engine:
675 verb = "provides" if len(names) == 1 else "provide"
676 problems.append(
677 f"the {phase} phase: {' and '.join(sorted(names))} {verb} {key},"
678 " which the engine running the phase provides; a step reads it"
679 )
680 elif len(names) > 1:
681 problems.append(
682 f"the {phase} phase: {' and '.join(sorted(names))} both provide"
683 f" {key}; one step of a package provides a key"
684 )
685 if ISOLATED not in engine:
686 problems += [
687 f"the {phase} phase: {record.name} declares isolated, and the phase"
688 " has one side: its declaration gives no isolated key"
689 for record in records
690 if record.isolated is not None
691 ]
692 for record in records:
693 for key in record.reads:
694 if key not in providers and key not in engine:
695 problems.append(
696 f"the {phase} phase: {record.name} reads {key}, which no step of"
697 " the package provides"
698 )
699 try:
700 phase_order(records)
701 except OrderCycle as error:
702 cycle = " before ".join((*error.members, error.members[0]))
703 problems.append(
704 f"the {phase} phase: its steps' order has no start: {cycle}; remove a"
705 " key read or an after among them"
706 )
707 return problems
711 """One footman step of a walk: a step record over one package, or over the whole.
713 Attributes:
714 record: The step.
715 package: The package it runs over; None for a workspace step.
716 title: What footman shows for it: the step's name, then the
717 package's path.
718 done: Set once the unit ran, whatever happened.
719 failed: Whether it failed; a unit that waits on it refuses.
720 needs: The units that run before it.
721 """
726 self.record = record
727 self.package = package
728 self.title = record.name if package is None else f"{record.name} {package.path}"
729 self.done = threading.Event()
730 self.failed = False
731 self.needs: list[_Unit] = []
734#: How long a waiting step looks at the unit it waits on before it
735#: yields to footman again.
740 ctx: PhaseContext,
741 steps: Iterable[StepRecord],
742 *,
743 inputs: Mapping[str, Mapping[str, object]] | None = None,
744 fixing: bool = False,
745) -> PhaseContext:
746 """Run *steps* as footman steps inside one ``parallel()`` block.
748 A step runs on the side of its phase the engine gives
749 ([livery.workshop._phases.serves][]), and a step serving the other
750 side is not run. A workspace step runs once. A package step runs
751 once per package in scope (the context's ``judged``) it judges:
752 one whose set holds an extension the step names and none that
753 deletes the step, that has not turned the step off, that has a
754 file the step's claims reach, and whose change is not confined to
755 its tests unless the step is tests-only; each other package is
756 said and not run. Independent
757 steps run concurrently, and a step starts once the steps it names
758 in ``after`` and the providers of the keys it reads have run, for
759 the same package. A step whose earlier step failed is not run and
760 says so; the earlier failure is the verdict, and the steps that
761 depend on nothing failed still run, as footman runs a block.
762 *inputs* are the keys the engine gives each phase, by phase, each
763 phase's judged against its declaration before any step runs.
764 *fixing* runs each step's fix body instead of its main, a step
765 without one running nothing, in the given order: a fix rewrites
766 what its own tool reads, and the gate runs its fixers one at a
767 time, so ``after`` orders nothing there.
769 Returns:
770 The context the steps shared, which the engine reads a provided
771 key or the exit code back from.
773 Raises:
774 PhaseError: before any step runs, when the context or *inputs*
775 names a phase nobody declares, when *inputs* gives a phase
776 a key it does not take or a value of another type, or when
777 the steps cannot run as declared
778 ([livery.workshop._phases.phase_problems][]).
779 Exception: the first failing step's, raised again once every
780 step ran.
781 """
782 from livery.workshop._steps import (
783 PACKAGE,
784 judges_package,
785 members,
786 option_value,
787 reads_files,
788 )
790 records = tuple(steps)
791 where = f"{ctx.package.path}: " if ctx.package is not None else ""
792 given = {phase: dict(keys) for phase, keys in (inputs or {}).items()}
793 if ctx.phase:
794 # A context naming a phase nobody declares refuses here, before
795 # any step runs, as an input to such a phase does.
796 phase_spec(ctx.phase)
797 _judge_inputs(given, where)
798 problems: list[str] = []
799 for phase in dict.fromkeys(record.phase for record in records):
800 problems += phase_problems(
801 phase, (record for record in records if record.phase == phase)
802 )
803 if problems:
804 raise PhaseError(where + "; ".join(problems))
805 ordered = phase_order(records, after=not fixing)
806 held = replace(
807 ctx, _steps={record.name: record for record in records}, _given=given
808 )
809 units: list[_Unit] = []
810 removed: dict[str, dict[str, tuple[str, str]]] = {}
811 for record in ordered:
812 if (record.fix if fixing else record.run) is None:812 ↛ 813line 812 didn't jump to line 813 because the condition on line 812 was never true
813 continue
814 if not serves(record, given.get(record.phase, {})):
815 continue
816 if record.scope != PACKAGE:
817 units.append(_Unit(record, None))
818 continue
819 for package in members(held):
820 if not judges_package(record, held, package):
821 continue
822 if package.path not in removed:
823 removed[package.path] = set_deletions(package)
824 deleted = removed[package.path].get(step_address(record))
825 if deleted is not None:
826 deleter, reason = deleted
827 print(
828 f" {record.name}: {package.path} skips ({deleter} deletes it:"
829 f" {reason})"
830 )
831 continue
832 if not reads_files(record, held, (package.path,)):
833 print(f" {record.name}: {package.path} has no file it reads; not run")
834 continue
835 if not option_value(record, package, "enabled"):
836 print(
837 f" {record.name}: {package.path} skips (turned off in"
838 f" {package.path}/workshop.toml)"
839 )
840 continue
841 units.append(_Unit(record, package))
842 _link(units, ordered, after=not fixing)
843 bodies = [_resolved(unit, fixing) for unit in units]
844 with parallel() as block:
845 for unit, body in zip(units, bodies, strict=True):
846 built = step(_body(held, unit, body), title=unit.title)
847 block(built.opts(capture=False)())
848 return held
852 """Run the steps registered for the phase *ctx* names, through the walk.
854 The steps the mount registered for ``ctx.phase``, in registration
855 order: each listed extension's own, and those of the
856 ``[for.<target>]`` tables that apply. They run as `walk_steps` runs
857 any steps, so a phase the gate never runs, one declared with
858 ``gated = false``, runs this way over the workspace its context
859 names. A workspace step that names extensions runs once while a
860 package the context judges holds one of them; while none does, a
861 line says so and the step does not run. A phase with no registered
862 step runs nothing.
864 Returns:
865 The context the steps shared.
867 Raises:
868 PhaseError: when *ctx* names no phase, or one nobody declares.
869 Exception: the first failing step's, raised again once every
870 step ran.
871 """
872 running, _idle = _registered(ctx)
873 return walk_steps(ctx, running)
877 ctx: PhaseContext,
878) -> tuple[tuple[StepRecord, ...], tuple[StepRecord, ...]]:
879 """The steps registered for ``ctx.phase``: those that run, and those that do not.
881 A workspace step that names extensions runs while a package the
882 context judges holds one of them, and a line says so for each one
883 that does not.
885 Raises:
886 PhaseError: when *ctx* names no phase.
887 """
888 from livery.workshop._steps import PACKAGE, judges_unit, steps_by_name
890 if not ctx.phase:
891 raise PhaseError(
892 "a run of the registered steps names its phase, and this context names none"
893 )
894 running: list[StepRecord] = []
895 idle: list[StepRecord] = []
896 for record in steps_by_name().values():
897 if record.phase != ctx.phase:
898 continue
899 named = record.scope != PACKAGE and bool(record.extensions)
900 if named and not any(judges_unit(record, unit) for unit in ctx.judged):
901 wanted = " or ".join(record.extensions)
902 print(f" {record.name}: no package holds {wanted}; not run")
903 idle.append(record)
904 continue
905 running.append(record)
906 return tuple(running), tuple(idle)
909#: The record of what each phase's last run through `run_phase` declared
910#: it writes: the checkout's own state, never committed.
913#: The prefix of an output that names a path in each package.
918 """Why *output* cannot name what a step writes, or ``""`` when it can.
920 An output is a path relative to the workspace root, or
921 ``package/<path>`` for that path in each package; a directory ends
922 in ``/``. It lies below the root, or below the package, and it is
923 never ``docs/`` itself, the root's or a package's, which holds the
924 authored pages. The text completes a sentence about the output.
925 """
926 import re
928 inner = output.removeprefix(IN_EACH_PACKAGE)
929 where = "each package" if inner != output else "the root"
930 parts = tuple(part for part in inner.split("/") if part not in ("", "."))
931 if (
932 "\\" in output
933 or inner.startswith("/")
934 or re.match(r"[A-Za-z]:", inner)
935 or ".." in parts
936 ):
937 return (
938 f"names {output!r}, which leaves the root; an output is a path below"
939 " the root, or package/<path> for the path in each package"
940 )
941 if not parts:
942 return f"names {output!r}, {where} itself; an output is a path below it"
943 if parts == ("docs",):
944 tree = "each package's" if inner != output else "the root's"
945 return (
946 f"names {output!r}, {tree} docs/ itself, which holds the authored"
947 " pages; an output is a path below it"
948 )
949 return ""
953 """Each phase's outputs, as its last run through `run_phase` recorded them.
955 A record that cannot be read, or holds another shape, records
956 nothing: the next run writes it again.
957 """
958 import json
960 try:
961 held = json.loads((root / OUTPUTS_RECORD).read_text(encoding="utf-8"))
962 except (OSError, ValueError):
963 return {}
964 if not isinstance(held, dict) or held.get("schema") != 1:
965 return {}
966 phases = held.get("phases")
967 if not isinstance(phases, dict):967 ↛ 968line 967 didn't jump to line 968 because the condition on line 967 was never true
968 return {}
969 return {
970 phase: tuple(output for output in listed if isinstance(output, str))
971 for phase, listed in cast("dict[str, object]", phases).items()
972 if isinstance(listed, list)
973 }
977 root: Path,
978 phase: str,
979 running: tuple[str, ...],
980 idle: tuple[str, ...],
981 packages: tuple[Package, ...],
982) -> None:
983 """Remove what no running step of *phase* writes, and record what they do.
985 Each output the phase's last run recorded, or a registered step
986 that does not run declares, goes unless a running step declares
987 it; one line names each path removed. The record then holds the
988 running steps' outputs.
989 """
990 import json
991 import shutil
993 held = _outputs_held(root)
994 kept = set(running)
995 stale = [
996 output
997 for output in dict.fromkeys((*held.get(phase, ()), *idle))
998 if output not in kept and not output_refusal(output)
999 ]
1000 for output in stale:
1001 inner = output.removeprefix(IN_EACH_PACKAGE)
1002 homes = [p.directory for p in packages] if inner != output else [root]
1003 for home in homes:
1004 path = home / inner
1005 if path.is_symlink() or path.is_file():
1006 path.unlink()
1007 elif path.is_dir():1007 ↛ 1010line 1007 didn't jump to line 1010 because the condition on line 1007 was always true
1008 shutil.rmtree(path)
1009 else:
1010 continue
1011 shown = path.relative_to(root).as_posix() + (
1012 "/" if output.endswith("/") else ""
1013 )
1014 print(f" removed {shown}: no running step of the {phase} phase writes it")
1015 recorded = tuple(sorted(kept))
1016 if held.get(phase, ()) == recorded:
1017 return
1018 held[phase] = recorded
1019 path = root / OUTPUTS_RECORD
1020 path.parent.mkdir(parents=True, exist_ok=True)
1021 document = {
1022 "schema": 1,
1023 "phases": {name: list(held[name]) for name in sorted(held) if held[name]},
1024 }
1025 path.write_text(json.dumps(document, indent=2) + "\n", encoding="utf-8")
1029 """Run a phase the gate never runs over the workspace at *root*; the shared context.
1031 The steps the listed extensions register for *phase*, their own and
1032 those of the ``[for.<target>]`` tables that apply, run side by side
1033 through the walk every phase runs through, each workspace step once
1034 and each package step once per package whose set holds an
1035 extension it names. A workspace step that names extensions runs
1036 while a package holds one of them; while none does, a line says so
1037 and the step does not run. A phase with no registered step runs
1038 nothing.
1040 Before the steps run, the paths a step declares under ``outputs``
1041 are settled: each one the phase's last run recorded, or a step that
1042 does not run now declares, is removed unless a step that runs now
1043 declares it, and a line names each path removed. So a step that
1044 stops running, its extension unlisted or no longer installed or the
1045 step turned off, leaves nothing behind wherever it wrote, and a
1046 running step finds its own output as it left it. The record lives
1047 in the checkout's ``.workshop/state/``.
1049 Args:
1050 root: The workspace root.
1051 phase: The phase, one declared with ``gated = false``.
1052 outputs: Paths this run writes that no step declares, such as
1053 those of the verbs a step runs: each in the shape a step
1054 declares one, and removed and recorded as a running step's
1055 are.
1057 Returns:
1058 The context the steps shared.
1060 Raises:
1061 PhaseError: when nobody declares *phase*, naming the declared
1062 phases; when the gate runs it; or when an output in
1063 *outputs* is not a path below the root, or names ``docs/``
1064 itself.
1065 Exception: the first failing step's, raised again once every
1066 step ran.
1067 """
1068 from livery.workshop._packages import discover_packages
1070 spec = phase_spec(phase)
1071 if spec.gated:
1072 raise PhaseError(
1073 f"the gate runs the {phase} phase; run_phase runs a phase declared"
1074 " with gated = false, and run_phase_over runs a gated phase's steps"
1075 " over named packages"
1076 )
1077 given = tuple(outputs)
1078 for output in given:
1079 refusal = output_refusal(output)
1080 if refusal:
1081 raise PhaseError(f"an output of the {phase} phase {refusal}")
1082 ctx = PhaseContext(root=root, packages=discover_packages(root), phase=phase)
1083 running, idle = _registered(ctx)
1084 _remove_outputs(
1085 root,
1086 phase,
1087 (*(o for record in running for o in record.outputs), *given),
1088 tuple(o for record in idle for o in record.outputs),
1089 ctx.packages,
1090 )
1091 return walk_steps(ctx, running)
1095 """Refuse an undeclared phase, a key it does not take, or a mistyped value."""
1096 for phase, keys in given.items():
1097 engine = phase_spec(phase).keys
1098 for key, value in keys.items():
1099 if key not in engine:
1100 known = ", ".join(sorted(engine)) or "nothing"
1101 raise PhaseError(
1102 f"{where}the engine gives {key} to the {phase} phase, which takes"
1103 f" {known} from it"
1104 )
1105 if not _holds(engine[key], value):
1106 raise PhaseError(
1107 f"{where}the engine gives {key} to the {phase} phase as"
1108 f" {type(value).__name__}, which the phase takes as {engine[key]}"
1109 )
1113 """Give each of *units* the units it waits on: its earlier steps, same package.
1115 A package unit waits on an earlier step's unit of its own package,
1116 and on a workspace unit; a workspace unit waits on every unit of
1117 an earlier step.
1118 """
1119 by_name = {record.name: record for record in ordered}
1120 providers = _providers(ordered)
1121 by_record: dict[str, list[_Unit]] = {}
1122 for unit in units:
1123 by_record.setdefault(unit.record.name, []).append(unit)
1124 for unit in units:
1125 for name in sorted(_earlier(unit.record, by_name, providers, after=after)):
1126 for other in by_record.get(name, ()):
1127 same = (
1128 unit.package is None
1129 or other.package is None
1130 or unit.package.path == other.package.path
1131 )
1132 if same:1132 ↛ 1126line 1132 didn't jump to line 1126 because the condition on line 1132 was always true
1133 unit.needs.append(other)
1137 """The body *unit* runs, its module imported now, on the calling thread.
1139 A declared step names its function, imported when it is first
1140 called; the walk imports every step's module before the block
1141 starts, so no two steps import at once, and a module that fails to
1142 import names its step before any step runs.
1143 """
1144 from livery.workshop._declaration import Reference
1146 body = unit.record.fix if fixing else unit.record.run
1147 assert body is not None # a unit is built for a step with a body alone
1148 if isinstance(body, Reference):
1149 return cast("Callable[[PhaseContext], None]", body.resolve())
1150 return body
1154 ctx: PhaseContext, unit: _Unit, body: Callable[[PhaseContext], None]
1155) -> Callable[[], Generator[None, Any, None]]:
1156 """The footman step's body for *unit*: wait for what it needs, then run *body*.
1158 A generator, so each wait is a checkpoint where footman cancels
1159 the step; the unit is marked done however the body ends, so a unit
1160 waiting on it never waits past its end.
1161 """
1162 from livery.workshop._composition import extension_set
1164 record = unit.record
1166 def run() -> Generator[None, Any, None]:
1167 try:
1168 for need in unit.needs:
1169 while not need.done.wait(_POLL_SECONDS):1169 ↛ 1170line 1169 didn't jump to line 1170 because the condition on line 1169 was never true
1170 yield
1171 failed = [need for need in unit.needs if need.failed]
1172 if failed:
1173 # Said here as well as refused: the block raises the
1174 # earlier failure, and this reason rides the record.
1175 reason = f"{unit.title}: not run, since {failed[0].title} failed"
1176 print(f" {reason}")
1177 fail(reason)
1178 view = ctx if unit.package is None else ctx.for_package(unit.package)
1179 view = replace(view, step=record.name, phase=record.phase)
1180 if unit.package is not None:
1181 held = extension_set(unit.package)
1182 matched = " and ".join(n for n in record.extensions if n in held)
1183 print(
1184 f" {record.name}: {unit.package.path} runs (a {matched} package)"
1185 )
1186 body(view)
1187 except BaseException:
1188 unit.failed = True
1189 raise
1190 finally:
1191 unit.done.set()
1193 return run