Skip to content

livery.workshop._fragment_engine

livery.workshop._fragment_engine

Compose, render and deliver the files the extensions keep current.

Every piece of content an extension delivers into a workspace is a livery.workshop._fragment_engine.Fragment: an owner, a name, a target path and the template its content renders from. plan turns the listed extensions' fragments into one livery.workshop._fragment_engine.Output per file, and apply writes them and records what it wrote; drift compares them with the committed files instead.

How the fragments of one target combine depends on the target's type:

  • a TOML file joins its fragments' tables in extension order;
  • an ignore or attribute file joins their lines, each line once, and puts the repository's regions after every extension's lines;
  • a JSON file merges their objects key by key;
  • any other file has one owner.

The fragments apply in the order the caller gives: for a sync, the base first, then the extensions in composition order, each after the ones it requires, and the repository's own fragments last. An extension that takes over or removes a fragment that applies before its own declares it, with a reason, through replaces or deletes; a second owner of a one-owner file without that declaration refuses, and so does a declaration naming a fragment no listed extension ships.

A target is a path at the root, or package/<path> for that path in every package whose extensions include the owner.

The receipts in .workshop-rendered, at the root and in each package, record the digest of each file as the engine last wrote it, its regions left out, so an edit inside a region does not count as an edit of the file. A file whose fragments are gone is removed only while it still has those bytes; a file edited since is kept and named as the repository's own, and so is an edited file the engine would otherwise rewrite. Regions, the repository's own lines between marker comments, are read from the committed file and written back in place on every render.

Classes:

Name Description
Fragment

One piece of content an extension delivers.

Output

One file as the engine renders it.

Functions:

Name Description
composition

How the fragments of target combine, by the file's type.

plan

Every file the listed fragments render, composed and with its regions kept.

read_rendered

The receipts in directory, name to digest; empty without any.

receipts_from

The receipts text spells, name to digest; empty for anything else.

receipt_of

The receipt path earns as it stands, its regions left out; None if absent.

merge_receipts

Two sides' receipts merged key by key, over the base they share.

write_rendered

Write the receipts, or remove the file when there are none.

write_root_file

Write one file at the root as apply writes it, and its receipt; the lines.

apply

Write outputs, withdraw what is gone, and say what changed.

local_receipts

What the engine linked or copied for this checkout alone, path to receipt.

apply_untracked

Write the local outputs git does not track, and nothing else.

tracked_local

The local outputs among outputs that git tracks, in path order.

drift

One line per file whose committed bytes are not what it renders.

Attributes:

Name Type Description
PACKAGE_PREFIX

The target prefix that names a path inside each package of the owner.

RENDERED_MANIFEST

The receipt file, name to digest, at the root and in each package.

Composition

How the fragments of one target combine.

LOCAL_RECEIPT

The checkout's own receipts: what the engine linked or copied for it alone.

PACKAGE_PREFIX module-attribute

PACKAGE_PREFIX = 'package/'

The target prefix that names a path inside each package of the owner.

RENDERED_MANIFEST module-attribute

RENDERED_MANIFEST = '.workshop-rendered'

The receipt file, name to digest, at the root and in each package.

Composition module-attribute

Composition = Literal['tables', 'lines', 'json', 'whole']

How the fragments of one target combine.

LOCAL_RECEIPT module-attribute

LOCAL_RECEIPT = '.workshop/rendered/receipts.json'

The checkout's own receipts: what the engine linked or copied for it alone.

Written by every delivery, empty when there is nothing of its own, so its presence says the checkout's own files were delivered at all; a checkout without it never had them, which the per-command repair reads (livery.workshop._reconcile.current).

Fragment dataclass

Fragment(owner: str, name: str, target: str, content: str = '', source: Path | None = None, dynamic: Callable[[Mapping[str, Any]], str] | None = None, replaces: str = '', deletes: str = '', reason: str = '', contributes: bool = False)

One piece of content an extension delivers.

Exactly one of content, source and dynamic carries the template, except on a fragment that only deletes.

Attributes:

Name Type Description
owner str

The extension that ships it, by its listed name.

name str

The fragment's name, unique within its owner.

target str

The path it renders into, relative to the root, or package/<path> for every package of the owner.

content str

The template source as a string.

source Path | None

A file in the extension's wheel holding the template.

dynamic Callable[[Mapping[str, Any]], str] | None

A function of the render data returning the template source, for content no static template can express.

replaces str

<owner>:<name> of a lower fragment this one takes the place of.

deletes str

<owner>:<name> of a lower fragment this one removes.

reason str

Why the replacement or deletion is wanted; required with either.

contributes bool

Whether the fragment is data for its target's one template rather than text: its rendered content is a JSON object, merged by key with every other contribution to the target, and the template reads the result.

Methods:

Name Description
template

The template source this fragment renders from.

owner instance-attribute

owner: str

name instance-attribute

name: str

target instance-attribute

target: str

content class-attribute instance-attribute

content: str = ''

source class-attribute instance-attribute

source: Path | None = None

dynamic class-attribute instance-attribute

dynamic: Callable[[Mapping[str, Any]], str] | None = field(default=None, compare=False)

replaces class-attribute instance-attribute

replaces: str = ''

deletes class-attribute instance-attribute

deletes: str = ''

reason class-attribute instance-attribute

reason: str = ''

contributes class-attribute instance-attribute

contributes: bool = False

ref property

ref: str

The fragment as replaces and deletes name it: <owner>:<name>.

template

template(data: Mapping[str, Any]) -> str

The template source this fragment renders from.

Source code in packages/workshop/src/livery/workshop/_fragment_engine.py
def template(self, data: Mapping[str, Any]) -> str:
    """The template source this fragment renders from."""
    if self.dynamic is not None:
        return self.dynamic(data)
    if self.source is not None:
        return self.source.read_text(encoding="utf-8")
    return self.content

Output dataclass

Output(path: str, body: bytes, owners: tuple[str, ...], link: Path | None = None, local: bool = False)

One file as the engine renders it.

Attributes:

Name Type Description
path str

The file, relative to the workspace root, with /.

body bytes

The bytes the file should hold.

owners tuple[str, ...]

The fragments it was composed from, as <owner>:<name>, in the order they applied.

link Path | None

The shipped file or directory the output links to instead of holding bytes; None for a written file.

local bool

Whether the output belongs to this checkout alone and is never committed: a link into an installed wheel, a copy the agent reads. Its receipt is the checkout's own.

path instance-attribute

path: str

body instance-attribute

body: bytes

owners instance-attribute

owners: tuple[str, ...]
link: Path | None = None

local class-attribute instance-attribute

local: bool = False

composition

composition(target: str) -> Composition

How the fragments of target combine, by the file's type.

Source code in packages/workshop/src/livery/workshop/_fragment_engine.py
def composition(target: str) -> Composition:
    """How the fragments of *target* combine, by the file's type."""
    name = PurePosixPath(target).name
    if name.endswith(".toml"):
        return "tables"
    if name in (".gitignore", ".gitattributes") or name.endswith("ignore"):
        return "lines"
    if name.endswith(".json"):
        return "json"
    return "whole"

plan

plan(root: Path, fragments: Sequence[Fragment], order: Sequence[str], data: Mapping[str, Any], *, packages: Mapping[str, Sequence[str]] | None = None, package_data: Mapping[str, Mapping[str, Any]] | None = None) -> tuple[Output, ...]

Every file the listed fragments render, composed and with its regions kept.

Parameters:

Name Type Description Default
root Path

The workspace root, where the committed files are read for their regions.

required
fragments Sequence[Fragment]

What the listed extensions ship.

required
order Sequence[str]

The owners in the order they apply, the base first and the repository last.

required
data Mapping[str, Any]

The render data every template sees. A package target also sees package, the package's path.

required
packages Mapping[str, Sequence[str]] | None

Each package's path, relative to the root, to the extensions it lists.

None
package_data Mapping[str, Mapping[str, Any]] | None

Render data for one package's files, by its path: what differs per package, such as its name and its release baseline.

None

Returns:

Type Description
tuple[Output, ...]

One output per file, sorted by path.

Raises:

Type Description
Failed

for a second owner of a one-owner file, a replaces or deletes without a reason or naming a fragment no listed extension ships, a template that does not render, and parts that do not compose.

Source code in packages/workshop/src/livery/workshop/_fragment_engine.py
def plan(
    root: Path,
    fragments: Sequence[Fragment],
    order: Sequence[str],
    data: Mapping[str, Any],
    *,
    packages: Mapping[str, Sequence[str]] | None = None,
    package_data: Mapping[str, Mapping[str, Any]] | None = None,
) -> tuple[Output, ...]:
    """Every file the listed fragments render, composed and with its regions kept.

    Args:
        root: The workspace root, where the committed files are read
            for their regions.
        fragments: What the listed extensions ship.
        order: The owners in the order they apply, the base first and
            the repository last.
        data: The render data every template sees. A package target
            also sees `package`, the package's path.
        packages: Each package's path, relative to the root, to the
            extensions it lists.
        package_data: Render data for one package's files, by its
            path: what differs per package, such as its name and its
            release baseline.

    Returns:
        One output per file, sorted by path.

    Raises:
        Failed: for a second owner of a one-owner file, a `replaces`
            or `deletes` without a reason or naming a fragment no
            listed extension ships, a template that does not render,
            and parts that do not compose.
    """
    packaged = dict(packages or {})
    rank = {owner: index for index, owner in enumerate(order)}
    grouped: dict[str, list[Fragment]] = {}
    for fragment in _resolve(fragments, order):
        for path in _targets(fragment, packaged):
            grouped.setdefault(path, []).append(fragment)
    outputs: list[Output] = []
    for path, members in sorted(grouped.items()):
        members.sort(key=lambda fragment: rank[fragment.owner])
        package = _package_of(path, packaged)
        seen = (
            {**data, **(package_data or {}).get(package, {}), "package": package}
            if package
            else dict(data)
        )
        contributions = [fragment for fragment in members if fragment.contributes]
        templates = [fragment for fragment in members if not fragment.contributes]
        if contributions or (len(templates) == 1 and _collects(templates[0], seen)):
            if len(templates) != 1:
                fail(
                    f"{path}: contributions need one template to render them;"
                    f" it has {len(templates)}"
                )
            merged = _contributed(path, contributions, seen)
            seen = {
                **seen,
                "contributed": merged,
                "contributed_entries": _entries(merged),
            }
            text = _render(templates[0], seen)
        else:
            seen = {**seen, "contributed": {}, "contributed_entries": ""}
            parts = [(fragment, _render(fragment, seen)) for fragment in members]
            text = _compose(path, parts)
        committed = root / path
        if committed.is_file():
            text = _splice(text, committed.read_text(encoding="utf-8"))
        outputs.append(
            Output(path, text.encode(), tuple(fragment.ref for fragment in members))
        )
    return tuple(outputs)

read_rendered

read_rendered(directory: Path) -> dict[str, str]

The receipts in directory, name to digest; empty without any.

Source code in packages/workshop/src/livery/workshop/_fragment_engine.py
def read_rendered(directory: Path) -> dict[str, str]:
    """The receipts in *directory*, name to digest; empty without any."""
    path = directory / RENDERED_MANIFEST
    return receipts_from(path.read_text("utf-8")) if path.is_file() else {}

receipts_from

receipts_from(text: str) -> dict[str, str]

The receipts text spells, name to digest; empty for anything else.

Source code in packages/workshop/src/livery/workshop/_fragment_engine.py
def receipts_from(text: str) -> dict[str, str]:
    """The receipts *text* spells, name to digest; empty for anything else."""
    try:
        loaded = json.loads(text)
    except ValueError:
        return {}
    if not isinstance(loaded, dict):
        return {}
    return {str(k): str(v) for k, v in cast("dict[object, object]", loaded).items()}

receipt_of

receipt_of(path: Path) -> str | None

The receipt path earns as it stands, its regions left out; None if absent.

Source code in packages/workshop/src/livery/workshop/_fragment_engine.py
def receipt_of(path: Path) -> str | None:
    """The receipt *path* earns as it stands, its regions left out; None if absent."""
    return _owned(_committed(path)) if path.is_file() else None

merge_receipts

merge_receipts(base: Mapping[str, str], ours: Mapping[str, str], theirs: Mapping[str, str], *, beside: Path) -> dict[str, str]

Two sides' receipts merged key by key, over the base they share.

A receipt one side changed takes that side's digest, a removal included, and one both changed alike takes it too. One both changed differently takes the digest of its file as the merge left it in beside, the directory the receipts describe, and goes when the file did: a digest that does not match its file reads as the repository's edit, and the next render would keep that file instead of rewriting it.

Source code in packages/workshop/src/livery/workshop/_fragment_engine.py
def merge_receipts(
    base: Mapping[str, str],
    ours: Mapping[str, str],
    theirs: Mapping[str, str],
    *,
    beside: Path,
) -> dict[str, str]:
    """Two sides' receipts merged key by key, over the *base* they share.

    A receipt one side changed takes that side's digest, a removal
    included, and one both changed alike takes it too. One both
    changed differently takes the digest of its file as the merge left
    it in *beside*, the directory the receipts describe, and goes when
    the file did: a digest that does not match its file reads as the
    repository's edit, and the next render would keep that file instead
    of rewriting it.
    """
    merged: dict[str, str] = {}
    for name in sorted({*base, *ours, *theirs}):
        was, mine, yours = base.get(name), ours.get(name), theirs.get(name)
        if mine == yours or yours == was:
            chosen = mine
        elif mine == was:
            chosen = yours
        else:
            chosen = receipt_of(beside / name)
        if chosen is not None:
            merged[name] = chosen
    return merged

write_rendered

write_rendered(directory: Path, receipts: Mapping[str, str]) -> None

Write the receipts, or remove the file when there are none.

Source code in packages/workshop/src/livery/workshop/_fragment_engine.py
def write_rendered(directory: Path, receipts: Mapping[str, str]) -> None:
    """Write the receipts, or remove the file when there are none."""
    path = directory / RENDERED_MANIFEST
    if not receipts:
        if path.is_file():
            path.unlink()
        return
    path.write_bytes(
        (json.dumps(dict(sorted(receipts.items())), indent=2) + "\n").encode()
    )

write_root_file

write_root_file(root: Path, output: Output) -> list[str]

Write one file at the root as apply writes it, and its receipt; the lines.

output is a file a commit holds, at the root, never in a package. Every other file and receipt stays as it is: unlike apply, this withdraws nothing, so it writes one file while the rest of the delivery waits. A file edited here is kept and named, as apply keeps it.

Source code in packages/workshop/src/livery/workshop/_fragment_engine.py
def write_root_file(root: Path, output: Output) -> list[str]:
    """Write one file at the root as `apply` writes it, and its receipt; the lines.

    *output* is a file a commit holds, at the root, never in a package.
    Every other file and receipt stays as it is: unlike `apply`, this
    withdraws nothing, so it writes one file while the rest of the
    delivery waits. A file edited here is kept and named, as `apply`
    keeps it.
    """
    receipts = read_rendered(root)
    held, line = _write(root, output, receipts.get(output.path))
    if held:
        receipts[output.path] = _owned(output.body)
        write_rendered(root, receipts)
    return [line] if line else []

apply

apply(root: Path, outputs: Sequence[Output], *, packages: Sequence[str] = ()) -> list[str]

Write outputs, withdraw what is gone, and say what changed.

A file is written when it is missing, and rewritten when its bytes are still what the engine last wrote. A file edited since is kept and named, and so is a file the engine has no receipt for whose bytes differ. A receipted file no output renders any more is removed when unedited and kept and named otherwise; either way its receipt goes, so a kept file is the repository's from then on.

Parameters:

Name Type Description Default
root Path

The workspace root.

required
outputs Sequence[Output]

What plan rendered.

required
packages Sequence[str]

The package paths whose receipts the engine keeps, relative to the root.

()

Returns:

Type Description
list[str]

One line per file written, removed or kept, in path order.

Source code in packages/workshop/src/livery/workshop/_fragment_engine.py
def apply(
    root: Path,
    outputs: Sequence[Output],
    *,
    packages: Sequence[str] = (),
) -> list[str]:
    """Write *outputs*, withdraw what is gone, and say what changed.

    A file is written when it is missing, and rewritten when its bytes
    are still what the engine last wrote. A file edited since is kept
    and named, and so is a file the engine has no receipt for whose
    bytes differ. A receipted file no output renders any more is
    removed when unedited and kept and named otherwise; either way its
    receipt goes, so a kept file is the repository's from then on.

    Args:
        root: The workspace root.
        outputs: What `plan` rendered.
        packages: The package paths whose receipts the engine keeps,
            relative to the root.

    Returns:
        One line per file written, removed or kept, in path order.
    """
    lines = _apply_local(root, [output for output in outputs if output.local])
    outputs = [output for output in outputs if not output.local]
    homes = ["", *sorted(packages, key=len, reverse=True)]

    def home_of(path: str) -> str:
        for home in homes[1:]:
            if path.startswith(f"{home}/"):
                return home
        return ""

    receipts = {home: read_rendered(root / home) for home in homes}
    rendered: set[str] = set()
    for output in outputs:
        rendered.add(output.path)
        home = home_of(output.path)
        name = output.path.removeprefix(f"{home}/") if home else output.path
        held, line = _write(root, output, receipts[home].get(name))
        if line:
            lines.append(line)
        if held:
            receipts[home][name] = _owned(output.body)
    for home in homes:
        for name, recorded in sorted(receipts[home].items()):
            path = f"{home}/{name}" if home else name
            if path in rendered:
                continue
            del receipts[home][name]
            target = root / path
            if not target.is_file():
                continue
            if _owned(_committed(target)) == recorded:
                target.unlink()
                lines.append(f"  removed {path}: no listed extension renders it")
            else:
                lines.append(
                    f"  kept {path}: no listed extension renders it, and it was"
                    " edited here, so it stays as the repository's own"
                )
        write_rendered(root / home, receipts[home])
    return sorted(lines, key=lambda line: line.split()[1])

local_receipts

local_receipts(root: Path) -> dict[str, str]

What the engine linked or copied for this checkout alone, path to receipt.

Source code in packages/workshop/src/livery/workshop/_fragment_engine.py
def local_receipts(root: Path) -> dict[str, str]:
    """What the engine linked or copied for this checkout alone, path to receipt."""
    return _read_local(root)

apply_untracked

apply_untracked(root: Path, outputs: Sequence[Output]) -> list[str]

Write the local outputs git does not track, and nothing else.

The write fm sync --locked makes: a checkout's own files (the agent's fragments, skills and settings) are made current, and no file a commit holds is written, removed or receipted. A local output git tracks is left as it is; the drift check names it.

Source code in packages/workshop/src/livery/workshop/_fragment_engine.py
def apply_untracked(root: Path, outputs: Sequence[Output]) -> list[str]:
    """Write the local *outputs* git does not track, and nothing else.

    The write `fm sync --locked` makes: a checkout's own files (the
    agent's fragments, skills and settings) are made current, and no
    file a commit holds is written, removed or receipted. A local
    output git tracks is left as it is; the drift check names it.
    """
    # Even with nothing of its own, the delivery runs: it withdraws what
    # an earlier one wrote and leaves the receipt that says it ran.
    local = [output for output in outputs if output.local]
    tracked = set(tracked_local(root, local))
    return _apply_local(
        root, [output for output in local if output.path not in tracked]
    )

tracked_local

tracked_local(root: Path, outputs: Sequence[Output]) -> list[str]

The local outputs among outputs that git tracks, in path order.

A local output is the checkout's own, written for it alone and ignored through its directory's managed .gitignore; a tracked one was committed before that, and stays tracked whatever the ignore file says. Empty when git cannot answer.

Source code in packages/workshop/src/livery/workshop/_fragment_engine.py
def tracked_local(root: Path, outputs: Sequence[Output]) -> list[str]:
    """The local outputs among *outputs* that git tracks, in path order.

    A local output is the checkout's own, written for it alone and
    ignored through its directory's managed `.gitignore`; a tracked one
    was committed before that, and stays tracked whatever the ignore
    file says. Empty when git cannot answer.
    """
    from livery import git

    local = [output.path for output in outputs if output.local]
    if not local:
        return []
    try:
        return sorted(set(git.ls_files(root, *local)) & set(local))
    except git.GitError:
        return []

drift

drift(root: Path, outputs: Sequence[Output], *, packages: Sequence[str] = ()) -> list[str]

One line per file whose committed bytes are not what it renders.

A receipted file no output renders any more is drift too while it is unedited, since the next apply removes it; an edited one is the repository's and says nothing.

Source code in packages/workshop/src/livery/workshop/_fragment_engine.py
def drift(
    root: Path, outputs: Sequence[Output], *, packages: Sequence[str] = ()
) -> list[str]:
    """One line per file whose committed bytes are not what it renders.

    A receipted file no output renders any more is drift too while it is
    unedited, since the next apply removes it; an edited one is the
    repository's and says nothing.
    """
    lines: list[str] = []
    # What a checkout holds for itself alone is never committed, so the
    # gate does not judge it.
    outputs = [output for output in outputs if not output.local]
    rendered = {output.path for output in outputs}
    for home in ["", *packages]:
        for name, recorded in sorted(read_rendered(root / home).items()):
            path = f"{home}/{name}" if home else name
            target = root / path
            if path in rendered or not target.is_file():
                continue
            if _owned(_committed(target)) == recorded:
                lines.append(
                    f"  {path}: written for an owner no longer listed;"
                    f" `{prog()} sync` removes it"
                )
    for output in outputs:
        target = root / output.path
        if not target.is_file():
            lines.append(f"  {output.path}: missing; `{prog()} sync` writes it")
        elif _committed(target) != output.body:
            owners = ", ".join(output.owners)
            lines.append(f"  {output.path}: differs from what {owners} render")
    return lines