Skip to content

livery.extensions.docs._llms

livery.extensions.docs._llms

The agent-facing site files: llms.txt and llms-full.txt.

Ported from footman's generator. llms.txt (https://llmstxt.org) is the index: every page of the merged nav in order, each with a one-line description pulled from its first prose paragraph and an absolute link. llms-full.txt is the authored text in one file, snippet includes resolved inline. The machine-appended sections (package changelogs, the release view, the API trees) are excluded from the full file by construction: a context file that concatenates history teaches superseded behaviour as current, so only what an author put in a nav belongs in a model's context. The index still links every page, the changelogs included.

Both files are written into the built site's root after the site build; they are derived, never hand-edited, and never committed.

Functions:

Name Description
page_url

The published URL for page: a directory index lives at the directory.

first_sentence

The first sentence of the page's first prose paragraph.

resolve_snippets

Inline --8<-- "path" includes, the way the site build does.

llms_files

The two files' bodies: (index, full).

write_llms_files

Write both files into the built site's root; the names written.

Attributes:

Name Type Description
MACHINE_PREFIXES

MACHINE_PREFIXES module-attribute

MACHINE_PREFIXES = ('releases/',)

page_url

page_url(site: str, page: str) -> str

The published URL for page: a directory index lives at the directory.

Source code in packages/extensions/docs/src/livery/extensions/docs/_llms.py
def page_url(site: str, page: str) -> str:
    """The published URL for *page*: a directory index lives at the directory."""
    if page == "index.md" or page.endswith("/index.md"):
        stem = page.removesuffix("index.md").rstrip("/")
        return f"{site}{stem + '/' if stem else ''}"
    return f"{site}{page.removesuffix('.md')}/"

first_sentence

first_sentence(text: str, site: str, page: str) -> str

The first sentence of the page's first prose paragraph.

Prose, strictly: headings, list items, table rows, admonition bodies, and fenced code are structure, not description; served as a page's one-line summary they read as junk in every agent index built on the file.

Source code in packages/extensions/docs/src/livery/extensions/docs/_llms.py
def first_sentence(text: str, site: str, page: str) -> str:
    """The first sentence of the page's first prose paragraph.

    Prose, strictly: headings, list items, table rows, admonition
    bodies, and fenced code are structure, not description; served as
    a page's one-line summary they read as junk in every agent index
    built on the file.
    """
    if text.startswith("---\n"):
        _, _, text = text.partition("\n---\n")
    skip = ("#", "---", "[![", "!!!", ">", "<", "--8<--", "|", "- ", "* ")
    fenced = False
    admonition = False
    hanging = False
    paragraph: list[str] = []
    for raw in text.splitlines():
        line = raw.strip()
        if admonition:
            if not line or raw.startswith((" ", "\t")):
                continue
            admonition = False
        if line.startswith("!!!"):
            admonition = True
            continue
        if line.startswith("```"):
            fenced = not fenced
            continue
        digits = line.split(".", 1)[0]
        numbered = digits.isdigit() and line.startswith(f"{digits}.")
        if fenced or line.startswith(skip) or numbered:
            if paragraph:
                break
            # A wrapped list item or table row continues on the next
            # line without its marker; serving that tail as prose put
            # mid-sentence fragments in the index.
            hanging = True
            continue
        if not line:
            if paragraph:
                break
            hanging = False
            continue
        if hanging:
            continue
        paragraph.append(line)
    prose = _absolute_links(" ".join(paragraph), site, page)
    end = prose.find(". ")
    return prose[: end + 1] if end != -1 else prose

resolve_snippets

resolve_snippets(text: str, root: Path) -> str

Inline --8<-- "path" includes, the way the site build does.

Paths are repo-relative, matching the snippets configuration; a missing file keeps the directive, so the gap is visible rather than silent.

Source code in packages/extensions/docs/src/livery/extensions/docs/_llms.py
def resolve_snippets(text: str, root: Path) -> str:
    """Inline ``--8<-- "path"`` includes, the way the site build does.

    Paths are repo-relative, matching the snippets configuration; a
    missing file keeps the directive, so the gap is visible rather
    than silent.
    """

    def inline(match: re.Match[str]) -> str:
        path = root / match.group(1)
        if not path.is_file():
            return match.group(0)
        return resolve_snippets(path.read_text(encoding="utf-8").rstrip(), root)

    return re.sub(r"""^--8<-- ["']([^"']+)["']$""", inline, text, flags=re.MULTILINE)

llms_files

llms_files(root: Path) -> tuple[str, str]

The two files' bodies: (index, full).

Derived from the same emitted config the site builds from, so the index can never disagree with the nav. Without a declared site URL the links are root-relative, which still resolves on whatever host serves the site.

Source code in packages/extensions/docs/src/livery/extensions/docs/_llms.py
def llms_files(root: Path) -> tuple[str, str]:
    """The two files' bodies: (index, full).

    Derived from the same emitted config the site builds from, so
    the index can never disagree with the nav. Without a declared
    site URL the links are root-relative, which still resolves on
    whatever host serves the site.
    """
    table = docs_table(root)
    title = str(table.get("title", "")) or project_name(root)
    description = str(table.get("description", ""))
    site = str(table.get("site-url", "")) or "/"
    if not site.endswith("/"):
        site += "/"
    nav = tomllib.loads(zensical_config(root))["project"]["nav"]
    pages = _nav_pages(nav)
    members = tuple(package.member for package in discover_packages(root))
    index = [f"# {title}", ""]
    if description:
        index += [f"> {description}", ""]
    index += ["## Docs", ""]
    full = [f"# {title} - full documentation", ""]
    for page_title, page in pages:
        source = root / "docs" / page
        if not source.is_file():
            continue
        resolved = resolve_snippets(source.read_text(encoding="utf-8").rstrip(), root)
        url = page_url(site, page)
        summary = first_sentence(resolved, site, page)
        index.append(
            f"- [{page_title}]({url}): {summary}"
            if summary
            else f"- [{page_title}]({url})"
        )
        if _machine_page(page, members):
            continue
        full += ["", "---", "", f"<!-- {page_title} - {url} -->", "", resolved]
    return ("\n".join(index) + "\n", "\n".join(full) + "\n")

write_llms_files

write_llms_files(root: Path) -> list[str]

Write both files into the built site's root; the names written.

The site must already be built: these land beside its pages the way any root asset does, and a clean rebuild regenerates them.

Source code in packages/extensions/docs/src/livery/extensions/docs/_llms.py
def write_llms_files(root: Path) -> list[str]:
    """Write both files into the built site's root; the names written.

    The site must already be built: these land beside its pages the
    way any root asset does, and a clean rebuild regenerates them.
    """
    site_dir = root / "site"
    site_dir.mkdir(exist_ok=True)
    index, full = llms_files(root)
    (site_dir / "llms.txt").write_text(index, encoding="utf-8")
    (site_dir / "llms-full.txt").write_text(full, encoding="utf-8")
    return ["llms.txt", "llms-full.txt"]