Skip to content

livery.extensions.mkdocstrings._refs

livery.extensions.mkdocstrings._refs

The cross-references in docstrings, judged against the anchors the site renders.

A docstring names another object as [pkg.module.Name][], and the site build links the name to the anchor of the object's heading. A name with no anchor stops the strict build, whether or not the object exists. An anchor exists for each object a ::: directive renders, and for each member the python handler renders with it. The API reference has a page of one directive per module of each package it documents, and an authored page may carry directives of its own. The handler's options and the docs.members policy decide which members render: under the public policy an underscore function has no anchor, and a module no page renders, such as one of an extension that ships inside another member's wheel, has none.

lint.docrefs reads every docstring the site renders. On the API pages and the authored pages, those are the docstrings of what they render. On the task reference, they are the parts of a task's docstring its page shows: its summary, its long description, its parameters and its return value. Each reference there names an anchor, or it refuses with the docstring's place and the page it renders on. A docstring the site renders nowhere is not judged. Two kinds of reference are left to the strict build: one outside every member's namespace, which the site resolves against its inventories, and one into a member that declines its reference with [docs] api = false, since that member documents itself through pages its own generators write while the site builds, which the gate does not run. The pages render with the handler's options as the site's config sets them; options written under a ::: line of an authored page are not read.

The site is computed in a child process of the workspace's interpreter. It reads the sources with griffe, the library the site build reads them with, and mounts each task provider the members advertise with footman, as the task reference does.

Classes:

Name Description
Reference

One cross-reference in a source's docstrings.

Functions:

Name Description
references_in

The paths text references outside its code.

docstring_references

Each cross-reference in source's docstrings, with its place.

namespaces

The top-level names the members' sources provide.

site_request

What the child computes the site from: the inputs the docs build reads.

reference_problems

One line per reference in a docstring the site renders that names no anchor.

docrefs_run

Refuse every cross-reference the site cannot link: the lint phase's step.

main

The probe: the site the request on stdin describes, as JSON on stdout.

Attributes:

Name Type Description
REFERENCE
DIRECTIVE
DEFAULT_FILTERS

REFERENCE module-attribute

REFERENCE = re.compile('\\[(?P<path>[A-Za-z_]\\w*(?:\\.\\w+)+)\\]\\[\\]')

DIRECTIVE module-attribute

DIRECTIVE = re.compile('^(?:#{1,6} *)?::: ?(?P<name>\\S+) *$', re.M)

DEFAULT_FILTERS module-attribute

DEFAULT_FILTERS = ('!^_[^_]',)

Reference

Bases: NamedTuple

One cross-reference in a source's docstrings.

Attributes:

Name Type Description
docstring int

The line its docstring starts on, which is the line griffe gives the docstring when the site reads it.

line int

The line the reference is on.

path str

The dotted path it names.

docstring instance-attribute

docstring: int

line instance-attribute

line: int

path instance-attribute

path: str

references_in

references_in(text: str) -> set[str]

The paths text references outside its code.

Source code in packages/extensions/mkdocstrings/src/livery/extensions/mkdocstrings/_refs.py
def references_in(text: str) -> set[str]:
    """The paths *text* references outside its code."""
    return {match["path"] for match in REFERENCE.finditer(_without_code(text))}

docstring_references

docstring_references(source: Path) -> list[Reference]

Each cross-reference in source's docstrings, with its place.

A docstring is any string literal standing as a statement: a module's, a class's or a function's first, or one under an assignment, which the site reads as that name's. A source that does not parse has none here; the syntax gate names it.

Source code in packages/extensions/mkdocstrings/src/livery/extensions/mkdocstrings/_refs.py
def docstring_references(source: Path) -> list[Reference]:
    """Each cross-reference in *source*'s docstrings, with its place.

    A docstring is any string literal standing as a statement: a
    module's, a class's or a function's first, or one under an
    assignment, which the site reads as that name's. A source that
    does not parse has none here; the syntax gate names it.
    """
    try:
        tree = ast.parse(source.read_text("utf-8"))
    except (SyntaxError, UnicodeDecodeError):
        return []
    found: list[Reference] = []
    for node in ast.walk(tree):
        if not (
            isinstance(node, ast.Expr)
            and isinstance(node.value, ast.Constant)
            and isinstance(node.value.value, str)
        ):
            continue
        text = _without_code(node.value.value)
        for match in REFERENCE.finditer(text):
            line = node.lineno + text.count("\n", 0, match.start())
            found.append(Reference(node.lineno, line, match["path"]))
    return found

namespaces

namespaces(sources: list[Path]) -> set[str]

The top-level names the members' sources provide.

A package directory, namespace or regular, gives its name, and so does a module that is a distribution's whole source (tool.py gives tool).

Source code in packages/extensions/mkdocstrings/src/livery/extensions/mkdocstrings/_refs.py
def namespaces(sources: list[Path]) -> set[str]:
    """The top-level names the members' sources provide.

    A package directory, namespace or regular, gives its name, and so
    does a module that is a distribution's whole source (``tool.py``
    gives ``tool``).
    """
    names: set[str] = set()
    for src in sources:
        for child in src.iterdir():
            if child.is_dir() and child.name.isidentifier():
                names.add(child.name)
            elif child.suffix == ".py" and child.stem.isidentifier():
                names.add(child.stem)
    return names

site_request

site_request(root: Path) -> dict[str, Any]

What the child computes the site from: the inputs the docs build reads.

The members' sources; every ::: directive the python handler renders, with the page it publishes on, the API reference's one per module of each package it documents and then the authored pages'; the handler's options with the members policy applied, as the site's config writes them; the task providers each member advertises, with the member, while the root contract turns the task reference's step on; and the sources of the members that decline their reference.

Source code in packages/extensions/mkdocstrings/src/livery/extensions/mkdocstrings/_refs.py
def site_request(root: Path) -> dict[str, Any]:
    """What the child computes the site from: the inputs the docs build reads.

    The members' sources; every ``:::`` directive the python handler
    renders, with the page it publishes on, the API reference's one
    per module of each package it documents and then the authored
    pages'; the handler's options with the members policy applied, as
    the site's config writes them; the task providers each member
    advertises, with the member, while the root contract turns the
    task reference's step on; and the sources of the members that
    decline their reference.
    """
    from livery.extensions.docs import TASK_REFERENCE, advertised_providers
    from livery.extensions.mkdocstrings._config import options as handler_options
    from livery.extensions.mkdocstrings._pages import (
        API_DIR,
        declines_api,
        documented,
        pages,
    )
    from livery.workshop import discover_packages, step_enabled

    packages = discover_packages(root)
    directives: list[tuple[str, str]] = []
    reached = [package for package in packages if documented(package)]
    for package in reached:
        directives += [
            (f"packages/{package.member}/{API_DIR}/{page}", dotted)
            for page, dotted in pages(package)
        ]
    for page, path in _authored_pages(root, packages):
        text = _without_code(path.read_text("utf-8"))
        directives += [(page, match["name"]) for match in DIRECTIVE.finditer(text)]
    with_sources = [
        package for package in packages if (package.directory / "src").is_dir()
    ]
    referenced = step_enabled(root, TASK_REFERENCE)
    return {
        "sources": [
            str((package.directory / "src").resolve()) for package in with_sources
        ],
        "pages": directives,
        "options": handler_options(root) if reached else {},
        "providers": [
            (identity, package.member)
            for package in (packages if referenced else ())
            for identity in advertised_providers(package)
        ],
        "declined": [
            str((package.directory / "src").resolve())
            for package in with_sources
            if declines_api(package)
        ],
    }

reference_problems

reference_problems(root: Path) -> list[str]

One line per reference in a docstring the site renders that names no anchor.

Every member's sources are read, since a change anywhere may take an anchor away or put a docstring on a page. Each line names the reference's place, the reference, and the pages its docstring renders on. When no docstring holds a reference under a member's namespace, nothing is started.

Source code in packages/extensions/mkdocstrings/src/livery/extensions/mkdocstrings/_refs.py
def reference_problems(root: Path) -> list[str]:
    """One line per reference in a docstring the site renders that names no anchor.

    Every member's sources are read, since a change anywhere may take
    an anchor away or put a docstring on a page. Each line names the
    reference's place, the reference, and the pages its docstring
    renders on. When no docstring holds a reference under a member's
    namespace, nothing is started.
    """
    from livery.workshop import discover_packages

    sources = [
        package.directory / "src"
        for package in discover_packages(root)
        if (package.directory / "src").is_dir()
    ]
    judged = namespaces(sources)
    found = [
        (source, reference)
        for src in sources
        for source in sorted(src.rglob("*.py"))
        for reference in docstring_references(source)
        if reference.path.split(".")[0] in judged
    ]
    if not found:
        return []
    request = site_request(root)
    request["paths"] = sorted({reference.path for _, reference in found})
    request["files"] = sorted({str(source.resolve()) for source, _ in found})
    answer = _probe(root, request)
    missing = {str(path) for path in answer["missing"]}
    shown: dict[tuple[str, int], list[tuple[str, set[str] | None]]] = {}
    for file, line, page, paths in answer["rendered"]:
        shown.setdefault((str(file), int(line)), []).append(
            (str(page), None if paths is None else {str(path) for path in paths})
        )
    problems: list[str] = []
    for source, reference in found:
        if reference.path not in missing:
            continue
        place = (str(source.resolve()), reference.docstring)
        pages = sorted(
            {
                page
                for page, paths in shown.get(place, ())
                if paths is None or reference.path in paths
            }
        )
        if pages:
            problems.append(
                f"{source.relative_to(root).as_posix()}:{reference.line}:"
                f" [{reference.path}][] has no anchor on the site, where the"
                f" docstring renders on {', '.join(pages)}; write the name as"
                f" plain code instead: `{reference.path}`"
            )
    return problems

docrefs_run

docrefs_run(ctx: PhaseContext) -> None

Refuse every cross-reference the site cannot link: the lint phase's step.

The lines of reference_problems over the workspace, under one heading.

Source code in packages/extensions/mkdocstrings/src/livery/extensions/mkdocstrings/_refs.py
def docrefs_run(ctx: PhaseContext) -> None:
    """Refuse every cross-reference the site cannot link: the lint phase's step.

    The lines of `reference_problems` over the workspace, under one
    heading.
    """
    from livery.footman import fail

    problems = reference_problems(ctx.root)
    if problems:
        fail("cross-references the site cannot link:\n  " + "\n  ".join(problems))

main

main() -> None

The probe: the site the request on stdin describes, as JSON on stdout.

It answers with the docstrings of the requested files the site renders, each with its page, and with the requested paths that name no anchor and lie in no member that declines its reference.

Source code in packages/extensions/mkdocstrings/src/livery/extensions/mkdocstrings/_refs.py
def main() -> None:
    """The probe: the site the request on stdin describes, as JSON on stdout.

    It answers with the docstrings of the requested files the site
    renders, each with its page, and with the requested paths that
    name no anchor and lie in no member that declines its reference.
    """
    import griffe

    request = json.load(sys.stdin)
    sources = [Path(src) for src in request["sources"]]
    loader = griffe.GriffeLoader(
        search_paths=[str(src) for src in sources], allow_inspection=False
    )
    for top in sorted(namespaces(sources)):
        with contextlib.suppress(ImportError, KeyError):
            # A dotted name, never a path: the working directory has no say.
            loader.load(top, try_relative_path=False)
    loader.resolve_aliases(implicit=False, external=None)
    walk = _Walk(griffe, request["options"])
    collection = loader.modules_collection
    for page, identifier in request["pages"]:
        try:
            found = collection.get_member(identifier)
        except walk.missing:
            continue
        walk.render(found, page)
    files = set(request["files"])
    rendered: list[tuple[str, int, str, list[str] | None]] = [
        (file, line, page, None)
        for (file, line), pages in sorted(walk.rendered.items())
        if file in files
        for page in sorted(pages)
    ]
    providers = [
        (str(identity), str(member)) for identity, member in request["providers"]
    ]
    rendered += [entry for entry in _task_docstrings(providers) if entry[0] in files]
    declined = [Path(src) for src in request["declined"]]
    missing = [
        path
        for path in request["paths"]
        if path not in walk.anchors
        and not _declined(collection, path, declined, walk.missing)
    ]
    json.dump({"rendered": rendered, "missing": missing}, sys.stdout)