The cross-references in docstrings, judged against the anchors the site renders.
A docstring names another object as [pkg.module.Name][], and the
site build links the name to the anchor of the object's heading. A
name with no anchor stops the strict build, whether or not the object
exists. An anchor exists for each object a ::: directive renders,
and for each member the python handler renders with it. The API
reference has a page of one directive per module of each package it
documents, and an authored page may carry directives of its own. The
handler's options and the docs.members policy decide which members
render: under the public policy an underscore function has no
anchor, and a module no page renders, such as one of an extension that
ships inside another member's wheel, has none.
lint.docrefs reads every docstring the site renders. On the API
pages and the authored pages, those are the docstrings of what they
render. On the task reference, they are the parts of a task's
docstring its page shows: its summary, its long description, its
parameters and its return value. Each reference there names an
anchor, or it refuses with the docstring's place and the page it
renders on. A docstring the site renders nowhere is not judged. Two
kinds of reference are left to the strict build: one outside every
member's namespace, which the site resolves against its inventories,
and one into a member that declines its reference with
[docs] api = false, since that member documents itself through
pages its own generators write while the site builds, which the gate
does not run. The pages render with the handler's options as the
site's config sets them; options written under a ::: line of an
authored page are not read.
The site is computed in a child process of the workspace's
interpreter. It reads the sources with griffe, the library the site
build reads them with, and mounts each task provider the members
advertise with footman, as the task reference does.
Classes:
| Name |
Description |
Reference |
One cross-reference in a source's docstrings.
|
Functions:
| Name |
Description |
references_in |
The paths text references outside its code.
|
docstring_references |
Each cross-reference in source's docstrings, with its place.
|
namespaces |
The top-level names the members' sources provide.
|
site_request |
What the child computes the site from: the inputs the docs build reads.
|
reference_problems |
One line per reference in a docstring the site renders that names no anchor.
|
docrefs_run |
Refuse every cross-reference the site cannot link: the lint phase's step.
|
main |
The probe: the site the request on stdin describes, as JSON on stdout.
|
Attributes:
REFERENCE
module-attribute
REFERENCE = re.compile('\\[(?P<path>[A-Za-z_]\\w*(?:\\.\\w+)+)\\]\\[\\]')
DIRECTIVE
module-attribute
DIRECTIVE = re.compile('^(?:#{1,6} *)?::: ?(?P<name>\\S+) *$', re.M)
DEFAULT_FILTERS
module-attribute
DEFAULT_FILTERS = ('!^_[^_]',)
Reference
Bases: NamedTuple
One cross-reference in a source's docstrings.
Attributes:
| Name |
Type |
Description |
docstring |
int
|
The line its docstring starts on, which is the line
griffe gives the docstring when the site reads it.
|
line |
int
|
The line the reference is on.
|
path |
str
|
The dotted path it names.
|
docstring
instance-attribute
references_in
The paths text references outside its code.
Source code in packages/extensions/mkdocstrings/src/livery/extensions/mkdocstrings/_refs.py
| def references_in(text: str) -> set[str]:
"""The paths *text* references outside its code."""
return {match["path"] for match in REFERENCE.finditer(_without_code(text))}
|
docstring_references
Each cross-reference in source's docstrings, with its place.
A docstring is any string literal standing as a statement: a
module's, a class's or a function's first, or one under an
assignment, which the site reads as that name's. A source that
does not parse has none here; the syntax gate names it.
Source code in packages/extensions/mkdocstrings/src/livery/extensions/mkdocstrings/_refs.py
| def docstring_references(source: Path) -> list[Reference]:
"""Each cross-reference in *source*'s docstrings, with its place.
A docstring is any string literal standing as a statement: a
module's, a class's or a function's first, or one under an
assignment, which the site reads as that name's. A source that
does not parse has none here; the syntax gate names it.
"""
try:
tree = ast.parse(source.read_text("utf-8"))
except (SyntaxError, UnicodeDecodeError):
return []
found: list[Reference] = []
for node in ast.walk(tree):
if not (
isinstance(node, ast.Expr)
and isinstance(node.value, ast.Constant)
and isinstance(node.value.value, str)
):
continue
text = _without_code(node.value.value)
for match in REFERENCE.finditer(text):
line = node.lineno + text.count("\n", 0, match.start())
found.append(Reference(node.lineno, line, match["path"]))
return found
|
namespaces
The top-level names the members' sources provide.
A package directory, namespace or regular, gives its name, and so
does a module that is a distribution's whole source (tool.py
gives tool).
Source code in packages/extensions/mkdocstrings/src/livery/extensions/mkdocstrings/_refs.py
| def namespaces(sources: list[Path]) -> set[str]:
"""The top-level names the members' sources provide.
A package directory, namespace or regular, gives its name, and so
does a module that is a distribution's whole source (``tool.py``
gives ``tool``).
"""
names: set[str] = set()
for src in sources:
for child in src.iterdir():
if child.is_dir() and child.name.isidentifier():
names.add(child.name)
elif child.suffix == ".py" and child.stem.isidentifier():
names.add(child.stem)
return names
|
site_request
What the child computes the site from: the inputs the docs build reads.
The members' sources; every ::: directive the python handler
renders, with the page it publishes on, the API reference's one
per module of each package it documents and then the authored
pages'; the handler's options with the members policy applied, as
the site's config writes them; the task providers each member
advertises, with the member, while the root contract turns the
task reference's step on; and the sources of the members that
decline their reference.
Source code in packages/extensions/mkdocstrings/src/livery/extensions/mkdocstrings/_refs.py
| def site_request(root: Path) -> dict[str, Any]:
"""What the child computes the site from: the inputs the docs build reads.
The members' sources; every ``:::`` directive the python handler
renders, with the page it publishes on, the API reference's one
per module of each package it documents and then the authored
pages'; the handler's options with the members policy applied, as
the site's config writes them; the task providers each member
advertises, with the member, while the root contract turns the
task reference's step on; and the sources of the members that
decline their reference.
"""
from livery.extensions.docs import TASK_REFERENCE, advertised_providers
from livery.extensions.mkdocstrings._config import options as handler_options
from livery.extensions.mkdocstrings._pages import (
API_DIR,
declines_api,
documented,
pages,
)
from livery.workshop import discover_packages, step_enabled
packages = discover_packages(root)
directives: list[tuple[str, str]] = []
reached = [package for package in packages if documented(package)]
for package in reached:
directives += [
(f"packages/{package.member}/{API_DIR}/{page}", dotted)
for page, dotted in pages(package)
]
for page, path in _authored_pages(root, packages):
text = _without_code(path.read_text("utf-8"))
directives += [(page, match["name"]) for match in DIRECTIVE.finditer(text)]
with_sources = [
package for package in packages if (package.directory / "src").is_dir()
]
referenced = step_enabled(root, TASK_REFERENCE)
return {
"sources": [
str((package.directory / "src").resolve()) for package in with_sources
],
"pages": directives,
"options": handler_options(root) if reached else {},
"providers": [
(identity, package.member)
for package in (packages if referenced else ())
for identity in advertised_providers(package)
],
"declined": [
str((package.directory / "src").resolve())
for package in with_sources
if declines_api(package)
],
}
|
reference_problems
One line per reference in a docstring the site renders that names no anchor.
Every member's sources are read, since a change anywhere may take
an anchor away or put a docstring on a page. Each line names the
reference's place, the reference, and the pages its docstring
renders on. When no docstring holds a reference under a member's
namespace, nothing is started.
Source code in packages/extensions/mkdocstrings/src/livery/extensions/mkdocstrings/_refs.py
| def reference_problems(root: Path) -> list[str]:
"""One line per reference in a docstring the site renders that names no anchor.
Every member's sources are read, since a change anywhere may take
an anchor away or put a docstring on a page. Each line names the
reference's place, the reference, and the pages its docstring
renders on. When no docstring holds a reference under a member's
namespace, nothing is started.
"""
from livery.workshop import discover_packages
sources = [
package.directory / "src"
for package in discover_packages(root)
if (package.directory / "src").is_dir()
]
judged = namespaces(sources)
found = [
(source, reference)
for src in sources
for source in sorted(src.rglob("*.py"))
for reference in docstring_references(source)
if reference.path.split(".")[0] in judged
]
if not found:
return []
request = site_request(root)
request["paths"] = sorted({reference.path for _, reference in found})
request["files"] = sorted({str(source.resolve()) for source, _ in found})
answer = _probe(root, request)
missing = {str(path) for path in answer["missing"]}
shown: dict[tuple[str, int], list[tuple[str, set[str] | None]]] = {}
for file, line, page, paths in answer["rendered"]:
shown.setdefault((str(file), int(line)), []).append(
(str(page), None if paths is None else {str(path) for path in paths})
)
problems: list[str] = []
for source, reference in found:
if reference.path not in missing:
continue
place = (str(source.resolve()), reference.docstring)
pages = sorted(
{
page
for page, paths in shown.get(place, ())
if paths is None or reference.path in paths
}
)
if pages:
problems.append(
f"{source.relative_to(root).as_posix()}:{reference.line}:"
f" [{reference.path}][] has no anchor on the site, where the"
f" docstring renders on {', '.join(pages)}; write the name as"
f" plain code instead: `{reference.path}`"
)
return problems
|
docrefs_run
Refuse every cross-reference the site cannot link: the lint phase's step.
The lines of reference_problems over the workspace, under one
heading.
Source code in packages/extensions/mkdocstrings/src/livery/extensions/mkdocstrings/_refs.py
| def docrefs_run(ctx: PhaseContext) -> None:
"""Refuse every cross-reference the site cannot link: the lint phase's step.
The lines of `reference_problems` over the workspace, under one
heading.
"""
from livery.footman import fail
problems = reference_problems(ctx.root)
if problems:
fail("cross-references the site cannot link:\n " + "\n ".join(problems))
|
main
The probe: the site the request on stdin describes, as JSON on stdout.
It answers with the docstrings of the requested files the site
renders, each with its page, and with the requested paths that
name no anchor and lie in no member that declines its reference.
Source code in packages/extensions/mkdocstrings/src/livery/extensions/mkdocstrings/_refs.py
| def main() -> None:
"""The probe: the site the request on stdin describes, as JSON on stdout.
It answers with the docstrings of the requested files the site
renders, each with its page, and with the requested paths that
name no anchor and lie in no member that declines its reference.
"""
import griffe
request = json.load(sys.stdin)
sources = [Path(src) for src in request["sources"]]
loader = griffe.GriffeLoader(
search_paths=[str(src) for src in sources], allow_inspection=False
)
for top in sorted(namespaces(sources)):
with contextlib.suppress(ImportError, KeyError):
# A dotted name, never a path: the working directory has no say.
loader.load(top, try_relative_path=False)
loader.resolve_aliases(implicit=False, external=None)
walk = _Walk(griffe, request["options"])
collection = loader.modules_collection
for page, identifier in request["pages"]:
try:
found = collection.get_member(identifier)
except walk.missing:
continue
walk.render(found, page)
files = set(request["files"])
rendered: list[tuple[str, int, str, list[str] | None]] = [
(file, line, page, None)
for (file, line), pages in sorted(walk.rendered.items())
if file in files
for page in sorted(pages)
]
providers = [
(str(identity), str(member)) for identity, member in request["providers"]
]
rendered += [entry for entry in _task_docstrings(providers) if entry[0] in files]
declined = [Path(src) for src in request["declined"]]
missing = [
path
for path in request["paths"]
if path not in walk.anchors
and not _declined(collection, path, declined, walk.missing)
]
json.dump({"rendered": rendered, "missing": missing}, sys.stdout)
|