Skip to content

livery.extensions.python._parse

livery.extensions.python._parse

Parse each python source once, and say what it references.

The layering phase reads every python source through this module: the python extension's references answer and its layering steps, private-reaches, runner-terminal and declared-imports. parsed_source parses a file once for a given content, memoised by the file's path, size and modification time, so the steps, which run in one process, read one tree, and an unchanged file is never parsed twice. parsed_modules hands a rule the sources in its scope, every one or the files a narrowed run selects, RuleContext is what a rule judges, and graph_changed is what each step's inputs widen on.

Classes:

Name Description
Parsed

One parsed source.

RuleContext

What a layering rule over the python sources judges: the workspace and a scope.

ParsedModule

One python source as a rule sees it.

Functions:

Name Description
parsed_source

path parsed once for its content; None when it does not parse.

references_of

Every dotted name tree imports, in source order, once for each way.

parsed_modules

Every python source the workspace holds, or the named ones, parsed once.

dotted_name

The dotted module path of source, a file under the src tree src.

graph_changed

Whether a contract, a native manifest or a root src/ changed.

Parsed dataclass

Parsed(tree: Module, references: tuple[ReferencedName, ...])

One parsed source.

Attributes:

Name Type Description
tree Module

The module's syntax tree.

references tuple[ReferencedName, ...]

Every dotted name it imports, once for each place it imports it from: at import time or lazily, and under which guard.

tree instance-attribute

tree: Module

references instance-attribute

references: tuple[ReferencedName, ...]

RuleContext dataclass

RuleContext(root: Path, packages: tuple[Package, ...], files: frozenset[str] | None = None)

What a layering rule over the python sources judges: the workspace and a scope.

Attributes:

Name Type Description
root Path

The workspace root.

packages tuple[Package, ...]

Every package the workspace carries.

files frozenset[str] | None

The root-relative sources the run judges; None for every one.

Methods:

Name Description
packages_in_scope

The packages holding a source this run judges, every one without a scope.

root instance-attribute

root: Path

packages instance-attribute

packages: tuple[Package, ...]

files class-attribute instance-attribute

files: frozenset[str] | None = None

packages_in_scope

packages_in_scope() -> tuple[Package, ...]

The packages holding a source this run judges, every one without a scope.

Source code in packages/extensions/python/src/livery/extensions/python/_parse.py
def packages_in_scope(self) -> tuple[Package, ...]:
    """The packages holding a source this run judges, every one without a scope."""
    if self.files is None:
        return self.packages
    files = self.files
    return tuple(
        package
        for package in self.packages
        if any(path.startswith(f"{package.path}/") for path in files)
    )

ParsedModule dataclass

ParsedModule(package: Package | None, path: Path, relative: str, area: str, dotted: str, tree: Module)

One python source as a rule sees it.

Attributes:

Name Type Description
package Package | None

The package the file belongs to; None for the root's own tasks.py.

path Path

The file, absolute.

relative str

The file, relative to the workspace root, posix.

area str

src, tests, or root.

dotted str

The module's dotted path under src; empty elsewhere.

tree Module

The syntax tree.

package instance-attribute

package: Package | None

path instance-attribute

path: Path

relative instance-attribute

relative: str

area instance-attribute

area: str

dotted instance-attribute

dotted: str

tree instance-attribute

tree: Module

parsed_source

parsed_source(path: Path) -> Parsed | None

path parsed once for its content; None when it does not parse.

The memo key is the path with its size and modification time, so an edited file parses again and an unchanged one never does.

Source code in packages/extensions/python/src/livery/extensions/python/_parse.py
def parsed_source(path: Path) -> Parsed | None:
    """*path* parsed once for its content; None when it does not parse.

    The memo key is the path with its size and modification time, so
    an edited file parses again and an unchanged one never does.
    """
    try:
        stat = path.stat()
    except OSError:
        return None
    key = (path, stat.st_size, stat.st_mtime_ns)
    found = _PARSED.get(key)
    if found is None:
        found = _parse(path)
        if found is None:
            return None
        _PARSED[key] = found
    return found

references_of

references_of(tree: Module) -> tuple[ReferencedName, ...]

Every dotted name tree imports, in source order, once for each way.

A from import references the module and each name imported from it: under a PEP 420 namespace the module alone can be the namespace, and the package that owns the code is one segment further down. A name imported twice the same way is listed once. A relative import names the source's own package and is left out.

An import inside a function or a method body is lazy: importing the module does not run it. One in the body of if TYPE_CHECKING: is made by a type checker alone. One in the body of a try with a handler for ImportError, ModuleNotFoundError or every exception is optional: the source runs without the module.

Only a statement imports, so the walk reads the statements and never descends into an expression.

Source code in packages/extensions/python/src/livery/extensions/python/_parse.py
def references_of(tree: ast.Module) -> tuple[ReferencedName, ...]:
    """Every dotted name *tree* imports, in source order, once for each way.

    A ``from`` import references the module and each name imported
    from it: under a PEP 420 namespace the module alone can be the
    namespace, and the package that owns the code is one segment
    further down. A name imported twice the same way is listed once. A
    relative import names the source's own package and is left out.

    An import inside a function or a method body is lazy: importing the
    module does not run it. One in the body of ``if TYPE_CHECKING:`` is
    made by a type checker alone. One in the body of a ``try`` with a
    handler for ``ImportError``, ``ModuleNotFoundError`` or every
    exception is optional: the source runs without the module.

    Only a statement imports, so the walk reads the statements and
    never descends into an expression.
    """
    found: dict[ReferencedName, None] = {}
    pending = [(statement, _Where()) for statement in reversed(tree.body)]
    while pending:
        statement, where = pending.pop()
        if isinstance(statement, ast.Import):
            names = [alias.name for alias in statement.names]
        elif isinstance(statement, ast.ImportFrom):
            module = statement.module
            names = (
                [module, *(f"{module}.{alias.name}" for alias in statement.names)]
                if module and statement.level == 0
                else []
            )
        else:
            pending.extend(reversed(list(_inside(statement, where))))
            continue
        for name in names:
            found.setdefault(
                ReferencedName(
                    name,
                    lazy=where.lazy,
                    type_checking=where.type_checking,
                    optional=where.optional,
                )
            )
    return tuple(found)

parsed_modules

parsed_modules(root: Path, packages: tuple[Package, ...], files: frozenset[str] | None = None) -> tuple[ParsedModule, ...]

Every python source the workspace holds, or the named ones, parsed once.

The root's own tasks.py, then each package's src and tests trees in package order. files keeps the root-relative sources it names, and a named file that no longer exists is left out. A file that does not parse is left out: the syntax gate names it.

Source code in packages/extensions/python/src/livery/extensions/python/_parse.py
def parsed_modules(
    root: Path, packages: tuple[Package, ...], files: frozenset[str] | None = None
) -> tuple[ParsedModule, ...]:
    """Every python source the workspace holds, or the named ones, parsed once.

    The root's own ``tasks.py``, then each package's ``src`` and
    ``tests`` trees in package order. *files* keeps the root-relative
    sources it names, and a named file that no longer exists is left
    out. A file that does not parse is left out: the syntax gate names
    it.
    """

    def wanted(source: Path) -> bool:
        return files is None or source.relative_to(root).as_posix() in files

    found: list[ParsedModule] = []
    tasks = root / "tasks.py"
    if tasks.is_file() and wanted(tasks):
        _collect(found, None, tasks, root, "root", tasks.parent)
    for package in packages:
        for area in ("src", "tests"):
            base = package.directory / area
            if not base.is_dir():
                continue
            if files is not None:
                prefix = f"{package.path}/{area}/"
                named = sorted(
                    root / path
                    for path in files
                    if path.startswith(prefix)
                    and path.endswith(".py")
                    and (root / path).is_file()
                )
                for source in named:
                    _collect(found, package, source, root, area, base)
                continue
            for source in sorted(base.rglob("*.py")):
                _collect(found, package, source, root, area, base)
    return tuple(found)

dotted_name

dotted_name(source: Path, src: Path) -> str

The dotted module path of source, a file under the src tree src.

livery/forge/_dev/__init__.py under src is livery.forge._dev.

Source code in packages/extensions/python/src/livery/extensions/python/_parse.py
def dotted_name(source: Path, src: Path) -> str:
    """The dotted module path of *source*, a file under the ``src`` tree *src*.

    ``livery/forge/_dev/__init__.py`` under ``src`` is ``livery.forge._dev``.
    """
    dotted = ".".join(source.relative_to(src).parts)
    return dotted.removesuffix(".py").removesuffix(".__init__")

graph_changed

graph_changed(changes: Changes) -> bool

Whether a contract, a native manifest or a root src/ changed.

The layering steps' inputs widen on them: the distributions' roots and declarations come from the manifests, and the allowances from the root contract, so a change there judges every source, as the base's layering steps do. The workshop answers it (livery.workshop.graph_changed); a step's widen reference names the extension's own code, so this function hands it on.

Source code in packages/extensions/python/src/livery/extensions/python/_parse.py
def graph_changed(changes: Changes) -> bool:
    """Whether a contract, a native manifest or a root ``src/`` changed.

    The layering steps' inputs widen on them: the distributions' roots
    and declarations come from the manifests, and the allowances from
    the root contract, so a change there judges every source, as the
    base's layering steps do. The workshop answers it
    ([livery.workshop.graph_changed][]); a step's ``widen`` reference
    names the extension's own code, so this function hands it on.
    """
    from livery.workshop import graph_changed as workspace_graph_changed

    return workspace_graph_changed(changes)