Skip to content

livery.extensions.cmake._readings

livery.extensions.cmake._readings

What a native measurer read of each source file, and the part it becomes.

A measured test run of the gate build is read by its compiler family's measurer, and each reader here turns that measurer's output into a Reading per source file: gcov's JSON (from_gcov_json), llvm-cov's JSON export (from_llvm_json), and the Cobertura report Microsoft's engine writes (from_cobertura). write_part writes a package's readings as its part in the coverage extension's line format, under PARTS, where coverage gathers it.

Each reader keeps the most its measurer gives. gcov gives each line's count, each line's branches by index with the path an exception takes tagged, and its functions under their demangled names. llvm-cov gives each line's count as its own line view counts it, each condition's true and false branches, its functions, whose names demangle turns readable, each code region as a range, with a macro counted where it is called, and the code the preprocessor removed as excluded lines. A Cobertura report gives each line's count, how many of a line's branches ran, and its methods. The floors judge the lines alone; the rest reaches the pages.

Classes:

Name Description
Reading

What a native measurer read of one source file.

Functions:

Name Description
from_gcov_json

What gcov's JSON output names, per file as gcov names it; a non-document refuses.

from_llvm_json

What llvm-cov export -format=text prints, per file as llvm-cov names it.

line_counts

Each line's count from a file's segments, as llvm-cov's line view counts it.

demangle

Rename every function of readings to its demangled name, in place.

from_cobertura

What a Cobertura report names, per file; a report that is not XML refuses.

relativise

readings keyed by workspace-relative posix paths; files outside root dropped.

within

The files of readings under package, keyed as they are.

write_part

Write readings as package_path's part under root; the part's path.

remove_parts

Delete package_path's parts of an earlier run under root, any measurer's.

Attributes:

Name Type Description
PARTS
FORMAT
THROW
MEASURERS dict[str, dict[str, Any]]
CODE_REGION
EXPANSION_REGION
SKIPPED_REGION
CONDITIONS

PARTS module-attribute

PARTS = '.workshop/coverage/parts'

FORMAT module-attribute

FORMAT = 3

THROW module-attribute

THROW = 'throw'

MEASURERS module-attribute

MEASURERS: dict[str, dict[str, Any]] = {'gcov': {'name': 'gcov', 'branches': 'indexes', 'columns': False}, 'llvm': {'name': 'llvm-cov', 'branches': 'indexes', 'columns': True}, 'msvc': {'name': 'dotnet-coverage', 'branches': 'none', 'columns': False}}

CODE_REGION module-attribute

CODE_REGION = 0

EXPANSION_REGION module-attribute

EXPANSION_REGION = 1

SKIPPED_REGION module-attribute

SKIPPED_REGION = 2

CONDITIONS module-attribute

CONDITIONS = re.compile('\\((\\d+)/(\\d+)\\)')

Reading dataclass

Reading(lines: dict[int, int] = dict(), excluded: set[int] = set(), branches: dict[tuple[int, int], tuple[int, str]] = dict(), branch_counts: dict[int, tuple[int, int]] = dict(), functions: dict[tuple[str, int], tuple[int, int | None]] = dict(), ranges: dict[tuple[int, int, int, int], int] = dict())

What a native measurer read of one source file.

Attributes:

Name Type Description
lines dict[int, int]

How many times each line ran, every line the measurer instruments, run or not.

excluded set[int]

The lines the preprocessor removed from the build.

branches dict[tuple[int, int], tuple[int, str]]

Each branch's runs and tag by its line and its index among the line's branches; the tag is throw for the path an exception takes, else empty.

branch_counts dict[int, tuple[int, int]]

For a measurer that counts a line's branches without naming them: the line's branches, and how many ran.

functions dict[tuple[str, int], tuple[int, int | None]]

Each function's last line and calls, by its name and first line; the calls are None for a measurer that counts none.

ranges dict[tuple[int, int, int, int], int]

How many times each stretch of code ran, by its start line and column and its end line and the column after its last; columns count bytes from 1.

Methods:

Name Description
add

Add other, another counter file's reading of the same source, to this one.

add_branch

Count taken runs of branch index of line, tagged tag.

add_function

Count calls of the function name from line start to line end.

lines class-attribute instance-attribute

lines: dict[int, int] = field(default_factory=dict)

excluded class-attribute instance-attribute

excluded: set[int] = field(default_factory=set)

branches class-attribute instance-attribute

branches: dict[tuple[int, int], tuple[int, str]] = field(default_factory=dict)

branch_counts class-attribute instance-attribute

branch_counts: dict[int, tuple[int, int]] = field(default_factory=dict)

functions class-attribute instance-attribute

functions: dict[tuple[str, int], tuple[int, int | None]] = field(default_factory=dict)

ranges class-attribute instance-attribute

ranges: dict[tuple[int, int, int, int], int] = field(default_factory=dict)

add

add(other: Reading) -> None

Add other, another counter file's reading of the same source, to this one.

Counts add up; a count of a line's branches keeps the most of each.

Source code in packages/extensions/cmake/src/livery/extensions/cmake/_readings.py
def add(self, other: Reading) -> None:
    """Add *other*, another counter file's reading of the same source, to this one.

    Counts add up; a count of a line's branches keeps the most of
    each.
    """
    for line, runs in other.lines.items():
        self.lines[line] = self.lines.get(line, 0) + runs
    self.excluded |= other.excluded
    for (line, index), (taken, tag) in other.branches.items():
        self.add_branch(line, index, taken, tag)
    for line, (total, ran) in other.branch_counts.items():
        was_total, was_ran = self.branch_counts.get(line, (0, 0))
        self.branch_counts[line] = (max(was_total, total), max(was_ran, ran))
    for (name, start), (end, calls) in other.functions.items():
        self.add_function(name, start, end, calls)
    for extent, count in other.ranges.items():
        self.ranges[extent] = self.ranges.get(extent, 0) + count

add_branch

add_branch(line: int, index: int, taken: int, tag: str) -> None

Count taken runs of branch index of line, tagged tag.

Source code in packages/extensions/cmake/src/livery/extensions/cmake/_readings.py
def add_branch(self, line: int, index: int, taken: int, tag: str) -> None:
    """Count *taken* runs of branch *index* of *line*, tagged *tag*."""
    was, was_tag = self.branches.get((line, index), (0, ""))
    self.branches[(line, index)] = (was + taken, was_tag or tag)

add_function

add_function(name: str, start: int, end: int, calls: int | None) -> None

Count calls of the function name from line start to line end.

Calls a measurer counts add up; a function no reading counted the calls of keeps None.

Source code in packages/extensions/cmake/src/livery/extensions/cmake/_readings.py
def add_function(self, name: str, start: int, end: int, calls: int | None) -> None:
    """Count *calls* of the function *name* from line *start* to line *end*.

    Calls a measurer counts add up; a function no reading counted
    the calls of keeps None.
    """
    was_end, was_calls = self.functions.get((name, start), (end, None))
    total = (
        None
        if was_calls is None and calls is None
        else (was_calls or 0) + (calls or 0)
    )
    self.functions[(name, start)] = (max(was_end, end), total)

from_gcov_json

from_gcov_json(text: str) -> dict[str, Reading]

What gcov's JSON output names, per file as gcov names it; a non-document refuses.

gcov prints one document per counter file, one per line, or one whole. A line gcov names twice, and a file two documents name, has its counts added. Each line's branches are numbered in gcov's order, the ones gcov marks as an exception's tagged throw, when gcov ran with --branch-probabilities; a function is named as gcov demangles it.

Source code in packages/extensions/cmake/src/livery/extensions/cmake/_readings.py
def from_gcov_json(text: str) -> dict[str, Reading]:
    """What gcov's JSON output names, per file as gcov names it; a non-document refuses.

    gcov prints one document per counter file, one per line, or one
    whole. A line gcov names twice, and a file two documents name, has
    its counts added. Each line's branches are numbered in gcov's
    order, the ones gcov marks as an exception's tagged ``throw``,
    when gcov ran with ``--branch-probabilities``; a function is named
    as gcov demangles it.
    """
    readings: dict[str, Reading] = {}
    for document in _json_documents(text):
        files = document.get("files") if isinstance(document, dict) else None
        if not isinstance(files, list):
            fail("gcov printed no files table; its JSON format has one")
        for entry in files:
            if not isinstance(entry, dict):
                continue
            name = str(entry.get("file", ""))
            rows = entry.get("lines")
            if not name or not isinstance(rows, list):
                continue
            readings.setdefault(name, Reading()).add(_gcov_file(entry, rows))
    return readings

from_llvm_json

from_llvm_json(text: str) -> dict[str, Reading]

What llvm-cov export -format=text prints, per file as llvm-cov names it.

A line's count is the one llvm-cov's own line view gives, the view its lcov export prints, read from the file's segments (line_counts). Each branch region is a condition: its true branch numbered twice its place among its line's conditions, its false branch one more. A function's code regions in its own file give its first and last line, and its calls are its count; its name is llvm-cov's, a mangled one until demangle reads it. A function's code regions become ranges, and a macro's call counts as a range where it is called, never inside the macro; regions of one stretch, as each instance of a template gives, add their counts. Its skipped regions, the code the preprocessor removed, mark their lines excluded.

Raises:

Type Description
Failed

when text is not llvm-cov's JSON export, naming why.

Source code in packages/extensions/cmake/src/livery/extensions/cmake/_readings.py
def from_llvm_json(text: str) -> dict[str, Reading]:
    """What ``llvm-cov export -format=text`` prints, per file as llvm-cov names it.

    A line's count is the one llvm-cov's own line view gives, the view
    its lcov export prints, read from the file's segments
    (`line_counts`). Each branch region is a condition: its true
    branch numbered twice its place among its line's conditions, its
    false branch one more. A function's code regions in its own file
    give its first and last line, and its calls are its count; its
    name is llvm-cov's, a mangled one until `demangle` reads it. A
    function's code regions become ranges, and a macro's call counts
    as a range where it is called, never inside the macro; regions of
    one stretch, as each instance of a template gives, add their
    counts. Its skipped regions, the code the preprocessor removed,
    mark their lines excluded.

    Raises:
        Failed: when *text* is not llvm-cov's JSON export, naming why.
    """
    try:
        document = json.loads(text)
    except ValueError as error:
        fail(f"llvm-cov printed no JSON export ({error})")
    exports = document.get("data") if isinstance(document, dict) else None
    if not isinstance(exports, list):
        fail("llvm-cov printed no data table; its JSON export has one")
    readings: dict[str, Reading] = {}
    for export in exports:
        if not isinstance(export, dict):
            continue
        for entry in export.get("files") or []:
            if not isinstance(entry, dict) or not entry.get("filename"):
                continue
            found = readings.setdefault(str(entry["filename"]), Reading())
            for line, runs in line_counts(entry.get("segments") or []).items():
                found.lines[line] = found.lines.get(line, 0) + runs
            for line, index, taken in _conditions(entry.get("branches") or []):
                found.add_branch(line, index, taken, "")
        kinds: dict[tuple[str, tuple[int, int, int, int]], int] = {}
        for function in export.get("functions") or []:
            if isinstance(function, dict):
                _llvm_function(function, readings, kinds)
    return readings

line_counts

line_counts(segments: Iterable[Any]) -> dict[int, int]

Each line's count from a file's segments, as llvm-cov's line view counts it.

A segment is [line, column, count, has count, starts a region, is a gap]. A line is counted when a region with a count starts on it, or the last segment before it carries a count, unless the line starts a region the preprocessor skipped; its count is the most of the carried count and the counts of the regions starting on it, gaps aside. The lines run from the first to the last line a segment names.

Source code in packages/extensions/cmake/src/livery/extensions/cmake/_readings.py
def line_counts(segments: Iterable[Any]) -> dict[int, int]:
    """Each line's count from a file's segments, as llvm-cov's line view counts it.

    A segment is ``[line, column, count, has count, starts a region,
    is a gap]``. A line is counted when a region with a count starts
    on it, or the last segment before it carries a count, unless the
    line starts a region the preprocessor skipped; its count is the
    most of the carried count and the counts of the regions starting
    on it, gaps aside. The lines run from the first to the last line a
    segment names.
    """
    ordered = [item for item in segments if isinstance(item, list) and len(item) >= 6]
    counts: dict[int, int] = {}
    wrapped: list[Any] | None = None
    current: list[list[Any]] = []
    line = 1
    index = 0
    while index < len(ordered):
        if current:
            wrapped = current[-1]
        current = []
        while index < len(ordered) and ordered[index][0] == line:
            current.append(ordered[index])
            index += 1
        mapped, runs = _line_stats(current, wrapped)
        if mapped:
            counts[line] = runs
        line += 1
    return counts

demangle

demangle(readings: Mapping[str, Reading], demangled: Callable[[list[str]], list[str]]) -> None

Rename every function of readings to its demangled name, in place.

demangled answers each of the names it is given with the name to show, in order, as llvm-cxxfilt does on its standard input; an answer of another length leaves every name as it is.

Source code in packages/extensions/cmake/src/livery/extensions/cmake/_readings.py
def demangle(
    readings: Mapping[str, Reading], demangled: Callable[[list[str]], list[str]]
) -> None:
    """Rename every function of *readings* to its demangled name, in place.

    *demangled* answers each of the names it is given with the name
    to show, in order, as ``llvm-cxxfilt`` does on its standard input;
    an answer of another length leaves every name as it is.
    """
    names = sorted(
        {name for reading in readings.values() for name, _ in reading.functions}
    )
    if not names:
        return
    answered = demangled(names)
    if len(answered) != len(names):
        return
    shown = dict(zip(names, answered, strict=True))
    for reading in readings.values():
        functions = dict(reading.functions)
        reading.functions.clear()
        for (name, start), (end, calls) in functions.items():
            reading.add_function(shown[name] or name, start, end, calls)

from_cobertura

from_cobertura(text: str) -> dict[str, Reading]

What a Cobertura report names, per file; a report that is not XML refuses.

A class's filename is taken as it is when absolute, else under the report's first <source>. A line a class names twice, under one of its methods and again under the class, counts once, at its most; two classes of one file add their counts. A line's condition-coverage of (k/n) is how many of its n branches ran, and each method is a function over the lines it names.

Source code in packages/extensions/cmake/src/livery/extensions/cmake/_readings.py
def from_cobertura(text: str) -> dict[str, Reading]:
    """What a Cobertura report names, per file; a report that is not XML refuses.

    A class's ``filename`` is taken as it is when absolute, else under
    the report's first ``<source>``. A line a class names twice, under
    one of its methods and again under the class, counts once, at its
    most; two classes of one file add their counts. A line's
    ``condition-coverage`` of ``(k/n)`` is how many of its n branches
    ran, and each method is a function over the lines it names.
    """
    try:
        document = ElementTree.fromstring(text)
    except ElementTree.ParseError as error:
        fail(f"the coverage report is not XML ({error})")
    sources = [
        (node.text or "").strip() for node in document.iter("source") if node.text
    ]
    readings: dict[str, Reading] = {}
    for klass in document.iter("class"):
        name = klass.get("filename", "")
        if not name:
            continue
        if not _absolute(name) and sources:
            name = (
                sources[0].rstrip("\\/") + ("\\" if "\\" in sources[0] else "/") + name
            )
        readings.setdefault(name, Reading()).add(_cobertura_class(klass))
    return readings

relativise

relativise(readings: Mapping[str, Reading], root: Path) -> dict[str, Reading]

readings keyed by workspace-relative posix paths; files outside root dropped.

A measurer names files the way the compiler saw them, absolute and with the host's separators; the row and the floors want the path the repository knows. Two names of one file add up.

Source code in packages/extensions/cmake/src/livery/extensions/cmake/_readings.py
def relativise(readings: Mapping[str, Reading], root: Path) -> dict[str, Reading]:
    """*readings* keyed by workspace-relative posix paths; files outside *root* dropped.

    A measurer names files the way the compiler saw them, absolute and
    with the host's separators; the row and the floors want the path
    the repository knows. Two names of one file add up.
    """
    base = root.resolve()
    kept: dict[str, Reading] = {}
    for name, reading in readings.items():
        candidate = Path(PureWindowsPath(name).as_posix() if "\\" in name else name)
        if not candidate.is_absolute():
            candidate = base / candidate
        try:
            relative = candidate.resolve().relative_to(base).as_posix()
        except ValueError:
            continue
        kept.setdefault(relative, Reading()).add(reading)
    return kept

within

within(readings: Mapping[str, Reading], package: Package) -> dict[str, Reading]

The files of readings under package, keyed as they are.

Source code in packages/extensions/cmake/src/livery/extensions/cmake/_readings.py
def within(readings: Mapping[str, Reading], package: Package) -> dict[str, Reading]:
    """The files of *readings* under *package*, keyed as they are."""
    prefix = f"{package.path}/"
    return {name: found for name, found in readings.items() if name.startswith(prefix)}

write_part

write_part(root: Path, package_path: str, readings: Mapping[str, Reading], family: str) -> Path

Write readings as package_path's part under root; the part's path.

The part names its measurer by the compiler family that read it, counts how many times each line ran, and judges lines alone. Per file: the instrumented lines as its statements and the ones that ran with their counts, the lines the preprocessor removed that are no statement as excluded, the branches by index, the counts of a line's branches, the functions, the ranges for a measurer that reads columns, and the git blob id of the file's text with each CRLF read as LF, so a Windows checkout and a Linux one give one digest. A line number below 1 is no line and is left out.

Source code in packages/extensions/cmake/src/livery/extensions/cmake/_readings.py
def write_part(
    root: Path, package_path: str, readings: Mapping[str, Reading], family: str
) -> Path:
    """Write *readings* as *package_path*'s part under *root*; the part's path.

    The part names its measurer by the compiler *family* that read it,
    counts how many times each line ran, and judges lines alone. Per
    file: the instrumented lines as its statements and the ones that
    ran with their counts, the lines the preprocessor removed that are
    no statement as excluded, the branches by index, the counts of a
    line's branches, the functions, the ranges for a measurer that
    reads columns, and the git blob id of the file's text with each
    CRLF read as LF, so a Windows checkout and a Linux one give one
    digest. A line number below 1 is no line and is left out.
    """
    from livery import git

    measurer = MEASURERS.get(family, {"name": family, "branches": "none"})
    files = {
        name: _entry(root / name, reading, columns=bool(measurer.get("columns")))
        for name, reading in sorted(readings.items())
    }
    part = {
        "format": FORMAT,
        "unit": package_path,
        "measurer": {
            "name": measurer["name"],
            "version": "",
            "runtime": "",
            "counts": "counts",
            "branches": measurer["branches"],
            "regions": ["function"],
            "judges": ["lines"],
            "columns": bool(measurer.get("columns")),
        },
        "contexts": [],
        "files": files,
    }
    path = root / PARTS / str(measurer["name"]) / f"{git.slug(package_path)}.json"
    path.parent.mkdir(parents=True, exist_ok=True)
    path.write_text(json.dumps(part, sort_keys=True) + "\n", encoding="utf-8")
    return path

remove_parts

remove_parts(root: Path, package_path: str) -> None

Delete package_path's parts of an earlier run under root, any measurer's.

Source code in packages/extensions/cmake/src/livery/extensions/cmake/_readings.py
def remove_parts(root: Path, package_path: str) -> None:
    """Delete *package_path*'s parts of an earlier run under *root*, any measurer's."""
    from livery import git

    for stale in (root / PARTS).glob(f"*/{git.slug(package_path)}.json"):
        stale.unlink()