Skip to content

livery.workshop._envfile

livery.workshop._envfile

The one dotenv implementation: parsing, substitution, the cascade.

Every .repo.env in the wild is written against these rules:

  • Precedence (highest first, kind-major): the pre-existing process environment, .repo.env.local files (nearest to farthest), .repo.shared.env in the runner's config directory, committed .repo.env files (nearest to farthest).
  • Values: ~ expands at the start of a raw value; double quotes strip and allow substitution; single quotes strip and are literal; an unquoted value loses inline # comments and trailing whitespace.
  • Substitution: $VAR and ${VAR} resolve from the stack first, then the pre-existing environment, else empty; \$ stays a literal dollar; iterated until stable.
  • Provenance is tracked per key (which extension supplied the value); the display and the agent emission select on it.

Classes:

Name Description
Source

Where a stack value came from (lower is higher precedence).

EnvStack

The resolved env-file stack: values plus per-key provenance.

Functions:

Name Description
is_secret_name

Whether key names a secret by the suffix convention.

parse_env_file

Parse an env file into key -> raw value pairs.

load_extension

Fold one file into stack; pre-existing environment wins.

resolve_all

Substitute every value until stable; honour literals and escapes.

cascade_dirs

Directories from the repo ceiling down to cwd, farthest first.

load_cascade

Load the multi-level env-file cascade, the one config mechanism.

quote_value

value spelled so the parser reads it back unchanged.

set_value

Set key=value in path, preserving the file's other content.

shared_file

The shared rung's file: the CI rung when the runner set one.

member_keys

Every key the cascade's files define (membership, no values).

Attributes:

Name Type Description
SECRET_SUFFIXES
RUNG_PREFIX
KEY_RE
MAX_SUBSTITUTION_PASSES

How many times one value may be re-substituted before it is given up.

SECRET_SUFFIXES module-attribute

SECRET_SUFFIXES = ('_TOKEN', '_SECRET', '_PASSWORD', '_KEY', '_API_KEY')

RUNG_PREFIX module-attribute

RUNG_PREFIX = 'RUNG_'

KEY_RE module-attribute

KEY_RE = re.compile('^\\s*([A-Za-z_][A-Za-z0-9_]*)\\s*=\\s*(.*)')

MAX_SUBSTITUTION_PASSES module-attribute

MAX_SUBSTITUTION_PASSES = 16

How many times one value may be re-substituted before it is given up.

Each pass resolves one level of reference, so a legitimate chain settles in a handful. Two keys referring to each other (A=$B beside B=$A) never settle: the value alternates, so it never equals its own one-step predecessor and an unbounded loop runs forever. That loop is inside the cascade every fm command loads, so the bound is what keeps a mistyped pair from hanging the whole toolchain.

Source

Bases: IntEnum

Where a stack value came from (lower is higher precedence).

Attributes:

Name Type Description
environment

The pre-existing process environment: always wins.

local

.repo.env.local (per machine, gitignored).

shared

.repo.shared.env in the runner's config directory.

repo

.repo.env (committed repo config).

environment class-attribute instance-attribute

environment = 0

The pre-existing process environment: always wins.

local class-attribute instance-attribute

local = 1

.repo.env.local (per machine, gitignored).

shared class-attribute instance-attribute

shared = 2

.repo.shared.env in the runner's config directory.

repo class-attribute instance-attribute

repo = 3

.repo.env (committed repo config).

EnvStack dataclass

EnvStack(values: dict[str, str] = dict(), sources: dict[str, Source] = dict())

The resolved env-file stack: values plus per-key provenance.

Methods:

Name Description
managed

The keys the files supplied (everything not pre-existing).

Attributes:

Name Type Description
values dict[str, str]
sources dict[str, Source]

values class-attribute instance-attribute

values: dict[str, str] = field(default_factory=dict)

sources class-attribute instance-attribute

sources: dict[str, Source] = field(default_factory=dict)

managed

managed() -> dict[str, str]

The keys the files supplied (everything not pre-existing).

Source code in packages/workshop/src/livery/workshop/_envfile.py
def managed(self) -> dict[str, str]:
    """The keys the files supplied (everything not pre-existing)."""
    return {
        key: value
        for key, value in self.values.items()
        if self.sources[key] is not Source.environment
    }

is_secret_name

is_secret_name(key: str) -> bool

Whether key names a secret by the suffix convention.

Source code in packages/workshop/src/livery/workshop/_envfile.py
def is_secret_name(key: str) -> bool:
    """Whether *key* names a secret by the suffix convention."""
    return any(key.endswith(suffix) for suffix in SECRET_SUFFIXES)

parse_env_file

parse_env_file(path: Path) -> dict[str, str]

Parse an env file into key -> raw value pairs.

Comments and blank lines are skipped, surrounding quotes stripped, ${VAR} left raw. The membership-only readers use this; full resolution goes through livery.workshop._envfile.load_cascade. A missing file answers an empty dict.

Source code in packages/workshop/src/livery/workshop/_envfile.py
def parse_env_file(path: Path) -> dict[str, str]:
    """Parse an env file into ``key -> raw value`` pairs.

    Comments and blank lines are skipped, surrounding quotes
    stripped, ``${VAR}`` left raw. The membership-only readers use
    this; full resolution goes through
    ``livery.workshop._envfile.load_cascade``. A missing file
    answers an empty dict.
    """
    result: dict[str, str] = {}
    if not path.is_file():
        return result
    for raw in path.read_text(encoding="utf-8").splitlines():
        line = raw.strip()
        if not line or line.startswith("#"):
            continue
        match = KEY_RE.match(line)
        if match:
            key = match.group(1)
            value = match.group(2).strip()
            if (value.startswith('"') and value.endswith('"')) or (
                value.startswith("'") and value.endswith("'")
            ):
                value = value[1:-1]
            result[key] = value
    return result

load_extension

load_extension(path: Path, source: Source, stack: EnvStack, environ: dict[str, str]) -> None

Fold one file into stack; pre-existing environment wins.

Source code in packages/workshop/src/livery/workshop/_envfile.py
def load_extension(
    path: Path,
    source: Source,
    stack: EnvStack,
    environ: dict[str, str],
) -> None:
    """Fold one file into *stack*; pre-existing environment wins."""
    if not path.is_file():
        return
    for line in path.read_text(encoding="utf-8").splitlines():
        stripped = line.strip()
        if not stripped or stripped.startswith("#"):
            continue
        match = KEY_RE.match(stripped)
        if match is None:
            continue
        key, raw = match.group(1), match.group(2)
        env_value = environ.get(key, "")
        if env_value:
            stack.values[key] = _expand_tilde(env_value)
            stack.sources[key] = Source.environment
            continue
        # Later loads override earlier ones; the caller feeds files
        # from lowest precedence (repo) to highest (local).
        stack.values[key] = _parse_raw(_expand_tilde(raw))
        stack.sources[key] = source

resolve_all

resolve_all(stack: EnvStack, environ: dict[str, str]) -> None

Substitute every value until stable; honour literals and escapes.

A value that has not settled within livery.workshop._envfile.MAX_SUBSTITUTION_PASSES keeps its raw text, unexpanded: showing $B is the answer a reader can act on, and half an expansion would look like a resolved value.

Source code in packages/workshop/src/livery/workshop/_envfile.py
def resolve_all(stack: EnvStack, environ: dict[str, str]) -> None:
    """Substitute every value until stable; honour literals and escapes.

    A value that has not settled within
    ``livery.workshop._envfile.MAX_SUBSTITUTION_PASSES`` keeps
    its raw text, unexpanded: showing ``$B`` is the answer a reader
    can act on, and half an expansion would look like a resolved
    value.
    """
    for key, value in list(stack.values.items()):
        if value.startswith(_LITERAL):
            stack.values[key] = value[len(_LITERAL) :]
            continue
        raw = value
        previous = None
        for _ in range(MAX_SUBSTITUTION_PASSES):
            if value == previous:
                break
            previous = value
            value = _substitute(value, stack.values, environ)
        else:
            value = raw.replace("\\$", "$")
        stack.values[key] = value.replace(_ESCAPED_DOLLAR, "$")

cascade_dirs

cascade_dirs(repo_root: Path, cwd: Path) -> list[Path]

Directories from the repo ceiling down to cwd, farthest first.

Farthest first, so folding with later-wins gives the nearest file precedence within its kind.

Source code in packages/workshop/src/livery/workshop/_envfile.py
def cascade_dirs(repo_root: Path, cwd: Path) -> list[Path]:
    """Directories from the repo ceiling down to *cwd*, farthest first.

    Farthest first, so folding with later-wins gives the nearest
    file precedence within its kind.
    """
    root = repo_root.resolve()
    cur = cwd.resolve()
    if root != cur and root not in cur.parents:
        return [root]
    dirs: list[Path] = []
    while True:
        dirs.append(cur)
        if cur == root:
            break
        cur = cur.parent
    dirs.reverse()
    return dirs

load_cascade

load_cascade(repo_root: Path, cwd: Path, *, shared_dir: Path | None = None, environ: dict[str, str] | None = None) -> EnvStack

Load the multi-level env-file cascade, the one config mechanism.

.repo.env (committed) and .repo.env.local (machine, secrets, gitignored) at every level from the repo ceiling down to cwd. Precedence is kind-major: real environment, then .local files nearest to farthest, then the shared file, then committed files nearest to farthest, so a machine override beats committed config at any depth.

shared_dir is where .repo.shared.env lives; None means the runner's own config directory, asked of footman. Under CI the entry's emission points WORKSHOP_SHARED_ENV_FILE at a runner-local file it wrote from the job's secrets, and that file IS the shared slot: same precedence, same masking, gone with the runner. environ defaults to os.environ and is injectable for tests.

Source code in packages/workshop/src/livery/workshop/_envfile.py
def load_cascade(
    repo_root: Path,
    cwd: Path,
    *,
    shared_dir: Path | None = None,
    environ: dict[str, str] | None = None,
) -> EnvStack:
    """Load the multi-level env-file cascade, the one config mechanism.

    ``.repo.env`` (committed) and ``.repo.env.local`` (machine,
    secrets, gitignored) at every level from the repo ceiling down
    to *cwd*. Precedence is kind-major: real environment, then
    ``.local`` files nearest to farthest, then the shared file, then
    committed files nearest to farthest, so a machine override beats
    committed config at any depth.

    *shared_dir* is where ``.repo.shared.env`` lives; None means the
    runner's own config directory, asked of footman. Under CI the
    entry's emission points ``WORKSHOP_SHARED_ENV_FILE`` at a
    runner-local file it wrote from the job's secrets, and that file
    IS the shared slot: same precedence, same masking, gone with the
    runner. *environ*
    defaults to ``os.environ`` and is injectable for tests.
    """
    env = dict(os.environ) if environ is None else environ
    dirs = cascade_dirs(repo_root, cwd)
    committed = [d / ".repo.env" for d in dirs]
    local = [d / ".repo.env.local" for d in dirs]

    stack = EnvStack()
    for path in committed:
        load_extension(path, Source.repo, stack, env)
    load_extension(shared_file(shared_dir, env), Source.shared, stack, env)
    for path in local:
        load_extension(path, Source.local, stack, env)
    resolve_all(stack, env)
    return stack

quote_value

quote_value(value: str) -> str

value spelled so the parser reads it back unchanged.

An unquoted value loses everything from its first # onward and all trailing whitespace, so a value carrying either is written inside double quotes, which the parser strips again. A line break cannot be spelled on one line at all, the value would read back as several lines with the second silently discarded, so it is refused rather than written.

Raises:

Type Description
ValueError

value contains a line break.

Source code in packages/workshop/src/livery/workshop/_envfile.py
def quote_value(value: str) -> str:
    """*value* spelled so the parser reads it back unchanged.

    An unquoted value loses everything from its first ``#`` onward
    and all trailing whitespace, so a value carrying either is
    written inside double quotes, which the parser strips again. A
    line break cannot be spelled on one line at all, the value would
    read back as several lines with the second silently discarded,
    so it is refused rather than written.

    Raises:
        ValueError: *value* contains a line break.
    """
    if "\n" in value or "\r" in value:
        raise ValueError("a line break cannot be stored in an env file")
    if "#" in value or value != value.strip():
        return f'"{value}"'
    return value

set_value

set_value(path: Path, key: str, value: str) -> None

Set key=value in path, preserving the file's other content.

The stack's one write primitive: creates the file and parents when missing, replaces the key's line in place when present, appends with a blank separator otherwise. The value is spelled by livery.workshop._envfile.quote_value, so what is read back is what was set.

Source code in packages/workshop/src/livery/workshop/_envfile.py
def set_value(path: Path, key: str, value: str) -> None:
    """Set ``key=value`` in *path*, preserving the file's other content.

    The stack's one write primitive: creates the file and parents
    when missing, replaces the key's line in place when present,
    appends with a blank separator otherwise. The value is spelled
    by ``livery.workshop._envfile.quote_value``, so what is read
    back is what was set.
    """
    lines = path.read_text(encoding="utf-8").splitlines() if path.is_file() else []
    new_line = f"{key}={quote_value(value)}"
    for index, line in enumerate(lines):
        match = KEY_RE.match(line)
        if match and match.group(1) == key:
            lines[index] = new_line
            break
    else:
        if lines and lines[-1].strip():
            lines.append("")
        lines.append(new_line)
    path.parent.mkdir(parents=True, exist_ok=True)
    path.write_text("\n".join(lines) + "\n", encoding="utf-8")

shared_file

shared_file(shared_dir: Path | None, environ: dict[str, str]) -> Path

The shared rung's file: the CI rung when the runner set one.

WORKSHOP_SHARED_ENV_FILE names a runner-local file the entry's emission materialised from the repository's secrets; when it is unset the rung is .repo.shared.env in shared_dir (the runner's config directory when None).

Source code in packages/workshop/src/livery/workshop/_envfile.py
def shared_file(shared_dir: Path | None, environ: dict[str, str]) -> Path:
    """The shared rung's file: the CI rung when the runner set one.

    ``WORKSHOP_SHARED_ENV_FILE`` names a runner-local file the
    entry's emission materialised from the repository's secrets; when
    it is unset the rung is ``.repo.shared.env`` in *shared_dir*
    (the runner's config directory when None).
    """
    override = environ.get("WORKSHOP_SHARED_ENV_FILE", "")
    if override:
        return Path(override)
    if shared_dir is None:
        import livery.footman as footman

        shared_dir = footman.config_dir()
    return shared_dir / ".repo.shared.env"

member_keys

member_keys(repo_root: Path, cwd: Path, shared_dir: Path) -> set[str]

Every key the cascade's files define (membership, no values).

The one definition of which files make up the stack, for the membership consumers (the agent emission, env.show): the same files livery.workshop._envfile.load_cascade folds for a run from cwd.

Source code in packages/workshop/src/livery/workshop/_envfile.py
def member_keys(repo_root: Path, cwd: Path, shared_dir: Path) -> set[str]:
    """Every key the cascade's files define (membership, no values).

    The one definition of which files make up the stack, for the
    membership consumers (the agent emission, ``env.show``): the
    same files ``livery.workshop._envfile.load_cascade`` folds
    for a run from *cwd*.
    """
    keys: set[str] = set()
    for directory in cascade_dirs(repo_root, cwd):
        keys |= set(parse_env_file(directory / ".repo.env"))
        keys |= set(parse_env_file(directory / ".repo.env.local"))
    keys |= set(parse_env_file(shared_file(shared_dir, dict(os.environ))))
    return keys