livery.strongroom¶
livery.strongroom ¶
Content-addressed storage: one address space, every tenant.
The store names bytes by their digest, keeps trees and versions as blobs in specified formats, moves refs by compare-and-swap with a record beside each, and hands a real path to a program that needs one. It knows no tool, no call and no dataset: a consumer composes the formats and owns its namespaces.
This release carries the formats: livery.strongroom.canonical for
the one hashed encoding, livery.strongroom.Digest for names,
livery.strongroom.Tree and livery.strongroom.Version for the
two objects with structure, and livery.strongroom.RefRecord and
livery.strongroom.Tombstone for the two records beside names. The
standard they implement is the spec/ directory beside this package,
with the golden vectors the tests run.
livery.strongroom.Store is the local store over those formats:
objects landed by digest and verified, refs moved by compare-and-swap
under a per-ref lock with a record beside each, and a mutation class
per namespace. Its refusals are the classes under
livery.strongroom.StoreError. A store opened with sources
(livery.strongroom.FolderSource, livery.strongroom.HttpSource,
livery.strongroom.OriginHint) fetches what it lacks through them,
verified and in order, with livery.strongroom.Store.fetch, and
builds a mirror with livery.strongroom.Store.fill. The lifecycle
is the store's too: livery.strongroom.Store.publish_begin and
livery.strongroom.Store.publish_commit for the fail-closed
publish, livery.strongroom.Store.sweep for reachability, and
livery.strongroom.Store.erase for the tombstone. The materialiser
is livery.strongroom.Store.view, livery.strongroom.Store.collect
and livery.strongroom.Store.drop_view: the only route from a digest
to a path, by the cheapest safe rung, under a doctrine about what may
be removed. livery.strongroom.testing runs the conformance
scenarios under spec/conformance against any implementation of the
same API; importing this package does not load it. The codec,
livery.strongroom.cbor, is part of the public API, declared here and
imported by its own path; importing this module does not load it either.
Modules:
| Name | Description |
|---|---|
cbor |
Deterministic CBOR: one value, one encoding, one name. |
testing |
The conformance kit: the scenarios under |
Classes:
| Name | Description |
|---|---|
FormatError |
A value does not satisfy one of the store's formats. |
Algorithm |
One registry entry: how an algorithm names bytes. |
Digest |
A parsed digest string. |
HashConstructor |
A hashlib-style constructor: optional initial bytes, a hasher back. |
Hasher |
What a registry entry's constructor returns: hashlib's shape. |
ErasedObject |
The named object was erased; its tombstone says why. |
GroupHalfApplied |
A commit already applied part of the group; only a commit finishes it. |
IntegrityError |
Bytes do not match their name: a landing mismatch or a corrupt object. |
LockTimeout |
A live lock on the ref was not released within the timeout. |
ManifestError |
The root manifest is missing, malformed, or of another store. |
MissingObject |
The named object is not in the store. |
NoSuchPending |
The named pending publish does not exist: retired, committed, or never begun. |
NotAGroup |
The pending ref is a single publish's, not a group's. |
NotFastForward |
A monotone ref's new target does not descend from its current one. |
RefConflict |
A compare-and-swap lost: the ref does not name the expected previous digest. |
RefProtected |
The ref's mutation class does not allow dropping it. |
RefTampered |
A ref and its record disagree: an out-of-band edit or a torn update. |
StoreError |
The store refused an operation; the message says why. |
UnknownNamespace |
A ref names a namespace the store was not opened with. |
WriteOnceRefused |
A write-once ref already names a different digest. |
Subject |
A subject slot: a person, a call key, a receipt digest, or code. |
Group |
A group of moves that has begun and not yet committed. |
Move |
One ref move of a group. |
Transaction |
A group under a |
Pending |
A publish that has begun and not yet committed. |
SweepReport |
What a sweep did. |
RefRecord |
What a ref update writes beside the ref. |
Tombstone |
What an erased object becomes, in every tier. |
RungUnavailable |
The rung cannot produce this path here; try the next one. |
FolderSource |
A folder in the store layout: a mirror, or a read-only share. |
HttpSource |
An HTTP base URL serving the store layout: nginx over a folder. |
OriginHint |
One URL that yields one named object: the ingress form. |
Unreachable |
A source did not answer for this object; the fetch skips it and reports. |
Landed |
What a landing produced. |
Manifest |
The root manifest: which layout and which algorithm this store speaks. |
Namespace |
A ref namespace and its mutation class. |
ScrubReport |
What a scrub found. |
Store |
A local store: |
Entry |
A blob or a subtree inside a tree. |
Link |
A symlink inside a tree; the target string is its whole content. |
Tree |
A directory: its entries, sorted by name bytes. |
Version |
Provenance over a tree. |
DropReport |
What dropping a view removed and what it left. |
ShedReport |
What a shed did. |
ViewEntry |
One path a view created, and how. |
ViewRecord |
A view: local state in the index, and a root while its directory exists. |
Functions:
| Name | Description |
|---|---|
canonical |
Encode value as RFC 8785 canonical JSON, UTF-8. |
digest_of |
Name data under algorithm. |
digest_stream |
Name the bytes read from stream to its end, without holding them. |
check_timestamp |
Return value when it is an RFC 3339 UTC instant, or refuse. |
fetch_url |
Open url for reading, or raise Unreachable naming why. |
now |
The current instant, RFC 3339 in UTC to the second. |
silent |
The default progress sink: report nothing. |
check_name |
Return name when it is portable, or refuse it. |
check_target |
Return a symlink target when it is portable, or refuse it. |
Attributes:
| Name | Type | Description |
|---|---|---|
Value |
TypeAlias
|
A JSON value the store's formats may carry: no float, nested the same way. |
ALGORITHMS |
Final[dict[str, Algorithm]]
|
The registry. Only cryptographic, full-length digests may name bytes. |
SHA256 |
Final
|
The required algorithm, and the only one in the registry. |
SubjectKind |
Who or what may fill a subject slot. |
|
PENDING |
The namespace of publishes that have begun and not committed. |
|
PINS |
The namespace of explicit roots. |
|
RUNGS |
tuple[Rung, ...]
|
|
MadeRung |
The rungs that create a path inside a view: every rung but reference. |
|
Rung |
The ladder, cheapest first. |
|
FillPolicy |
What a folder hit does: land a local copy, or answer into the folder. |
|
Progress |
Where a fetch reports a skipped or refused source; never prints on its own. |
|
Source |
Anything a store may consult for an object. |
|
LAYOUT_VERSION |
The layout this implementation speaks. |
|
MANIFEST_NAME |
The root manifest's file name. |
|
MUTATION_CLASSES |
tuple[MutationClass, ...]
|
|
OWNED |
tuple[Namespace, ...]
|
The two namespaces the store declares itself; see |
Clock |
Returns the current instant as an RFC 3339 UTC string. |
|
MutationClass |
How a namespace's refs may move; see |
|
ObjectState |
The three states an object name can be in. |
|
NAME_BUDGET |
The most UTF-8 bytes one name may take. |
|
EntryKind |
What a digest-bearing entry names. |
|
ENTRY_RUNGS |
tuple[EntryRung, ...]
|
|
PATH_BUDGET |
The most UTF-8 bytes a path inside a view may take, separators included. |
|
EntryRung |
How one view entry was made: a ladder rung, a real symlink, or not at all. |
|
__version__ |
|
Value
module-attribute
¶
Value: TypeAlias = 'bool | int | str | list[Any] | dict[str, Any] | None'
A JSON value the store's formats may carry: no float, nested the same way.
ALGORITHMS
module-attribute
¶
The registry. Only cryptographic, full-length digests may name bytes.
SHA256
module-attribute
¶
The required algorithm, and the only one in the registry.
SubjectKind
module-attribute
¶
SubjectKind = Literal['person', 'call', 'receipt', 'code']
Who or what may fill a subject slot.
PENDING
module-attribute
¶
The namespace of publishes that have begun and not committed.
RUNGS
module-attribute
¶
MadeRung
module-attribute
¶
MadeRung = Literal['clone', 'hardlink', 'link', 'copy']
The rungs that create a path inside a view: every rung but reference.
Rung
module-attribute
¶
Rung = Literal['reference', 'clone', 'hardlink', 'link', 'copy']
The ladder, cheapest first.
FillPolicy
module-attribute
¶
FillPolicy = Literal['copy', 'reference']
What a folder hit does: land a local copy, or answer into the folder.
Progress
module-attribute
¶
Where a fetch reports a skipped or refused source; never prints on its own.
Source
module-attribute
¶
Source = FolderSource | HttpSource | OriginHint
Anything a store may consult for an object.
MUTATION_CLASSES
module-attribute
¶
MUTATION_CLASSES: tuple[MutationClass, ...] = ('write-once', 'monotone', 'volatile')
OWNED
module-attribute
¶
The two namespaces the store declares itself; see spec/namespaces.md.
Clock
module-attribute
¶
Returns the current instant as an RFC 3339 UTC string.
MutationClass
module-attribute
¶
MutationClass = Literal['write-once', 'monotone', 'volatile']
How a namespace's refs may move; see spec/namespaces.md.
ObjectState
module-attribute
¶
ObjectState = Literal['present', 'absent', 'erased']
The three states an object name can be in.
ENTRY_RUNGS
module-attribute
¶
PATH_BUDGET
module-attribute
¶
The most UTF-8 bytes a path inside a view may take, separators included.
EntryRung
module-attribute
¶
EntryRung = Literal['clone', 'hardlink', 'link', 'copy', 'symlink', 'parked']
How one view entry was made: a ladder rung, a real symlink, or not at all.
__all__
module-attribute
¶
__all__ = ['ALGORITHMS', 'ENTRY_RUNGS', 'LAYOUT_VERSION', 'MANIFEST_NAME', 'MUTATION_CLASSES', 'NAME_BUDGET', 'OWNED', 'PATH_BUDGET', 'PENDING', 'PINS', 'RUNGS', 'SHA256', 'Algorithm', 'Clock', 'Digest', 'DropReport', 'Entry', 'EntryKind', 'EntryRung', 'ErasedObject', 'FillPolicy', 'FolderSource', 'FormatError', 'Group', 'GroupHalfApplied', 'HashConstructor', 'Hasher', 'HttpSource', 'IntegrityError', 'Landed', 'Link', 'LockTimeout', 'MadeRung', 'Manifest', 'ManifestError', 'MissingObject', 'Move', 'MutationClass', 'Namespace', 'NoSuchPending', 'NotAGroup', 'NotFastForward', 'ObjectState', 'OriginHint', 'Pending', 'Progress', 'RefConflict', 'RefProtected', 'RefRecord', 'RefTampered', 'Rung', 'RungUnavailable', 'ScrubReport', 'ShedReport', 'Source', 'Store', 'StoreError', 'Subject', 'SubjectKind', 'SweepReport', 'Tombstone', 'Transaction', 'Tree', 'UnknownNamespace', 'Unreachable', 'Value', 'Version', 'ViewEntry', 'ViewRecord', 'WriteOnceRefused', '__version__', 'canonical', 'cbor', 'check_name', 'check_target', 'check_timestamp', 'digest_of', 'digest_stream', 'fetch_url', 'now', 'silent', 'testing']
FormatError ¶
Bases: ValueError
A value does not satisfy one of the store's formats.
The message names the rule that refused it. Every format refusal raises this one class, so a caller validating a whole tree catches one thing.
Algorithm
dataclass
¶
Algorithm(name: str, encoded: Pattern[str], constructor: HashConstructor)
One registry entry: how an algorithm names bytes.
Attributes:
| Name | Type | Description |
|---|---|---|
name |
str
|
the algorithm's spelling in a digest string. |
encoded |
Pattern[str]
|
the pattern the encoded part must match in full. |
constructor |
HashConstructor
|
the hashlib constructor that computes it. |
Digest
dataclass
¶
A parsed digest string.
Attributes:
| Name | Type | Description |
|---|---|---|
algorithm |
str
|
the registry entry's name, |
encoded |
str
|
the lowercase hex of the digest. |
Methods:
| Name | Description |
|---|---|
__str__ |
|
parse |
Parse a digest string, refusing anything the registry lacks. |
path_parts
property
¶
The object's path under objects/: algorithm, fan-out, rest.
parse
classmethod
¶
Parse a digest string, refusing anything the registry lacks.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
text
|
str
|
the |
required |
Returns:
| Type | Description |
|---|---|
Digest
|
The digest. |
Raises:
| Type | Description |
|---|---|
FormatError
|
when the text is not in the OCI grammar, the algorithm is not registered, or the encoded part is not the full lowercase form the algorithm requires. A truncated or uppercase digest is refused so one object has one name. |
Source code in packages/strongroom/src/livery/strongroom/_digest.py
HashConstructor ¶
Hasher ¶
ErasedObject ¶
Bases: StoreError
The named object was erased; its tombstone says why.
GroupHalfApplied ¶
Bases: StoreError
A commit already applied part of the group; only a commit finishes it.
IntegrityError ¶
Bases: StoreError
Bytes do not match their name: a landing mismatch or a corrupt object.
LockTimeout ¶
Bases: StoreError
A live lock on the ref was not released within the timeout.
ManifestError ¶
Bases: StoreError
The root manifest is missing, malformed, or of another store.
MissingObject ¶
Bases: StoreError
The named object is not in the store.
NoSuchPending ¶
Bases: StoreError
The named pending publish does not exist: retired, committed, or never begun.
NotAGroup ¶
Bases: StoreError
The pending ref is a single publish's, not a group's.
NotFastForward ¶
Bases: RefConflict
A monotone ref's new target does not descend from its current one.
RefConflict ¶
Bases: StoreError
A compare-and-swap lost: the ref does not name the expected previous digest.
RefProtected ¶
Bases: StoreError
The ref's mutation class does not allow dropping it.
RefTampered ¶
Bases: StoreError
A ref and its record disagree: an out-of-band edit or a torn update.
UnknownNamespace ¶
Bases: StoreError
A ref names a namespace the store was not opened with.
WriteOnceRefused ¶
Bases: RefConflict
A write-once ref already names a different digest.
Subject
dataclass
¶
Subject(kind: SubjectKind, value: str)
A subject slot: a person, a call key, a receipt digest, or code.
A version's producer, a ref record's writer and a tombstone's authority are subjects. The slot is shaped so that attested code fits it from the first persisted byte.
Attributes:
| Name | Type | Description |
|---|---|---|
kind |
SubjectKind
|
which of the four the subject is. |
value |
str
|
the subject's identity, its spelling owned by the kind: a person's identifier, a call key, a receipt or code digest as a digest string. |
Methods:
| Name | Description |
|---|---|
to_json |
The subject as a JSON object. |
from_json |
Decode a subject, refusing an unknown kind or an empty value. |
to_json ¶
from_json
classmethod
¶
Decode a subject, refusing an unknown kind or an empty value.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
value
|
Value
|
the JSON value to decode. |
required |
where
|
str
|
the field being decoded, for the refusal's message. |
required |
Returns:
| Type | Description |
|---|---|
Subject
|
The subject. |
Raises:
| Type | Description |
|---|---|
FormatError
|
when the shape or the kind is wrong. |
Source code in packages/strongroom/src/livery/strongroom/_fields.py
Group
dataclass
¶
Group(id: str, moves: tuple[Move, ...], applied: int, lease: int, manifest: Digest, record: RefRecord)
A group of moves that has begun and not yet committed.
Attributes:
| Name | Type | Description |
|---|---|---|
id |
str
|
the pending ref's name under |
moves |
tuple[Move, ...]
|
the moves, in the order they apply. |
applied |
int
|
how many of them a commit has already applied; more than zero only after a commit stopped part-way. |
lease |
int
|
the seconds the group expected to take, from |
manifest |
Digest
|
the tree of every move's target the pending ref names. |
record |
RefRecord
|
the pending ref's record, the journal in its |
Move
dataclass
¶
Move(namespace: str, path: str, digest: Digest, previous: Digest | None, receipt: Digest | None = None, meta: dict[str, Value] = dict())
One ref move of a group.
Attributes:
| Name | Type | Description |
|---|---|---|
namespace |
str
|
a declared namespace. |
path |
str
|
the ref's path. |
digest |
Digest
|
what the ref names afterwards. |
previous |
Digest | None
|
what the ref is expected to name now; None to create. |
receipt |
Digest | None
|
the receipt of the moving call, if any. |
meta |
dict[str, Value]
|
the consumer's own object for the ref's record. |
Methods:
| Name | Description |
|---|---|
to_json |
The move as a JSON object, the journal's entry. |
from_json |
A move from a journal entry. |
to_json ¶
The move as a JSON object, the journal's entry.
Source code in packages/strongroom/src/livery/strongroom/_groups.py
from_json
classmethod
¶
A move from a journal entry.
Raises:
| Type | Description |
|---|---|
FormatError
|
when the entry is not a move. |
Source code in packages/strongroom/src/livery/strongroom/_groups.py
Transaction ¶
A group under a with: moves recorded as they come, the exit decides.
Entering begins the group. A clean exit commits it, every move or
none, and records then holds the records written. An exception
before the commit began retires the group, nothing moved. A
commit refused on its checks retires the group too, since nothing
moved, and the refusal is what the with raises. A commit that
stopped between two applies re-raises and leaves the group for a
later livery.strongroom.Store.commit on its id.
Attributes:
| Name | Type | Description |
|---|---|---|
id |
the group's pending ref name, known after entry. |
|
records |
tuple[RefRecord, ...]
|
the records the commit wrote, in move order; empty until the commit. |
Methods:
| Name | Description |
|---|---|
move |
Record a move; the group's state after it. |
Source code in packages/strongroom/src/livery/strongroom/_groups.py
move ¶
move(namespace: str, path: str, digest: Digest, *, previous: Digest | None, receipt: Digest | None = None, meta: dict[str, Value] | None = None) -> Group
Record a move; the group's state after it.
Source code in packages/strongroom/src/livery/strongroom/_groups.py
Pending
dataclass
¶
SweepReport
dataclass
¶
SweepReport(reached: tuple[Digest, ...] = tuple(), removed: tuple[Digest, ...] = tuple(), scratch_removed: tuple[Path, ...] = tuple(), pending: tuple[str, ...] = tuple())
What a sweep did.
Attributes:
| Name | Type | Description |
|---|---|---|
reached |
tuple[Digest, ...]
|
every object marking reached, present or not. |
removed |
tuple[Digest, ...]
|
every object removed as unreached. |
scratch_removed |
tuple[Path, ...]
|
every scratch file removed as older than the age bound. |
pending |
tuple[str, ...]
|
the pending refs found by the re-scan immediately before deleting, every one a root. |
RefRecord
dataclass
¶
RefRecord(digest: Digest, previous: Digest | None, by: Subject, at: str, receipt: Digest | None = None, meta: dict[str, Value] = dict())
What a ref update writes beside the ref.
Attributes:
| Name | Type | Description |
|---|---|---|
digest |
Digest
|
what the ref names after the update. |
previous |
Digest | None
|
what it named before, or None on creation. |
by |
Subject
|
who moved it. |
at |
str
|
when, as an RFC 3339 UTC instant. |
receipt |
Digest | None
|
across a trust boundary, the signed receipt of the call that moved the ref; None under local custody. |
meta |
dict[str, Value]
|
an opaque object a consumer may use, such as validators for a URL or a freshness window for a query. Strongroom stores it and never reads it. |
Methods:
| Name | Description |
|---|---|
__post_init__ |
|
to_json |
The record as a JSON object. |
from_json |
Decode a record from its JSON value. |
encode |
The record's canonical JSON bytes. |
decode |
Decode a record from its stored bytes. |
__post_init__ ¶
to_json ¶
The record as a JSON object.
Source code in packages/strongroom/src/livery/strongroom/_records.py
from_json
classmethod
¶
Decode a record from its JSON value.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
value
|
Value
|
the JSON value. |
required |
Returns:
| Type | Description |
|---|---|
RefRecord
|
The record. |
Raises:
| Type | Description |
|---|---|
FormatError
|
when the shape or a field is wrong. |
Source code in packages/strongroom/src/livery/strongroom/_records.py
decode
classmethod
¶
Decode a record from its stored bytes.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data
|
bytes
|
the record file's bytes. |
required |
Returns:
| Type | Description |
|---|---|
RefRecord
|
The record. |
Raises:
| Type | Description |
|---|---|
FormatError
|
when the bytes are not JSON, not a record, or not canonical. |
Source code in packages/strongroom/src/livery/strongroom/_records.py
Tombstone
dataclass
¶
What an erased object becomes, in every tier.
Attributes:
| Name | Type | Description |
|---|---|---|
digest |
Digest
|
the erased object's name, which stays valid in every tree that names it. |
at |
str
|
when, as an RFC 3339 UTC instant. |
authority |
Subject
|
who erased it. |
receipt |
Digest | None
|
the receipt of the erasing call, or None under local custody. |
reason |
str
|
free text, shown to a reader whose view meets the erased entry. |
Methods:
| Name | Description |
|---|---|
__post_init__ |
|
to_json |
The tombstone as a JSON object. |
from_json |
Decode a tombstone from its JSON value. |
encode |
The tombstone's canonical JSON bytes. |
decode |
Decode a tombstone from its stored bytes. |
__post_init__ ¶
to_json ¶
The tombstone as a JSON object.
Source code in packages/strongroom/src/livery/strongroom/_records.py
from_json
classmethod
¶
Decode a tombstone from its JSON value.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
value
|
Value
|
the JSON value. |
required |
Returns:
| Type | Description |
|---|---|
Tombstone
|
The tombstone. |
Raises:
| Type | Description |
|---|---|
FormatError
|
when the shape or a field is wrong. |
Source code in packages/strongroom/src/livery/strongroom/_records.py
decode
classmethod
¶
Decode a tombstone from its stored bytes.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data
|
bytes
|
the tombstone file's bytes. |
required |
Returns:
| Type | Description |
|---|---|
Tombstone
|
The tombstone. |
Raises:
| Type | Description |
|---|---|
FormatError
|
when the bytes are not JSON, not a tombstone, or not canonical. |
Source code in packages/strongroom/src/livery/strongroom/_records.py
FolderSource
dataclass
¶
FolderSource(path: Path, fill: FillPolicy = 'copy')
A folder in the store layout: a mirror, or a read-only share.
Attributes:
| Name | Type | Description |
|---|---|---|
path |
Path
|
the folder's root, holding a manifest of the same layout and algorithm as the store that consults it. |
fill |
FillPolicy
|
|
Methods:
| Name | Description |
|---|---|
__post_init__ |
|
describe |
The source, for a report. |
__post_init__ ¶
HttpSource
dataclass
¶
An HTTP base URL serving the store layout: nginx over a folder.
Attributes:
| Name | Type | Description |
|---|---|---|
base_url |
str
|
the URL the layout hangs from; a trailing slash is ignored. |
connect_timeout |
float
|
seconds to wait for the server to answer at all; an unanswered source is skipped and reported. |
transfer_timeout |
float
|
seconds a stalled transfer may wait between bytes. |
Methods:
| Name | Description |
|---|---|
__post_init__ |
|
describe |
The source, for a report. |
url |
The URL of relative, a path in the layout. |
OriginHint
dataclass
¶
One URL that yields one named object: the ingress form.
A tool spec's {url, sha256} per host is an origin hint, and so is
a vendor download. Consulted only for its own digest, and never
when the store is offline.
Attributes:
| Name | Type | Description |
|---|---|---|
digest |
Digest
|
the object the URL yields. |
url |
str
|
where to get it. |
connect_timeout |
float
|
seconds to wait for an answer. |
transfer_timeout |
float
|
seconds a stalled transfer may wait. |
Methods:
| Name | Description |
|---|---|
__post_init__ |
|
describe |
The source, for a report. |
__post_init__ ¶
Unreachable ¶
Bases: Exception
A source did not answer for this object; the fetch skips it and reports.
Attributes:
| Name | Type | Description |
|---|---|---|
status |
The HTTP status the source answered with, when the refusal is an answer (a 404, a 503) rather than a connection that failed or a chain that did not end; None otherwise. A caller deciding whether to try again reads it: a connection failure and a 5xx are worth another try, a 404 is not. |
Source code in packages/strongroom/src/livery/strongroom/_sources.py
Landed
dataclass
¶
Manifest
dataclass
¶
The root manifest: which layout and which algorithm this store speaks.
Attributes:
| Name | Type | Description |
|---|---|---|
layout |
int
|
the layout version. |
algorithm |
str
|
the one algorithm of this address space. |
Methods:
| Name | Description |
|---|---|
encode |
The manifest's canonical JSON bytes. |
decode |
Decode a manifest, refusing bytes that are not one. |
decode
classmethod
¶
Decode a manifest, refusing bytes that are not one.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data
|
bytes
|
the manifest file's bytes. |
required |
Returns:
| Type | Description |
|---|---|
Manifest
|
The manifest. |
Raises:
| Type | Description |
|---|---|
ManifestError
|
when the bytes are not a manifest. |
Source code in packages/strongroom/src/livery/strongroom/_store.py
Namespace
dataclass
¶
Namespace(name: str, mutation: MutationClass)
A ref namespace and its mutation class.
Attributes:
| Name | Type | Description |
|---|---|---|
name |
str
|
the namespace, lowercase letters, digits and dashes. |
mutation |
MutationClass
|
how its refs may move. |
Methods:
| Name | Description |
|---|---|
__post_init__ |
|
__post_init__ ¶
Source code in packages/strongroom/src/livery/strongroom/_store.py
ScrubReport
dataclass
¶
Store ¶
Store(root: Path, algorithm: Algorithm, namespaces: Iterable[Namespace], *, clock: Clock, lock_stale: float, lock_timeout: float, sources: Iterable[Source], offline: bool, progress: Progress)
A local store: objects/ by digest, refs/ by name.
Open one with livery.strongroom.Store.create or
livery.strongroom.Store.open. Every namespace a caller will
write is declared at open with its mutation class; pins/ and
pending/ are declared by the store.
Attributes:
| Name | Type | Description |
|---|---|---|
root |
Path
|
the store's directory. |
algorithm |
Algorithm
|
the address space's algorithm. |
namespaces |
dict[str, Namespace]
|
every declared namespace by name. |
sources |
tuple[Source, ...]
|
the sources consulted in order for an object the local directory lacks. |
offline |
bool
|
whether origin hints are never consulted. |
Methods:
| Name | Description |
|---|---|
create |
Create a store at root and open it. |
open |
Open the store at root. |
object_path |
The path an object of digest lives at, present or not. |
state |
Whether digest is present, absent, or erased here. |
land |
Land bytes as an object, verifying them on the way in. |
put |
Land data and return its name. |
path |
A read-only path to the object, verified on first access. |
read |
The object's bytes, through livery.strongroom.Store.path. |
verify |
Hash the object in full and mark it verified. |
verified_size |
The size this store verified the object at, or None if never. |
objects |
Every object present here, in no particular order. |
scrub |
Hash every object and remove the ones that do not match. |
fetch |
A path to the object, consulting the sources when it is not here. |
fill |
Land digests into the folder tier at target. |
tombstone |
The tombstone under the object's path, or None when there is none. |
write_tombstone |
Write stone under its object's path, in this tier. |
evict |
Remove the object's bytes and its marks; the name is untouched. |
clock |
The current instant, from the clock the store was opened with. |
publish_begin |
Begin a publish: root target under |
publish_commit |
Commit a publish: move the real ref, then drop the pending one. |
retire |
Drop a pending ref deliberately, for a publish that will not commit. |
pendings |
Every pending publish's id, sorted; groups included. |
begin |
Open a group of ref moves: an empty journal under |
add |
Append a move to the group; its manifest roots the target from now on. |
commit |
Apply the group's moves, every one or none, and drop the pending ref. |
group |
The group under |
groups |
Every group that has begun and not committed, sorted by id. |
transaction |
A group under a |
drop_ref |
Remove a volatile ref and its record by compare-and-swap. |
pin |
Root digest under |
unpin |
Drop |
erase |
Erase the object's bytes and keep the fact as a tombstone. |
reachable |
Every object reachable from digest, itself included. |
roots |
Every root: what every ref on disk names, and every live view's tree. |
sweep |
Remove every object no root reaches, and old scratch. |
view |
Fill at from the tree by the cheapest safe rung per entry. |
views |
Every recorded view, live or not, by id. |
view_record |
The record of one view, or None. |
drop_view |
Remove what the view's record lists, and nothing else. |
collect |
Read the declared paths under at back into a tree, landing every object. |
prefetch |
Fetch digest and everything it reaches, for offline use. |
shed |
Evict every local copy source holds, except what a live view needs. |
head |
Whether the HTTP source answers a HEAD for relative. |
ref_path |
The file a ref lives at, present or not. |
ref |
What the ref names, or None when it does not exist. |
record |
The record beside the ref, or None when there is none. |
refs |
Every ref path in the namespace, sorted. |
set_ref |
Move a ref by compare-and-swap, writing its record. |
Source code in packages/strongroom/src/livery/strongroom/_store.py
namespaces
instance-attribute
¶
create
classmethod
¶
create(root: Path, *, algorithm: str = 'sha256', namespaces: Iterable[Namespace] = (), clock: Clock = now, lock_stale: float = 600.0, lock_timeout: float = 30.0, sources: Iterable[Source] = (), offline: bool = False, progress: Progress = silent) -> Store
Create a store at root and open it.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
root
|
Path
|
the directory; created when missing. |
required |
algorithm
|
str
|
a registry entry's name. |
'sha256'
|
namespaces
|
Iterable[Namespace]
|
the caller's namespaces, each with its class. |
()
|
clock
|
Clock
|
the instant source for records. |
now
|
lock_stale
|
float
|
seconds after which a lock is broken even when its holder looks alive: the pid of a holder that died can belong to another process by then. |
600.0
|
lock_timeout
|
float
|
seconds to wait for a live lock. |
30.0
|
sources
|
Iterable[Source]
|
where to look for an object the store lacks, in order. A folder source is checked now; an HTTP source on first use. |
()
|
offline
|
bool
|
never consult an origin hint. |
False
|
progress
|
Progress
|
where a fetch reports a skipped or refused source; silent by default. |
silent
|
Returns:
| Type | Description |
|---|---|
Store
|
The open store. |
Raises:
| Type | Description |
|---|---|
ManifestError
|
when root already holds a manifest. |
FormatError
|
when the algorithm is not registered. |
Source code in packages/strongroom/src/livery/strongroom/_store.py
open
classmethod
¶
open(root: Path, *, namespaces: Iterable[Namespace] = (), clock: Clock = now, lock_stale: float = 600.0, lock_timeout: float = 30.0, sources: Iterable[Source] = (), offline: bool = False, progress: Progress = silent) -> Store
Open the store at root.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
root
|
Path
|
the store's directory. |
required |
namespaces
|
Iterable[Namespace]
|
the caller's namespaces, each with its class. |
()
|
clock
|
Clock
|
the instant source for records. |
now
|
lock_stale
|
float
|
seconds after which a lock is broken even when its holder looks alive: the pid of a holder that died can belong to another process by then. |
600.0
|
lock_timeout
|
float
|
seconds to wait for a live lock. |
30.0
|
sources
|
Iterable[Source]
|
where to look for an object the store lacks, in order. A folder source is checked now; an HTTP source on first use. |
()
|
offline
|
bool
|
never consult an origin hint. |
False
|
progress
|
Progress
|
where a fetch reports a skipped or refused source; silent by default. |
silent
|
Returns:
| Type | Description |
|---|---|
Store
|
The store. |
Raises:
| Type | Description |
|---|---|
ManifestError
|
when the manifest is missing or malformed, or names a layout or an algorithm this implementation does not speak; or when a folder source's manifest is missing or of another store. |
ValueError
|
when a namespace is declared twice with different classes. |
Source code in packages/strongroom/src/livery/strongroom/_store.py
object_path ¶
The path an object of digest lives at, present or not.
Raises:
| Type | Description |
|---|---|
IntegrityError
|
when the digest is of another algorithm than this store's; one address space, one algorithm. |
Source code in packages/strongroom/src/livery/strongroom/_store.py
state ¶
state(digest: Digest) -> ObjectState
Whether digest is present, absent, or erased here.
Source code in packages/strongroom/src/livery/strongroom/_store.py
land ¶
Land bytes as an object, verifying them on the way in.
The bytes stream through the digest into a .part scratch file
and are moved into place atomically. Landing is idempotent: an
object that already exists is success without a write. Two
writers of the same digest land identical bytes, so no lock is
taken.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
source
|
IO[bytes] | bytes
|
the bytes, or a binary stream read to its end. |
required |
expected
|
Digest | None
|
the digest the bytes must have, when known. A mismatch removes the scratch and raises. |
None
|
Returns:
| Type | Description |
|---|---|
Landed
|
The landing. |
Raises:
| Type | Description |
|---|---|
IntegrityError
|
when expected is given and the bytes do not match it. |
ErasedObject
|
when the object was erased here; its tombstone stands until a sweep decides otherwise. |
Source code in packages/strongroom/src/livery/strongroom/_store.py
put ¶
path ¶
A read-only path to the object, verified on first access.
Size is checked against the verified mark on every access; an object without a mark (landed by copy, not by this store) is hashed in full and marked.
Raises:
| Type | Description |
|---|---|
MissingObject
|
when the object is absent. |
ErasedObject
|
when the object was erased. |
IntegrityError
|
when the bytes do not match the name; the object and its mark are removed so a fill lands it again. |
Source code in packages/strongroom/src/livery/strongroom/_store.py
read ¶
The object's bytes, through livery.strongroom.Store.path.
verify ¶
Hash the object in full and mark it verified.
Returns:
| Type | Description |
|---|---|
int
|
The object's size. |
Raises:
| Type | Description |
|---|---|
MissingObject
|
when the object is absent. |
IntegrityError
|
when the bytes do not match; the object and its mark are removed. |
Source code in packages/strongroom/src/livery/strongroom/_store.py
verified_size ¶
The size this store verified the object at, or None if never.
Source code in packages/strongroom/src/livery/strongroom/_store.py
objects ¶
Every object present here, in no particular order.
Source code in packages/strongroom/src/livery/strongroom/_store.py
scrub ¶
scrub() -> ScrubReport
Hash every object and remove the ones that do not match.
Source code in packages/strongroom/src/livery/strongroom/_store.py
fetch ¶
A path to the object, consulting the sources when it is not here.
Sources are consulted in order. A hit is verified on arrival: a copy-policy folder, an HTTP tier and an origin hint land the bytes here; a reference-policy folder answers with a path into the folder and lands nothing. An unreachable source is skipped and reported; a source serving wrong bytes is refused and reported, and the next one consulted. An origin hint is consulted only for its own digest and never when offline.
Returns:
| Type | Description |
|---|---|
Path
|
A read-only path: this store's object, or the folder's |
Path
|
under a reference policy. |
Raises:
| Type | Description |
|---|---|
MissingObject
|
when no source has it. The message names the digest, and when an origin hint was passed over because the store is offline, the URL that would have satisfied it. |
ErasedObject
|
when the object was erased here. |
ManifestError
|
when an HTTP source's manifest is of another store. |
Source code in packages/strongroom/src/livery/strongroom/_store.py
fill ¶
Land digests into the folder tier at target.
The folder becomes a store of this store's algorithm when it is not one already, and afterwards opens as a livery.strongroom.FolderSource that serves every filled digest. Each object is fetched through this store's sources.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
target
|
Path
|
the folder; created when missing. |
required |
digests
|
Iterable[Digest]
|
the objects to land there. |
required |
Returns:
| Type | Description |
|---|---|
list[Landed]
|
One landing per digest, in order. |
Raises:
| Type | Description |
|---|---|
ManifestError
|
when target is a store of another layout or algorithm. |
MissingObject
|
when a digest is not obtainable. |
Source code in packages/strongroom/src/livery/strongroom/_store.py
tombstone ¶
The tombstone under the object's path, or None when there is none.
Raises:
| Type | Description |
|---|---|
IntegrityError
|
when the file under the tombstone's name is not a tombstone. |
Source code in packages/strongroom/src/livery/strongroom/_store.py
write_tombstone ¶
write_tombstone(stone: Tombstone) -> None
Write stone under its object's path, in this tier.
Source code in packages/strongroom/src/livery/strongroom/_store.py
evict ¶
evict(digest: Digest) -> None
Remove the object's bytes and its marks; the name is untouched.
Source code in packages/strongroom/src/livery/strongroom/_store.py
publish_begin ¶
Begin a publish: root target under pending/<id> first.
The pending ref is a root while it exists, so a sweep during the publish keeps every object the target names. Nothing removes a pending ref on a clock; a crashed publish is retired deliberately with livery.strongroom.Store.retire.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
target
|
Digest
|
the tree or version to publish; present here. |
required |
by
|
Subject
|
who publishes. |
required |
lease
|
float
|
seconds the publish expects to take; recorded in the pending ref's record for whoever retires it. |
3600.0
|
Returns:
| Type | Description |
|---|---|
Pending
|
The pending publish. |
Raises:
| Type | Description |
|---|---|
MissingObject
|
when target is not present here. |
Source code in packages/strongroom/src/livery/strongroom/_store.py
publish_commit ¶
publish_commit(pending_id: str, namespace: str, path: str, *, previous: Digest | None, by: Subject, receipt: Digest | None = None, meta: dict[str, Value] | None = None) -> RefRecord
Commit a publish: move the real ref, then drop the pending one.
The move obeys the namespace's mutation class exactly as livery.strongroom.Store.set_ref does.
Raises:
| Type | Description |
|---|---|
NoSuchPending
|
when |
RefConflict
|
when the real ref does not name previous, or its class refuses the move. |
Source code in packages/strongroom/src/livery/strongroom/_store.py
retire ¶
Drop a pending ref deliberately, for a publish that will not commit.
Returns:
| Type | Description |
|---|---|
Digest
|
What the pending ref named. |
Raises:
| Type | Description |
|---|---|
NoSuchPending
|
when it does not exist. |
GroupHalfApplied
|
when it is a group a commit already applied part of; only a commit finishes it. |
Source code in packages/strongroom/src/livery/strongroom/_store.py
pendings ¶
begin ¶
add ¶
add(group_id: str, namespace: str, path: str, digest: Digest, *, previous: Digest | None, receipt: Digest | None = None, meta: dict[str, Value] | None = None) -> Group
Append a move to the group; its manifest roots the target from now on.
The arguments are livery.strongroom.Store.set_ref's. The move is checked at commit, not here; only the target's presence is required now.
Raises:
| Type | Description |
|---|---|
NoSuchPending
|
when the group does not exist. |
NotAGroup
|
when |
GroupHalfApplied
|
when a commit already applied part of it. |
MissingObject
|
when digest is not present here. |
RefConflict
|
when the group already moves that ref. |
Source code in packages/strongroom/src/livery/strongroom/_store.py
commit ¶
Apply the group's moves, every one or none, and drop the pending ref.
Holds the maintenance lease and the moved refs' locks for the checks and the moves only. A commit that stopped between two applies is finished by calling this again with the same id.
Returns:
| Type | Description |
|---|---|
tuple[RefRecord, ...]
|
One record per move, in order. |
Raises:
| Type | Description |
|---|---|
NoSuchPending
|
when the group does not exist. |
NotAGroup
|
when |
RefConflict
|
when a ref does not name its expected previous or its class refuses the move; nothing moved. |
LockTimeout
|
when the lease or a lock outlasts the timeout. |
Source code in packages/strongroom/src/livery/strongroom/_store.py
group ¶
The group under pending/<group_id>.
Raises:
| Type | Description |
|---|---|
NoSuchPending
|
when it does not exist. |
NotAGroup
|
when it is a single publish's. |
Source code in packages/strongroom/src/livery/strongroom/_store.py
groups ¶
transaction ¶
transaction(*, by: Subject, lease: float = 3600.0) -> AbstractContextManager[Transaction]
A group under a with: record moves, and the exit commits or retires.
with store.transaction(by=me) as tx:
tx.move("tools", "bun@1.3", tree, previous=None)
tx.move("datasets", "corpus/main", version, previous=v1)
records = tx.records
See livery.strongroom.Transaction for what each exit does.
Source code in packages/strongroom/src/livery/strongroom/_store.py
drop_ref ¶
Remove a volatile ref and its record by compare-and-swap.
Raises:
| Type | Description |
|---|---|
RefProtected
|
when the namespace is write-once or monotone. |
RefConflict
|
when the ref does not name previous. |
Source code in packages/strongroom/src/livery/strongroom/_store.py
pin ¶
Root digest under pins/<name>, replacing an earlier pin of that name.
unpin ¶
Drop pins/<name> under the maintenance lease, never beside a sweep.
Returns:
| Type | Description |
|---|---|
Digest
|
What the pin named. |
Raises:
| Type | Description |
|---|---|
RefConflict
|
when the pin does not exist. |
LockTimeout
|
when a sweep holds the lease past the timeout. |
Source code in packages/strongroom/src/livery/strongroom/_store.py
erase ¶
Erase the object's bytes and keep the fact as a tombstone.
The name stays valid in every tree that carries it; a path to the object raises naming this tombstone's reason; landing the bytes again is refused while the tombstone stands. An absent object may be erased too, which refuses its future landing.
Source code in packages/strongroom/src/livery/strongroom/_store.py
reachable ¶
Every object reachable from digest, itself included.
Structured formats are read by shape, never by namespace: a version yields its tree, parents, receipt and attachments, a tree its entries, anything else nothing. An absent or erased digest is yielded and not walked.
Source code in packages/strongroom/src/livery/strongroom/_store.py
roots ¶
Every root: what every ref on disk names, and every live view's tree.
Namespaces are read from disk, not from the declaration at open, so a consumer's refs root its objects whoever opened the store. A ref file that is not a digest roots nothing. A view whose directory is gone is retired here rather than rooted.
Source code in packages/strongroom/src/livery/strongroom/_store.py
sweep ¶
sweep(*, scratch_age: float = 86400.0) -> SweepReport
Remove every object no root reaches, and old scratch.
Runs under the maintenance lease. Pending refs are read again immediately before deleting, so a publish that began during the sweep roots its objects. Tombstones are never removed.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
scratch_age
|
float
|
seconds after which an orphaned |
86400.0
|
Returns:
| Type | Description |
|---|---|
SweepReport
|
The report. |
Raises:
| Type | Description |
|---|---|
LockTimeout
|
when another maintenance holds the lease past the timeout. |
Source code in packages/strongroom/src/livery/strongroom/_store.py
view ¶
view(tree: Digest, at: Path, *, writable: bool = False, allow_hardlink: bool = False) -> ViewRecord
Fill at from the tree by the cheapest safe rung per entry.
The ladder is clone, hardlink, link, copy; a rung that refuses once is not tried again for this view. A hardlink is refused into a writable view unless allow_hardlink says the view's consumers never write in place; a link is refused from a writable view. An executable entry never shares an inode or a target's mode. A symlink entry becomes a real symlink where the platform allows, the within-view target's content as a marked copy where it does not, and a parked refusal when the target escapes the view. Blobs are fetched through the sources.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
tree
|
Digest
|
the tree to present; fetched if absent here. |
required |
at
|
Path
|
the directory to fill; missing or empty. |
required |
writable
|
bool
|
whether the caller will write in the view. |
False
|
allow_hardlink
|
bool
|
allow hardlinks into a writable view. |
False
|
Returns:
| Type | Description |
|---|---|
ViewRecord
|
The record, listing every path and its rung. The view is a |
ViewRecord
|
root while its directory exists. |
Raises:
| Type | Description |
|---|---|
FileExistsError
|
when at exists and is not an empty directory. |
MissingObject
|
when a blob is obtainable from no source, naming the entry's path. |
ErasedObject
|
when a blob was erased, naming the entry's path and the tombstone's reason. |
FormatError
|
when a path in the tree is over the budget. |
Source code in packages/strongroom/src/livery/strongroom/_store.py
views ¶
views() -> list[ViewRecord]
view_record ¶
view_record(view_id: str) -> ViewRecord | None
drop_view ¶
drop_view(view_id: str) -> DropReport
Remove what the view's record lists, and nothing else.
A path inside the view that the record does not list is left and named; so is a directory that then stays non-empty, the view's root included. A view whose directory is already gone is retired with nothing removed. The record goes either way.
Raises:
| Type | Description |
|---|---|
FileNotFoundError
|
when no such view is recorded. |
Source code in packages/strongroom/src/livery/strongroom/_store.py
collect ¶
Read the declared paths under at back into a tree, landing every object.
A declared directory is collected whole; an undeclared path is not read. Names must be portable and paths within the budget.
A blob's executable bit comes from a ladder: a path named in executable is executable on every platform, so one call lands one tree everywhere; otherwise the file mode where it carries the bit; where it does not, the record of the view that made the path, then the platform's reading of a new file, the extension on Windows; otherwise false.
A symlink whose target stays inside at is content, an absolute target recorded as the relative path from the link's directory. One that leaves at is the link rung's own symlink into this store, read through as the blob it presents, when the view's record says so or the target lands under this store's root; any other is refused.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
at
|
Path
|
the directory to read. |
required |
declared
|
Iterable[str]
|
the paths to collect, relative and forward-slashed. |
required |
executable
|
Iterable[str]
|
the paths that are executable, spelled the same. |
()
|
Returns:
| Type | Description |
|---|---|
Tree
|
The root tree, landed here with every subtree. |
Raises:
| Type | Description |
|---|---|
FileNotFoundError
|
when a declared path, or a declared executable, is absent. |
FormatError
|
when a name or a symlink target breaks a rule, a symlink escapes at, or a path is over the budget. |
Source code in packages/strongroom/src/livery/strongroom/_store.py
prefetch ¶
Fetch digest and everything it reaches, for offline use.
Returns:
| Type | Description |
|---|---|
list[Digest]
|
Every object fetched, the root first. |
Raises:
| Type | Description |
|---|---|
MissingObject
|
naming the first digest no source answered. |
Source code in packages/strongroom/src/livery/strongroom/_store.py
shed ¶
shed(source: Source) -> ShedReport
Evict every local copy source holds, except what a live view needs.
Holding is checked by existence in a folder, a HEAD on an HTTP source, or the digest an origin hint names; never by trust. The next fetch verifies on arrival, so a lying source costs a named miss and never a wrong byte. Objects a live view reaches through a hardlink or a link are kept.
Source code in packages/strongroom/src/livery/strongroom/_store.py
head ¶
head(source: HttpSource, relative: str) -> bool
Whether the HTTP source answers a HEAD for relative.
Source code in packages/strongroom/src/livery/strongroom/_store.py
ref_path ¶
The file a ref lives at, present or not.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
namespace
|
str
|
a declared namespace. |
required |
path
|
str
|
the ref's path within it, forward-slashed; each component is a portable name and none ends in a reserved suffix. |
required |
Raises:
| Type | Description |
|---|---|
UnknownNamespace
|
when the namespace was not declared. |
FormatError
|
when a path component breaks a rule. |
Source code in packages/strongroom/src/livery/strongroom/_store.py
ref ¶
What the ref names, or None when it does not exist.
Raises:
| Type | Description |
|---|---|
RefTampered
|
when the ref and its record disagree, or the record is missing or malformed. |
Source code in packages/strongroom/src/livery/strongroom/_store.py
record ¶
The record beside the ref, or None when there is none.
Raises:
| Type | Description |
|---|---|
RefTampered
|
when the record's bytes are not a record. |
Source code in packages/strongroom/src/livery/strongroom/_store.py
refs ¶
Every ref path in the namespace, sorted.
Source code in packages/strongroom/src/livery/strongroom/_store.py
set_ref ¶
set_ref(namespace: str, path: str, digest: Digest, *, previous: Digest | None, by: Subject, receipt: Digest | None = None, meta: dict[str, Value] | None = None) -> RefRecord
Move a ref by compare-and-swap, writing its record.
The move happens under the ref's lock. The namespace's mutation class is enforced: write-once refuses a second digest and treats the same digest as done; monotone requires the new target to be a version present here whose parents include the current target; volatile needs only the compare-and-swap.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
namespace
|
str
|
a declared namespace. |
required |
path
|
str
|
the ref's path. |
required |
digest
|
Digest
|
what the ref names afterwards. |
required |
previous
|
Digest | None
|
what the caller believes it names now; None to create. |
required |
by
|
Subject
|
who moves it. |
required |
receipt
|
Digest | None
|
the receipt of the moving call, if any. |
None
|
meta
|
dict[str, Value] | None
|
the consumer's own object for the record. |
None
|
Returns:
| Type | Description |
|---|---|
RefRecord
|
The record written, or the existing record when a |
RefRecord
|
write-once ref already names digest. |
Raises:
| Type | Description |
|---|---|
RefConflict
|
when the ref does not name previous. |
WriteOnceRefused
|
on a write-once ref that names another digest. |
NotFastForward
|
on a monotone ref whose new target does not descend from the current one, or is not a version here. |
LockTimeout
|
when a live lock outlasts the timeout. |
RefTampered
|
when the ref and its record disagree. |
Source code in packages/strongroom/src/livery/strongroom/_store.py
Entry
dataclass
¶
A blob or a subtree inside a tree.
Attributes:
| Name | Type | Description |
|---|---|---|
name |
str
|
the portable name, one path component. |
kind |
EntryKind
|
|
digest |
Digest
|
the object the entry names. |
size |
int
|
the blob's byte count, or the subtree's canonical JSON byte count, so a materialiser plans a view without opening a blob. |
executable |
bool
|
whether a blob is marked executable. Always false for a tree. |
Methods:
| Name | Description |
|---|---|
__post_init__ |
|
to_json |
The entry as a JSON object. |
__post_init__ ¶
Source code in packages/strongroom/src/livery/strongroom/_tree.py
to_json ¶
The entry as a JSON object.
Source code in packages/strongroom/src/livery/strongroom/_tree.py
Link
dataclass
¶
Tree
dataclass
¶
A directory: its entries, sorted by name bytes.
Build one with livery.strongroom.Tree.of, which sorts and validates; decode one with livery.strongroom.Tree.decode, which refuses a tree whose bytes are not already canonical.
Attributes:
| Name | Type | Description |
|---|---|---|
entries |
tuple[Entry | Link, ...]
|
the entries in name-byte order. |
Methods:
| Name | Description |
|---|---|
of |
Build a tree from entries, sorting them and refusing clashes. |
to_json |
The tree as a JSON object. |
from_json |
Decode a tree from its JSON value. |
encode |
The tree's canonical JSON bytes, which its digest names. |
decode |
Decode a tree from its stored bytes. |
digest |
The tree's name: the digest of its canonical JSON. |
of
classmethod
¶
Build a tree from entries, sorting them and refusing clashes.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
entries
|
Iterable[Entry | Link]
|
the directory's entries, in any order. |
required |
Returns:
| Type | Description |
|---|---|
Tree
|
The tree. |
Raises:
| Type | Description |
|---|---|
FormatError
|
when two entries share a name, or two names are equal when case is ignored. |
Source code in packages/strongroom/src/livery/strongroom/_tree.py
to_json ¶
from_json
classmethod
¶
Decode a tree from its JSON value.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
value
|
Value
|
the JSON value. |
required |
Returns:
| Type | Description |
|---|---|
Tree
|
The tree. |
Raises:
| Type | Description |
|---|---|
FormatError
|
when the shape is wrong, an entry breaks a rule, or the entries are not in name-byte order. |
Source code in packages/strongroom/src/livery/strongroom/_tree.py
decode
classmethod
¶
Decode a tree from its stored bytes.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data
|
bytes
|
the object's bytes. |
required |
Returns:
| Type | Description |
|---|---|
Tree
|
The tree. |
Raises:
| Type | Description |
|---|---|
FormatError
|
when the bytes are not JSON, not a tree, or not the canonical encoding of the tree they describe. A non-canonical tree would have a second name for the same content. |
Source code in packages/strongroom/src/livery/strongroom/_tree.py
Version
dataclass
¶
Version(tree: Digest, parents: tuple[Digest, ...], producer: Subject, created: str, message: str, receipt: Digest | None = None, attachments: Mapping[str, Digest] = dict())
Provenance over a tree.
Attributes:
| Name | Type | Description |
|---|---|---|
tree |
Digest
|
the content. |
parents |
tuple[Digest, ...]
|
the versions this one continues, in order. |
producer |
Subject
|
who or what produced it. |
created |
str
|
when, as an RFC 3339 UTC instant. |
message |
str
|
free text. |
receipt |
Digest | None
|
the receipt of the producing call, or None. |
attachments |
Mapping[str, Digest]
|
name to blob digest, for a consumer's own metadata. Names obey the portable-name rules. |
Methods:
| Name | Description |
|---|---|
__post_init__ |
|
to_json |
The version as a JSON object. |
from_json |
Decode a version from its JSON value. |
encode |
The version's canonical JSON bytes, which its digest names. |
decode |
Decode a version from its stored bytes. |
digest |
The version's name: the digest of its canonical JSON. |
attachments
class-attribute
instance-attribute
¶
__post_init__ ¶
to_json ¶
The version as a JSON object.
Source code in packages/strongroom/src/livery/strongroom/_version.py
from_json
classmethod
¶
Decode a version from its JSON value.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
value
|
Value
|
the JSON value. |
required |
Returns:
| Type | Description |
|---|---|
Version
|
The version. |
Raises:
| Type | Description |
|---|---|
FormatError
|
when the shape or a field is wrong. |
Source code in packages/strongroom/src/livery/strongroom/_version.py
decode
classmethod
¶
Decode a version from its stored bytes.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data
|
bytes
|
the object's bytes. |
required |
Returns:
| Type | Description |
|---|---|
Version
|
The version. |
Raises:
| Type | Description |
|---|---|
FormatError
|
when the bytes are not JSON, not a version, or not canonical. |
Source code in packages/strongroom/src/livery/strongroom/_version.py
DropReport
dataclass
¶
ShedReport
dataclass
¶
ViewEntry
dataclass
¶
ViewEntry(path: str, rung: EntryRung, digest: Digest | None = None, note: str = '', executable: bool = False)
One path a view created, and how.
Attributes:
| Name | Type | Description |
|---|---|---|
path |
str
|
relative to the view's root, forward-slashed. |
rung |
EntryRung
|
how it was made; |
digest |
Digest | None
|
the blob it presents, or None for a symlink entry. |
note |
str
|
why a symlink became a copy or was parked; empty otherwise. |
executable |
bool
|
whether the tree marked the blob executable; what
|
Methods:
| Name | Description |
|---|---|
to_json |
The entry as a JSON object. |
from_json |
Decode an entry, refusing a rung outside the ladder. |
to_json ¶
The entry as a JSON object.
Source code in packages/strongroom/src/livery/strongroom/_views.py
from_json
classmethod
¶
Decode an entry, refusing a rung outside the ladder.
Source code in packages/strongroom/src/livery/strongroom/_views.py
ViewRecord
dataclass
¶
ViewRecord(id: str, tree: Digest, at: str, writable: bool, created: str, entries: tuple[ViewEntry, ...] = tuple(), directories: tuple[str, ...] = tuple())
A view: local state in the index, and a root while its directory exists.
Attributes:
| Name | Type | Description |
|---|---|---|
id |
str
|
the record's name under |
tree |
Digest
|
the tree the view presents. |
at |
str
|
the view's root directory. |
writable |
bool
|
whether the caller said it would write. |
created |
str
|
when, an instant. |
entries |
tuple[ViewEntry, ...]
|
every path created, in creation order. |
directories |
tuple[str, ...]
|
every directory created, shallowest first. |
Methods:
| Name | Description |
|---|---|
encode |
The record's canonical JSON bytes. |
decode |
Decode a record. |
depends_on |
The objects this view reads through the store's own files. |
entries
class-attribute
instance-attribute
¶
directories
class-attribute
instance-attribute
¶
encode ¶
encode() -> bytes
The record's canonical JSON bytes.
Source code in packages/strongroom/src/livery/strongroom/_views.py
decode
classmethod
¶
decode(data: bytes) -> ViewRecord
Decode a record.
Raises:
| Type | Description |
|---|---|
FormatError
|
when the bytes are not a record. |
Source code in packages/strongroom/src/livery/strongroom/_views.py
depends_on ¶
The objects this view reads through the store's own files.
Source code in packages/strongroom/src/livery/strongroom/_views.py
canonical ¶
Encode value as RFC 8785 canonical JSON, UTF-8.
Object keys sort by their UTF-16 code units, which differs from code-point order above the Basic Multilingual Plane. Strings keep every character above U+001F literal.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
value
|
Value
|
the value to encode. Nested values are encoded the same way. |
required |
Returns:
| Type | Description |
|---|---|
bytes
|
The canonical bytes. Two equal values encode to equal bytes. |
Raises:
| Type | Description |
|---|---|
FormatError
|
on a float, an integer past 2^53 in magnitude, a non-string key, or a value of a type no format carries. |
Source code in packages/strongroom/src/livery/strongroom/_canonical.py
digest_of ¶
Name data under algorithm.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data
|
bytes
|
the bytes to name. |
required |
algorithm
|
str
|
a registry entry's name; |
name
|
Returns:
| Type | Description |
|---|---|
Digest
|
The digest. |
Raises:
| Type | Description |
|---|---|
FormatError
|
when the algorithm is not registered. |
Source code in packages/strongroom/src/livery/strongroom/_digest.py
digest_stream ¶
Name the bytes read from stream to its end, without holding them.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
stream
|
IO[bytes]
|
an open binary stream, read in chunks until empty. |
required |
algorithm
|
str
|
a registry entry's name; |
name
|
Returns:
| Type | Description |
|---|---|
Digest
|
The digest. |
Raises:
| Type | Description |
|---|---|
FormatError
|
when the algorithm is not registered. |
Source code in packages/strongroom/src/livery/strongroom/_digest.py
check_timestamp ¶
Return value when it is an RFC 3339 UTC instant, or refuse.
The form is YYYY-MM-DDTHH:MM:SS[.fraction]Z: seconds required,
up to nine fractional digits, the Z designator and no offset, so
one instant has one spelling.
Source code in packages/strongroom/src/livery/strongroom/_fields.py
fetch_url ¶
fetch_url(url: str, *, connect_timeout: float, transfer_timeout: float, method: str = 'GET', headers: Mapping[str, str] | None = None) -> Generator[HTTPResponse]
Open url for reading, or raise Unreachable naming why.
The one seam every network read goes through, so tests fake it and
a fetch never needs a server. The connect timeout bounds
establishing the connection; the transfer timeout then bounds
each wait for bytes, headers included. method is GET, or
HEAD to learn whether a source holds an object without reading it.
headers are sent with the request to url's host and port, and
to a redirect on that same host and port; a redirect anywhere else
gets none of them, so a credential meant for an API never reaches
the storage host the API redirects a download to.
A redirect (301, 302, 303, 307, 308) is followed to its Location,
resolved against the URL that answered it, with the same method
and both timeouts per hop, up to REDIRECTS hops: a forge serves
a release asset from a storage host it redirects to, and the bytes
are the same bytes. A chain longer than that, or a redirect naming
no location, is refused naming the chain.
Yields:
| Type | Description |
|---|---|
Generator[HTTPResponse]
|
The response, status 200, positioned at its first byte. |
Raises:
| Type | Description |
|---|---|
Unreachable
|
on a connection error, a timeout, a redirect chain that does not end, or any final status other than 200. A 404 is the source saying it does not have the object, which for the fetch is the same thing. |
Source code in packages/strongroom/src/livery/strongroom/_sources.py
155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 | |
check_name ¶
Return name when it is portable, or refuse it.
The rules: non-empty, not . or .., Unicode NFC, at most
livery.strongroom.NAME_BUDGET UTF-8 bytes, no separator, no
control character, none of <>:"|?*, no trailing dot or space,
and not a Windows-reserved stem such as CON or LPT1 in any
case, with or without an extension.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
name
|
str
|
one path component. |
required |
Returns:
| Type | Description |
|---|---|
str
|
The name, unchanged. |
Raises:
| Type | Description |
|---|---|
FormatError
|
naming the rule the name breaks. |
Source code in packages/strongroom/src/livery/strongroom/_tree.py
check_target ¶
Return a symlink target when it is portable, or refuse it.
A target is content: UTF-8 in NFC, forward slashes only, relative always. Whether it may escape its view is the view's policy.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
target
|
str
|
the link's target string. |
required |
Returns:
| Type | Description |
|---|---|
str
|
The target, unchanged. |
Raises:
| Type | Description |
|---|---|
FormatError
|
naming the rule the target breaks. |
Source code in packages/strongroom/src/livery/strongroom/_tree.py
__getattr__ ¶
Serve a public name from its module on first use, then keep it.