livery.strongroom._sources¶
livery.strongroom._sources ¶
Sources and tiers: where a store looks for an object it does not have.
A store instance is its local objects directory plus an ordered list
of sources. A source is a folder in the same layout, an HTTP base URL
serving the layout, or an origin hint: one URL that yields one named
object. Every source is verified and none is trusted: a hit is hashed
on arrival, an unreachable source is skipped after a short connect
timeout and reported, and offline means never an origin.
Reach for livery.strongroom.FolderSource, livery.strongroom.HttpSource and livery.strongroom.OriginHint to declare sources at livery.strongroom.Store.open, and livery.strongroom.Store.fetch to consult them.
Classes:
| Name | Description |
|---|---|
FolderSource |
A folder in the store layout: a mirror, or a read-only share. |
HttpSource |
An HTTP base URL serving the store layout: nginx over a folder. |
OriginHint |
One URL that yields one named object: the ingress form. |
Unreachable |
A source did not answer for this object; the fetch skips it and reports. |
Functions:
| Name | Description |
|---|---|
fetch_url |
Open url for reading, or raise Unreachable naming why. |
Attributes:
| Name | Type | Description |
|---|---|---|
FillPolicy |
What a folder hit does: land a local copy, or answer into the folder. |
|
Progress |
Where a fetch reports a skipped or refused source; never prints on its own. |
|
Source |
Anything a store may consult for an object. |
|
REDIRECTS |
How many redirects a fetch follows before it refuses the chain. |
FillPolicy
module-attribute
¶
FillPolicy = Literal['copy', 'reference']
What a folder hit does: land a local copy, or answer into the folder.
Progress
module-attribute
¶
Where a fetch reports a skipped or refused source; never prints on its own.
Source
module-attribute
¶
Source = FolderSource | HttpSource | OriginHint
Anything a store may consult for an object.
REDIRECTS
module-attribute
¶
How many redirects a fetch follows before it refuses the chain.
FolderSource
dataclass
¶
FolderSource(path: Path, fill: FillPolicy = 'copy')
A folder in the store layout: a mirror, or a read-only share.
Attributes:
| Name | Type | Description |
|---|---|---|
path |
Path
|
the folder's root, holding a manifest of the same layout and algorithm as the store that consults it. |
fill |
FillPolicy
|
|
Methods:
| Name | Description |
|---|---|
__post_init__ |
|
describe |
The source, for a report. |
__post_init__ ¶
HttpSource
dataclass
¶
An HTTP base URL serving the store layout: nginx over a folder.
Attributes:
| Name | Type | Description |
|---|---|---|
base_url |
str
|
the URL the layout hangs from; a trailing slash is ignored. |
connect_timeout |
float
|
seconds to wait for the server to answer at all; an unanswered source is skipped and reported. |
transfer_timeout |
float
|
seconds a stalled transfer may wait between bytes. |
Methods:
| Name | Description |
|---|---|
__post_init__ |
|
describe |
The source, for a report. |
url |
The URL of relative, a path in the layout. |
OriginHint
dataclass
¶
One URL that yields one named object: the ingress form.
A tool spec's {url, sha256} per host is an origin hint, and so is
a vendor download. Consulted only for its own digest, and never
when the store is offline.
Attributes:
| Name | Type | Description |
|---|---|---|
digest |
Digest
|
the object the URL yields. |
url |
str
|
where to get it. |
connect_timeout |
float
|
seconds to wait for an answer. |
transfer_timeout |
float
|
seconds a stalled transfer may wait. |
Methods:
| Name | Description |
|---|---|
__post_init__ |
|
describe |
The source, for a report. |
__post_init__ ¶
Unreachable ¶
Bases: Exception
A source did not answer for this object; the fetch skips it and reports.
Attributes:
| Name | Type | Description |
|---|---|---|
status |
The HTTP status the source answered with, when the refusal is an answer (a 404, a 503) rather than a connection that failed or a chain that did not end; None otherwise. A caller deciding whether to try again reads it: a connection failure and a 5xx are worth another try, a 404 is not. |
Source code in packages/strongroom/src/livery/strongroom/_sources.py
fetch_url ¶
fetch_url(url: str, *, connect_timeout: float, transfer_timeout: float, method: str = 'GET', headers: Mapping[str, str] | None = None) -> Generator[HTTPResponse]
Open url for reading, or raise Unreachable naming why.
The one seam every network read goes through, so tests fake it and
a fetch never needs a server. The connect timeout bounds
establishing the connection; the transfer timeout then bounds
each wait for bytes, headers included. method is GET, or
HEAD to learn whether a source holds an object without reading it.
headers are sent with the request to url's host and port, and
to a redirect on that same host and port; a redirect anywhere else
gets none of them, so a credential meant for an API never reaches
the storage host the API redirects a download to.
A redirect (301, 302, 303, 307, 308) is followed to its Location,
resolved against the URL that answered it, with the same method
and both timeouts per hop, up to REDIRECTS hops: a forge serves
a release asset from a storage host it redirects to, and the bytes
are the same bytes. A chain longer than that, or a redirect naming
no location, is refused naming the chain.
Yields:
| Type | Description |
|---|---|
Generator[HTTPResponse]
|
The response, status 200, positioned at its first byte. |
Raises:
| Type | Description |
|---|---|
Unreachable
|
on a connection error, a timeout, a redirect chain that does not end, or any final status other than 200. A 404 is the source saying it does not have the object, which for the fetch is the same thing. |
Source code in packages/strongroom/src/livery/strongroom/_sources.py
155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 | |