Quirks¶
A quirk without a fake fault is a debt. Every forge quirk discovered
gets a deterministic, millisecond reproduction and a regression test
the same day it is understood, plus its line here. A quirk that leaks
through the protocol surface gets a livery.forge.testing.Faults
fault mode (tests in packages/forge/tests/test_fake_faults.py); a
quirk a backend absorbs at its own boundary reproduces through that
backend's recorded cassettes, which replay the server's refusal
verbatim on every merge.
| Date | Forge | Quirk | Reproduction | Regression test |
|---|---|---|---|---|
| 2026-08-31 | Gitea | An armed auto-merge schedule can be silently lost: the arm is accepted and nothing is recorded, so the pull request later reads unarmed | fault mode Faults.lose_arm_schedule |
test_a_lost_arm_schedule_reads_unarmed_and_rearming_recovers |
| 2026-08-31 | Gitea | Merge answers 405 while the checks are not green or a mergeability recompute is in flight, on a pull request that looks mergeable; the arm shares the merge endpoint and is refused the same way, most often right after a force push; and a finished run's required context can lag the run by a beat, refusing "not all required status checks successful" seconds after the run reads green | fault mode Faults.merge_405_window; classified by livery.forge.classify_merge_refusal into the merge-hold vocabulary |
test_the_405_window_refuses_merges_then_passes, test_the_405_window_refuses_arms_the_same_way, test_merge_state |
| 2026-08-31 | Gitea | The Actions status queue can wedge on Gitea before 1.27.3 (fixed upstream there): jobs finish and their results are never applied, so the combined status stays pending forever; cancelling the run is the relief. The fault mode stays, it models any hung run | fault mode Faults.wedge_status_queue |
test_a_wedged_status_queue_holds_pending_until_cancel_relieves_it |
| 2026-08-31 | Gitea | A freshly pushed commit reads as having no statuses for a while, so "nothing reported" must mean "keep waiting", never "no CI configured" | fault mode Faults.slow_status_reads |
test_slow_status_reads_answer_none_before_the_truth |
| 2026-09-11 | GitHub | A poll can end with the server closing the connection without a response, 21 minutes into a follow; http.client raises its own RemoteDisconnected, not a URLError, so the client folds every transport error into livery.forge.ForgeError with no status, and a watcher retries through a bounded run of them |
fault mode Faults.drop_connections |
test_dropped_connections_raise_then_the_next_read_answers |
| 2026-08-31 | GitLab | Project deletion is asynchronous: the project is marked for deletion, its path stays occupied, a second DELETE answers 400 "already marked for deletion", the old path answers 405 "moved" for non-GET methods, and a GET on the old path serves the renamed corpse with 200. delete_repo absorbs both refusals as success and get_repo verifies path_with_namespace before answering |
gitlab cassettes | test_gitlab_conformance[repo-lifecycle] |
| 2026-08-31 | GitLab | Merge-when-pipeline-succeeds answers 405 until two asynchronous facts land: the mergeability recompute and the head pipeline's association with the merge request | gitlab cassettes | test_gitlab_conformance[arm-disarm], [armed-pr-merges-on-green] |
| 2026-08-31 | GitLab | The project's delete-branch-on-merge setting only pre-fills the UI checkbox; an API merge deletes the source branch only when the merge call passes should_remove_source_branch, and the deletion lands about a second after the merge |
gitlab cassettes | test_gitlab_conformance[pr-merge-now] |
| 2026-08-31 | GitLab | Pipeline cancellation is asynchronous: a pipeline reads canceling until its jobs acknowledge, and cancelling an already finished pipeline answers 200, so the backend probes and raises the terminal refusal itself |
gitlab cassettes | test_gitlab_conformance[cancel-run] |
| 2026-08-31 | GitHub | Merging an already merged pull request answers 200 "successfully merged" where Gitea and GitLab answer 405. The protocol normalises to GitHub's shape: merge_now on a merged pull request is success everywhere, the other backends absorbing their refusal after verifying the merge |
all three cassette sets | [pr-merge-now] on every backend |
| 2026-08-31 | GitHub | A push workflow triggers for tag pushes too, and GITHUB_TOKEN is an expression, not an ambient variable: a run step's shell sees it only when the step exports it in env:. Both cost a held job its release signal |
driver workflow (branches filter, env: block) |
test_github_conformance[armed-pr-merges-on-green] |
| 2026-08-31 | GitHub | Workflow-run creation for a push can be silently dropped under load, and issue listings run about nine seconds behind writes | driver verified pushes and await_issue |
test_github_conformance (whole set) |
| 2026-08-31 | GitLab | Current gitlab-ce refuses a merge whose request carries no sha ("SHA must be provided when merging"), where older servers accepted it. merge_now and arm read the merge request first and pin its head sha in the PUT, which also means the merge takes exactly the head that was read, never a racing push |
gitlab cassettes (the PUT bodies carry sha; replay pins the shape) |
test_gitlab_conformance[pr-merge-now], [arm-disarm] |
| 2026-08-31 | GitHub | A freshly committed workflow file is indexed asynchronously: a workflow_dispatch straight after the seed answers 404 until the file appears in the workflows listing |
driver polls the listing before returning a fresh repository (plain live runs) | test_github_conformance[dispatch] (live legs) |
| 2026-08-31 | GitHub | The cancel endpoints intermittently answer 5xx while a run is mid-transition | cancel_run retries 5xx briefly, the last answer verbatim |
test_github_conformance[cancel-run] (live legs) |
| 2026-08-31 | gitea.com | Repository creation answers 403 "maximum limit of repositories [limit: -1]" once an account holds too many repositories, so a burst of scratch creations trips it mid-suite even though each single create succeeds | live runs delete scratch at each scenario's end (driver.cleanup) |
release-legs gitea-cloud |
| 2026-08-31 | Gitea (nightly) | A 1.28-dev nightly build after the validated pin stops listing completed action tasks, so "runs listed and all completed" can never both hold and every settle wait times out; tags, jobs, and the rest behave | the compose file pins the server and runner images by validated digest | release-legs gitea-local (run 33411879570: release tags present, task listing empty) |
| 2026-09-01 | Gitea | git-cliff's api_url for a Gitea remote is the server root, not an API path: it appends /api/v1 itself, so a configured http://host/api/v1 becomes http://host/api/v1/api/v1/... and the server answers 404 |
the rendered cliff.toml names the server root; verified against the local container, private repository, authenticated |
test_a_refused_author_lookup_says_what_to_check |
| 2026-09-01 | Gitea | A remote lookup git-cliff cannot complete (a private repository without a token, a wrong api_url) aborts the process (exit 101, a Rust panic), rather than writing the entry without its authors | the workshop passes --offline when no credential is in reach, so a private forge still releases |
test_the_changelog_runs_offline_when_no_credential_is_in_reach |
| 2026-09-01 | GitHub | The timeline names the auto-merge enable event after the merge method: a squash arm records auto_squash_enabled (rebase auto_rebase_enabled), and plain auto_merge_enabled never appears for them; disabling is method-blind (auto_merge_disabled) |
github cassette (governance-reads.json) |
test_github_conformance[governance-reads] |
| 2026-09-01 | Gitea | The addresses a server puts in its own payloads come from its configured ROOT_URL, which need not be the host the API was reached on: one repository answers with http://gitea:3000/... links while the client is talking to http://localhost:3000. So a built address matches a reported one by path, never by host, and the address family promises the path shape alone |
gitea cassette (addresses.json carries both hosts) |
test_gitea_conformance[addresses] |
| 2026-09-02 | GitHub | A deleted branch's read answers for seconds afterwards on github.com: the delete is accepted and the branch endpoint keeps serving it, so an immediate existence check lies | driver-side await_branch poll |
test_github_conformance[branches] |
| 2026-09-15 | GitLab | A new project's default branch is protected on creation, by the instance or group default (maintainers push, no force push), so a creator's first force push over the init commit is refused by the pre-receive hook. the rule arrives from a background job about a second after the create returns, so create_repo waits for it and removes it, and the protocol's promise of an open default branch is pinned on every backend |
gitlab cassette (repo-lifecycle.json carries the DELETE .../protected_branches/main) |
test_gitlab_conformance[repo-lifecycle], and the same scenario on gitea, github and the fake |
| 2026-09-15 | GitHub | Reading a private repository's branch protection on a plan without the feature answers 403 ("Upgrade to GitHub Pro or make this repository public"), where a public repository without protection answers 404; the branch is open in both cases, so both read as no protection | github cassette (repo-lifecycle.json carries the 403) |
test_github_conformance[repo-lifecycle] |
| 2026-09-15 | GitLab | The PyPI registry's simple index links each file at the instance's external_url, whatever host the index was read on: a runner that reads http://gitlab:8929/.../simple/ is sent to http://localhost:8929/api/v4/projects/:id/packages/pypi/files/..., which inside its container is itself. The local instance's external_url is the compose service name, as Gitea's ROOT_URL is |
the dev compose's GITLAB_OMNIBUS_CONFIG |
the loop's setup gate installs the dev wheels from the registry: fm ci.e2e --forge=gitlab |
| 2026-09-15 | GitLab | For a beat after a merge request's pipeline succeeds, the merge answers 422 Branch cannot be merged while detailed_merge_status is still checking, and reads mergeable seconds later; the refusal is the recompute's window, not a verdict. classify_merge_refusal reads the hold through merge_hold and follows checking and mergeable-under-refusal as computing |
classify_merge_refusal("gitlab", ..., hold="checking") |
test_a_refusal_on_gitlab_or_github_classifies_the_published_hold |
| 2026-09-15 | GitLab | A merge request reads state: locked while the merge runs, between merge_when_pipeline_succeeds firing and merged; a caller mapping every state but opened to closed reads the beat as closed unmerged and stops following. locked reads as open |
crafted cassette in test_error_paths.py |
test_a_locked_merge_request_is_still_open |
| 2026-10-11 | Gitea | A run's, a job's or a step's time the server has not set is served as the Unix epoch in the server's offset (1970-01-01T01:00:00+01:00), never as null: a running job's end, a queued job's start. Read as a time it measures the job at minus the current date. The adapter reads the epoch, and Go's zero time, as unset, which the protocol promises as the empty string |
crafted cassette in test_error_paths.py |
test_gitea_reads_an_epoch_stamp_as_a_time_it_has_not_set |
| 2026-10-18 | GitHub | enablePullRequestAutoMerge refuses a commit headline over 320 UTF-8 bytes with "Commit title is too long (maximum is 80 characters)" and a commit body over 65535 bytes with "Commit message is too long (maximum is 16383 characters)": each message names the byte limit divided by four. Measured live: 320 bytes of ASCII, of two-byte or of four-byte characters arm, 321 bytes refuse; the arm with no headline takes a 480-character title. arm cuts a longer headline at the last word boundary within 320 bytes and refuses a longer body before any call, naming the limit and the body's length |
github cassette (arm-long-title.json carries the cut headline); crafted cassettes pin the GraphQL request |
test_github_conformance[arm-long-title], test_github_auto_merge.py |
| 2026-10-18 | GitHub | merge_commit_sha names a test merge commit while a pull request is open, once GitHub has computed its mergeability, in the single read and the listing row alike. Read as the merge, it reports an open pull request as merged under that commit, to a caller that follows merged_sha. The backend reports merged_sha only once merged, or a listing row's merged_at, says the pull request has merged |
github cassette (pr-list.json: the open pull request's rows carry the test merge commit) |
test_github_conformance[pr-list] |