fix(§22/registry): prune projects absent from the registry on reconcile (v0.55.0)
The registry mirror was additive-only — it upserted the projects/collections a deployment's projects.yaml declares but never removed ones it had dropped. Re-pinning to a different registry stranded the old project's collections + its cached entries as dead rows that, at scale, starved writes (PPE accumulated ~1.2k orphaned entry rows and had to be reset by hand). `registry.refresh_registry(prune=True)` now calls `projects.prune_absent_projects`, deleting a removed project with its collections and every project-scoped row (the 7 project_id-keyed tables + 13 collection_id-keyed entry-corpus tables + the non-keyed thread_messages descendant) in one FK-off transaction with a `PRAGMA foreign_key_check` backstop that rolls back rather than leave a dangling ref — mirroring restamp_default_project / reconcile_default_collection_id. Scoped for safety: whole-project granularity (a present project is untouched); the default project is never pruned; prune runs ONLY on the full-reconcile triggers (startup + the periodic sweep), never on incidental refreshes (collection create, webhook), so an in-app refresh can't wipe a project. Reached only after a successful parse of a non-empty registry (parse_registry rejects empty; the read raises on transport error), so a transient read can't trigger a wipe. Second of the two §9-surfaced framework fragilities. No migration. backend 685 green (+5: prune removes-all/no-op/never-default/empty-rejected/incidental-no-prune with FK-integrity backstop). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
+23
-2
@@ -333,11 +333,18 @@ async def _mirror_named_collections(config: Config, gitea: Gitea, doc: RegistryD
|
||||
_upsert_named_collection(proj, subdir, ce, sha)
|
||||
|
||||
|
||||
async def refresh_registry(config: Config, gitea: Gitea) -> None:
|
||||
async def refresh_registry(config: Config, gitea: Gitea, *, prune: bool = False) -> None:
|
||||
"""Mirror REGISTRY_REPO/projects.yaml into projects + deployment.
|
||||
|
||||
Idempotent. Raises RegistryError on a missing/invalid file and GiteaError
|
||||
on transport failure; the caller chooses fatal-vs-tolerated.
|
||||
|
||||
`prune=True` additionally removes projects the registry no longer declares
|
||||
(and their collections + entries) — see `projects.prune_absent_projects`.
|
||||
It is OFF by default and enabled only on the full-reconcile triggers
|
||||
(startup + the periodic sweep), never on incidental refreshes (a collection
|
||||
create, a webhook) that don't change the project set — keeping the
|
||||
destructive sweep on the few code paths that mean "reconcile to the registry."
|
||||
"""
|
||||
item = await gitea.get_contents(
|
||||
config.gitea_org, config.registry_repo, "projects.yaml", ref="main"
|
||||
@@ -355,4 +362,18 @@ async def refresh_registry(config: Config, gitea: Gitea) -> None:
|
||||
apply_registry(doc, sha, projects_mod.resolved_default_id(config))
|
||||
# §22 S2: discover + upsert named collections from each content repo.
|
||||
await _mirror_named_collections(config, gitea, doc, sha)
|
||||
log.info("registry: mirrored %d project(s) at %s", len(doc.projects), sha)
|
||||
# Prune projects the registry no longer declares (additive-only was a §9
|
||||
# fragility — a re-pin to a new registry stranded the old projects' entries,
|
||||
# starving writes). Safe here: we only reach this line after a successful
|
||||
# parse of a non-empty registry (parse_registry rejects an empty project list
|
||||
# and the read above raises on transport failure), so a transient failure
|
||||
# never prunes. Whole-project granularity; the default project is never pruned.
|
||||
pruned = (
|
||||
projects_mod.prune_absent_projects(config, {e.id for e in doc.projects})
|
||||
if prune else []
|
||||
)
|
||||
log.info(
|
||||
"registry: mirrored %d project(s) at %s%s",
|
||||
len(doc.projects), sha,
|
||||
f"; pruned {len(pruned)} absent: {pruned}" if pruned else "",
|
||||
)
|
||||
|
||||
Reference in New Issue
Block a user