Repository navigation
An interrupted graph rebuild leaves a full-size candidate file that nothing ever removes #205
Description
mex graph rebuild builds into graph.db.candidate-<token> and publishes it by
atomic rename. If the process dies partway — Ctrl+C, a CI timeout, a crash — the
candidate stays on disk permanently. No later command removes it, and no command
mentions it.
To reproduce
Kill a running mex graph rebuild, then run each maintenance command in turn:
$ ls .mex/
graph.db 6,774,784
graph.db.candidate-ea0c83460fd19a3425f426cd7a32ec… 6,774,784
graph.db.lock 112
| after | candidate present? | notes |
|---|---|---|
mex graph status |
yes | reports fresh, says nothing about it |
mex graph repair |
yes | |
mex graph refresh (successful) |
yes | |
mex graph rebuild (full, successful) |
yes | the stale .lock is cleared; only the candidate leaks |
The candidate is a byte-for-byte duplicate of the store, so every interruption
costs one full store on disk. It is invisible unless someone lists .mex/.
Cause
src/graph/maintenance.ts calls cleanupOwnedDatabase(paths, candidatePath)
only from within the run that created the candidate — its catch/finally paths.
A killed process never reaches that code, and there is no sweep: the file
contains no readdirSync or globSync of the graph directory at all, so no
later run can discover a previous run's leftovers.
The same applies to the other two owned prefixes in OWNED_DATABASE_PREFIXES —
graph.db.rollback- and graph.db.recovery-. Planting one of each confirms
both survive status, repair, refresh and a successful rebuild.
Impact
One full copy of the store leaked silently per interruption. On the 240 MB store
in #140 that is 240 MB each time, and that report describes builds being stopped
by a 30-minute timeout — exactly this path.
Note for anyone triaging this alongside #140
On 0.8.x an interrupted rebuild does not strand a WAL; it strands a
candidate plus a lock. The two failure modes look alike and are not:
stranded graph.db-wal |
orphan graph.db.candidate-* |
|
|---|---|---|
| what's wrong | status misreports a healthy store | a full-size copy stays on disk |
| cost | confusion | disk |
| fix today | mex graph repair |
delete it by hand |
| needs | wording | cleanup + a warning |
The wording half is filed separately as #204.
Suggested fix
At the start of a maintenance run, remove OWNED_DATABASE_PREFIXES files that
no live lock covers, and have graph status warn while one is present.
The lock question is already solved, which keeps this small: the maintenance
lease is per-project and exclusive, and acquireGraphMaintenanceLease already
reclaims a dead lock (readLockOwner / beforeDeadLockReclaim, around
src/graph/maintenance.ts:1537). Sweeping after the lease is held means no
concurrent run can own the files being removed, so the sweep needs no liveness
logic of its own — doing it before acquiring the lease would.
Activity
dyk1454683243-sudo commentedon Sep 19, 2026
Taking this: after
acquireGraphMaintenanceLeasesucceeds, sweep.mex/for orphanOWNED_DATABASE_PREFIXESfiles (graph.db.candidate-*,graph.db.rollback-*,graph.db.recovery-*) that are not the live in-progress candidate for that run, then havemex graph statusemit a read-only warning while one is present. No sweep before the lease, and no WAL wording work (that stays #204).dyk1454683243-sudo commentedon Sep 19, 2026
Draft PR is up at #214 — post-lease owned-prefix sweep plus a read-only
graph statuswarning. Independent of #204.fix(graph): sweep orphan build candidates after an interrupted mainte…