Skip to content

[trunk]PXC-5203: Cluster disruption or deadlock with LOCK TABLES/INSTANCE FOR BACKUP misuse - #2368

Merged
jaideepkarande merged 4 commits into
percona:trunkfrom
jaideepkarande:PXC-5203-trunk
Oct 14, 2026
Merged

jaideepkarande merged 4 commits into
percona:trunkfrom
jaideepkarande:PXC-5203-trunk

Conversation

@jaideepkarande

@jaideepkarande jaideepkarande commented Sep 14, 2026 •

Copy link
Copy Markdown
Contributor

Cluster disruption or deadlock with LOCK TABLES/INSTANCE FOR BACKUP misuse

https://perconadev.atlassian.net/browse/PXC-5203

Propagation of the 8.4 fix (#2318) to the trunk line, via the 9.7 line (#2367).

Problem

A session that ran its own DDL/DML while holding LOCK TABLES FOR BACKUP was
replicated via TOI before the local backup-lock check rejected it, so peers
applied the DDL while the originator failed it locally — causing a Galera
"inconsistency detected" vote and node eviction.

Resolution

The same issue was already fixed for LOCK INSTANCE FOR BACKUP; this extends that
condition to LOCK TABLES FOR BACKUP.

Testing

Jenkins: https://pxc.cd.percona.com/job/pxc-8.x-pipeline-parallel-mtr-AI/68/

Compatibility & upgrade impact

Dimension Impact Notes
In-place upgrade (on-disk / DD / redo-undo / system tables) No No format change; sql/wsrep_mysqld.cc logic only.
Downgrade No Nothing persisted differs.
Rolling / mixed-version cluster Yes — behavioural A statement that previously replicated is now rejected locally. An un-patched node in a mixed cluster still exhibits the old behaviour, so the inconsistency vote remains possible until all nodes carry the fix.
Replication (binlog format, GTID, cross-version) No No binlog or GTID change; the statement is rejected before TOI replication.
Client-server wire protocol No No protocol change.
System variables / config / defaults No No new or changed sysvar.
Cross-version behaviour Consistent Same change applies cleanly on 8.4, 9.7 and trunk.

PXC-5223 Packaging tasks for release - PXC 9.7.1
@jaideepkarande
jaideepkarande marked this pull request as ready for review September 14, 2026 19:08
@jaideepkarande

jaideepkarande commented Sep 15, 2026 •

Copy link
Copy Markdown
Contributor Author

Jenkins results — trunk

AI job build 7 · Debug · ubuntu:noble · commit b0a5d0ce23c
PXB: 9.7.1-rc1 → pxb-9.7, 8.4.0-6 → pxb-8.4, 9.1.0 → pxb-9.1
Result: ABORTED (partial) — 9938 passed, 5 failed, 4899 skipped.

This change's test passed

test enc_off enc_on
galera.pxc_backup_lock_toi_reject pass (282ms) pass (306ms)

It runs on the galera-small worker, which completed normally, so this verdict is
sound despite the overall build being aborted.

Caveat — the run is incomplete

Stage Test - 5 (galera-big) was ABORTED ~69 min in; the other seven stages
finished. So roughly 2200 fewer tests ran than in a full pass (14842 total vs ~17080).
I could not find an abort reason in the console — no Aborted by … line and no
timeout message — so this looks like an infrastructure drop rather than a test
failure. Nothing in the aborted worker relates to this change.

If you want a complete trunk run for the record, re-triggering is cheap — say the word.

Failures — none related to this change

test assessment
galera_fk_selfreferential (enc_off, enc_on) pre-existing — also fails on the clean trunk baseline (build 902) and on unrelated branches (builds 5, 10)

shutdown_report entries are MTR's end-of-worker report, not real tests.

@jaideepkarande

Copy link
Copy Markdown
Contributor Author

…R BACKUP misuse

https://perconadev.atlassian.net/browse/PXC-5203

A session that ran its own DDL/DML while holding LOCK TABLES FOR BACKUP
was replicated via TOI before the local backup-lock check rejected it,
so peers applied the DDL while the originator failed it locally -- causing
a Galera "inconsistency detected" vote and node eviction.

Resolution:
Similar issue was already fixed for "LOCK INSTANCE FOR BACKUP" extended the
condition for "LOCK TABLES FOR BACKUP"
@jaideepkarande
jaideepkarande merged commit 70515e1 into percona:trunk Oct 14, 2026
15 of 17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants