Repository navigation
Fix/build fee bump multi transaction - #605
henrypeters wants to merge 2 commits into
Conversation
88e678c to
4f5c40b
Compare
fametrano
left a comment
There was a problem hiding this comment.
With fee_rate, the merged replacement can pay less than the sum of the fees it replaces, and Core rejects it. On that path create_tx checks only the highest original feerate plus 1 sat/vB (src/wallet/mod.rs:1447-1458), not the sum. Passing a parent and its child together builds a replacement that spends the parent's output, rejected as bad-txns-spends-conflicting-tx. I measured both with testmempoolaccept on Bitcoin Core 31.1 regtest, and can share the numbers and the program.
|
Thanks for the detailed review and the regtest measurements! You are right on both points. For the first issue, I need to add an additional check in |
|
Bumping a parent together with its child is rejected, and a merged replacement of two independent txs can pay 351 sat less than the fees it replaces, on your head Test program (Cargo.toml, run.sh, src/main.rs)
[package]
name = "measure"
version = "0.1.0"
edition = "2021"
[dependencies]
# point this at a checkout of the branch under test
bdk_wallet = { path = "../bdk_wallet" }
serde_json = "1"
#!/bin/sh
# Reproduces two review points of PR 605 on a throwaway Bitcoin Core regtest node.
# Needs bitcoind and bitcoin-cli v31.1 on PATH, and cargo.
# Set the bdk_wallet path in Cargo.toml first. RPCPORT overrides port 18999.
set -e
P=${RPCPORT:-18999}
D=$(mktemp -d)
export CLI="bitcoin-cli -regtest -datadir=$D -rpcport=$P"
trap '$CLI stop >/dev/null 2>&1 || true
n=0; while [ -e "$D/regtest/bitcoind.pid" ] && [ "$n" -lt 30 ]; do n=$((n + 1)); sleep 1; done
rm -rf "$D"' EXIT
bitcoind -regtest -daemon -datadir="$D" -rpcport=$P -listen=0 -txindex=1 -fallbackfee=0.0001 >/dev/null
n=0
until $CLI getblockcount >/dev/null 2>&1; do
n=$((n + 1)); [ "$n" -lt 60 ] || { echo "bitcoind did not start" >&2; exit 1; }
sleep 1
done
$CLI createwallet miner >/dev/null
$CLI generatetoaddress 101 "$($CLI -rpcwallet=miner getnewaddress)" >/dev/null
cargo run --quiet
// Needs a running regtest bitcoind with a funded wallet "miner"; run it through run.sh.
// CLI is the bitcoin-cli command line, e.g. "bitcoin-cli -regtest -rpcport=18999".
use bdk_wallet::bitcoin::{consensus::encode::{deserialize_hex, serialize_hex}, Address, Amount, FeeRate, Network, OutPoint, Transaction, Txid};
use bdk_wallet::{KeychainKind, SignOptions, Wallet};
use std::process::Command;
use std::str::FromStr;
const DESC: &str = "wpkh(tprv8ZgxMBicQKsPdy6LMhUtFHAgpocR8GC6QmwMSFpZs7h6Eziw3SpThFfczTDh5rW2krkqffa11UpX3XkeTTB2FvzZKWXqPY54Y6Rq4AQ5R8L/84'/1'/0'/0/*)";
const CHANGE: &str = "wpkh(tprv8ZgxMBicQKsPdy6LMhUtFHAgpocR8GC6QmwMSFpZs7h6Eziw3SpThFfczTDh5rW2krkqffa11UpX3XkeTTB2FvzZKWXqPY54Y6Rq4AQ5R8L/84'/1'/0'/1/*)";
fn cli(args: &[&str]) -> String {
let base: Vec<String> = std::env::var("CLI").unwrap().split(' ').map(String::from).collect();
let out = Command::new(&base[0]).args(&base[1..]).args(args).output().unwrap();
assert!(out.status.success(), "{}", String::from_utf8_lossy(&out.stderr));
String::from_utf8_lossy(&out.stdout).trim().to_string()
}
fn miner_address() -> Address {
Address::from_str(&cli(&["-rpcwallet=miner", "getnewaddress"])).unwrap().assume_checked()
}
// Pay `n` coins of 0.01 BTC to the wallet, confirm them, and return their outpoints.
fn fund(w: &mut Wallet, n: usize) -> Vec<OutPoint> {
let txids: Vec<String> = (0..n)
.map(|_| {
let a = w.reveal_next_address(KeychainKind::External).address.to_string();
cli(&["-rpcwallet=miner", "sendtoaddress", &a, "0.01"])
})
.collect();
cli(&["generatetoaddress", "1", &miner_address().to_string()]);
let mut ops = vec![];
for (seen, t) in txids.iter().enumerate() {
let tx: Transaction = deserialize_hex(&cli(&["getrawtransaction", t])).unwrap();
for (i, o) in tx.output.iter().enumerate() {
if w.is_mine(o.script_pubkey.clone()) {
ops.push(OutPoint::new(tx.compute_txid(), i as u32));
}
}
w.apply_unconfirmed_txs([(tx, 1_000 + seen as u64)]);
}
ops
}
// Spend `utxo` (alone) at `rate` sat/vB, broadcast it, and record it in the wallet.
fn send(w: &mut Wallet, utxo: OutPoint, sats: u64, rate: u32, seen: u64) -> Transaction {
let mut b = w.build_tx();
b.add_utxo(utxo).unwrap();
b.manually_selected_only()
.add_recipient(miner_address().script_pubkey(), Amount::from_sat(sats))
.fee_rate(FeeRate::from_sat_per_vb_u32(rate));
let mut psbt = b.finish().unwrap();
assert!(w.sign(&mut psbt, SignOptions::default()).unwrap());
let tx = psbt.extract_tx().unwrap();
cli(&["sendrawtransaction", &serialize_hex(&tx)]);
w.apply_unconfirmed_txs([(tx.clone(), seen)]);
tx
}
fn mempool_fee(txid: &Txid) -> u64 {
let e: serde_json::Value = serde_json::from_str(&cli(&["getmempoolentry", &txid.to_string()])).unwrap();
(e["fees"]["base"].as_f64().unwrap() * 1e8).round() as u64
}
// Build the replacement for `txids` at `rate` sat/vB and ask Core whether it would accept it.
fn bump(w: &mut Wallet, label: &str, txids: &[Txid], rate: u32) {
println!("== {label}");
let fees: Vec<u64> = txids.iter().map(mempool_fee).collect();
let sum: u64 = fees.iter().sum();
println!(" replaced fees: {fees:?} = {sum}");
let psbt = (|| {
let mut b = w.build_fee_bump(txids.iter().copied()).map_err(|e| format!("{e:?}"))?;
b.fee_rate(FeeRate::from_sat_per_vb_u32(rate));
b.finish().map_err(|e| format!("{e:?}"))
})();
let mut psbt = match psbt {
Ok(p) => p,
Err(e) => { println!(" error: {e}"); return; }
};
w.sign(&mut psbt, SignOptions::default()).unwrap();
let fee = psbt.fee().unwrap().to_sat();
let tx = psbt.extract_tx().unwrap();
println!(" replacement: {} in / {} out, fee {fee} sat, vsize {}; fee - sum = {}",
tx.input.len(), tx.output.len(), tx.vsize(), fee as i64 - sum as i64);
let r: serde_json::Value = serde_json::from_str(&cli(&["testmempoolaccept", &format!("[\"{}\"]", serialize_hex(&tx))])).unwrap();
println!(" testmempoolaccept: allowed={} {}", r[0]["allowed"], r[0]["reject-details"].as_str().unwrap_or(""));
}
fn main() {
let mut w = Wallet::create(DESC, CHANGE).network(Network::Regtest).create_wallet_no_persist().unwrap();
let u = fund(&mut w, 3);
// Point 1: two independent transactions bumped together with `fee_rate`.
let a = send(&mut w, u[0], 300_000, 20, 2_000);
let b = send(&mut w, u[1], 300_000, 20, 2_001);
let ab = [a.compute_txid(), b.compute_txid()];
bump(&mut w, "point 1: two txs at 20 sat/vB, fee_rate 22", &ab, 22);
bump(&mut w, "point 1: same, fee_rate 25", &ab, 25);
// Point 2: a parent and its child passed together.
let p = send(&mut w, u[2], 300_000, 10, 2_002);
let change = w.list_unspent().find(|o| o.outpoint.txid == p.compute_txid()).unwrap().outpoint;
let c = send(&mut w, change, 200_000, 10, 2_003);
bump(&mut w, "point 2: parent + child together, fee_rate 20", &[p.compute_txid(), c.compute_txid()], 20);
} |
|
Apologies for the delay in getting back to this. Both issues you raised are now addressed:
I added tests for both cases: I wasn't able to run your regtest harness, but all unit tests pass. Happy to run it if you'd like — let me know if |
fametrano
left a comment
There was a problem hiding this comment.
Concept ACK
One issue should be fixed before merge: the new fee check refuses valid bumps, single-tx ones too.
I reran my program, with extra cases I can share, at d472d3f and 4f5c40b on Core 31.1 regtest. The check compares coin_selection.fee_amount, only the selected inputs' share, with the replaced fees, instead of the replacement's whole fee. Two txs paying 5620 sat are refused at every rate I tried from 22 to 41 sat/vB, built from 42; 4f5c40b built a 5988 sat replacement at 25 that Core accepted. One tx paying 2810 sat is refused at 25, 30 and 40 sat/vB, where 4f5c40b built replacements Core accepted. The lib test test_build_fee_bump_remove_change_output_single_desc now fails with FeeTooLow { required: 1001 SAT }; it passes on 4f5c40b and master.
Not blocking: a parent and child passed together are now refused, but a grandparent and grandchild listed without the middle tx still build a replacement Core rejects with bad-txns-spends-conflicting-tx at 30 sat/vB.
|
Thanks for the review. The blocking issue is fixed, the fee check now compares the replacement's total fee (all inputs minus all outputs) against the combined fee floor, instead of |
fametrano
left a comment
There was a problem hiding this comment.
tACK 92b960c
The blocking issue from my last review is fixed: the fee check now uses the replacement's whole fee, and valid bumps build again. I reran my program at 92b960c on Bitcoin Core 31.1 regtest, checking each replacement with testmempoolaccept (program and output: https://gist.github.com/fametrano/6d95907381ab774cdd66e599fed500c8/11ea838a694aa1c050aad1562a4f978abd5e177b). Two txs paying 5620 sat together now build replacements Core accepts at every rate I tried from 25 to 50 sat/vB, and are refused at 22, where 4f5c40b built one Core rejected. One tx paying 2810 sat builds accepted replacements at 25, 30, 40 and 50 sat/vB. A parent and child passed together are still refused. test_build_fee_bump_remove_change_output_single_desc passes again, and the test suite passes with and without bdk_wallet_unstable.
The grandparent and grandchild case is unchanged: listed without the middle tx, they still build a replacement Core rejects with bad-txns-spends-conflicting-tx at 30 sat/vB, because the descendant check looks only at direct parents. henrypeters#1, against your branch, checks all descendants; take it here or in the follow-up you suggested.
Not blocking my approval, but needed before merge: cargo fmt --all --check reports diffs in error.rs, mod.rs and tests/build_fee_bump.rs; clippy with -D warnings stops on the unused write import in error.rs; and the five commits are unsigned, which CONTRIBUTING.md requires and the check-signed-commits job checks. CI will flag all three once it runs.
92b960c to
b05eed1
Compare
|
Thanks for the approval. All three items are now addressed:
I also integrated your descendant check fix from henrypeters#1, the grandparent/grandchild case is now caught and all tests pass |
fametrano
left a comment
There was a problem hiding this comment.
tACK 38710a1
The fmt, clippy and signing points from my last review are fixed, and so is the grandparent and grandchild case. The branch is rebased on master 48bbd62, and git range-diff against 92b960c shows the five earlier commits unchanged, one new commit for fmt and clippy, and the head commit, which is my fix from henrypeters#1 with the same changes. All seven commits are signed, and GitHub verifies each. cargo fmt --all --check and clippy with -D warnings pass, and so do the tests with and without bdk_wallet_unstable. I reran my program on Bitcoin Core 31.1 regtest (https://gist.github.com/fametrano/6d95907381ab774cdd66e599fed500c8/964f4800d89f5f74b8d16a625c946f4043dd50b7): the grandparent and grandchild are now refused with TransactionIsDescendant, and every other case gives the same result as at 92b960c.
38710a1 to
3399eef
Compare
|
I squashed the 7 commits into 2(fix + test) for a cleaner history . No code changes, just re-organized. |
3399eef to
30f1863
Compare
fametrano
left a comment
There was a problem hiding this comment.
Concept ACK
The rebase onto master broke the build, so 30f1863 cannot be merged as it is.
Your squash was a pure reorganisation, as you said: 3399eef has the same tree as 38710a1, which I tested. The conflict with #423 was then resolved by keeping both versions of the foreign-input None arm in build_fee_bump: the old one inside the match, and #423's after the match closes. rustc stops at src/wallet/mod.rs:1880 with mismatched closing delimiter. Two calls that #423 added in tests/build_fee_bump.rs (lines 985 and 991) also still pass a single txid and need [txid]. CI has not run on this push, so it shows none of this yet.
With the old arm (lines 1832-1852) deleted and those two calls changed, clippy with -D warnings and the tests pass with and without bdk_wallet_unstable, and cargo fmt only changes indentation there. My regtest cases on Bitcoin Core 32.1 give the same results as at 38710a1. The diff, the build error and the output are here: https://gist.github.com/fametrano/000fdbd0ef4cb0a81cbc3af10baf4f4d/5a1a8eec2361c35e18d39ffc7bf077ccbb1d7783
Summary
build_fee_bumppreviously accepted a singleTxid, limiting fee bumping to one transaction at a time. Bitcoin's RBF rules allow a single replacement transaction to conflict with and replace multiple unconfirmed transactionssimultaneously, as long as the replacement spends at least one input from each transaction being replaced.
Fix
I replaced the
txid: Txidparameter withtxids: impl IntoIterator<Item = Txid>, alowing one or more transaction IDs to be passed. The function now:BuildFeeBumpError::NoTransactionsProvidedwhen an empty iterator is passedcombined
TxBuilderbeat all of them
NOTE: All existing call sites are updated to wrap single txids in
[txid]tosatisfy the new
IntoIteratorbound. Single txid behaviour is fullypreserved.
BIP125 compliance fixes (addressing review feedback from @fametrano):
BuildFeeBumpError::TransactionIsDescendant(Txid). Before building the replacement,build_fee_bumpchecks whether any provided txid spends an output of another provided txid. If so, it returns an error early, because the replacement would spend a ghost output that no longer exists once the parent is evicted (Core rejects withbad-txns-spends-conflicting-tx).FeeRatebumps (BIP125 Rule 3) — TheFeeRatepath increate_txonly enforced the rate floor (highest original rate + 1 sat/vB). When bumping multiple transactions, the replacement is smaller in vbytes than the originals combined, so a higher rate can still produce a lower absolute fee. Added a post-coin-selection check that rejects the replacement if its actual fee does not exceed the combined fee floor (previous_fee.absolute). This check only applies to theFeeRatepath; theFeeAmountpath already validates the absolute fee before coin selection.Testing
test_build_fee_bump_no_txids_returns_error— empty iterator returnsNoTransactionsProvidedtest_build_fee_bump_single_txid_still_works— single txid behaviouris preserved
test_build_fee_bump_multiple_txids_combines_inputs_and_recipients—bumping two transactions at once produces a replacement with inputs and
recipients from both original transactions
test_build_fee_bump_parent_child_returns_descendant_error— passing a parent and its child returnsTransactionIsDescendanttest_build_fee_bump_multiple_txids_fee_rate_absolute_too_low— bumping multiple txs at aFeeRatethat passes the rate floor but produces an absolute fee below the combined originals returns `FeeTooLow