Add chain syncing via compact block filters (BIP 157)#828
Draft
febyeji wants to merge 8 commits intolightningdevkit:mainfrom
Draft
Add chain syncing via compact block filters (BIP 157)#828febyeji wants to merge 8 commits intolightningdevkit:mainfrom
febyeji wants to merge 8 commits intolightningdevkit:mainfrom
Conversation
|
👋 I see @tnull was un-assigned. |
febyeji
force-pushed
the
add-cbf-chain-source
branch
5 times, most recently
from
6e39d0d to
86e88ce
Compare
febyeji
force-pushed
the
add-cbf-chain-source
branch
from
86e88ce to
3de4777
Compare
febyeji
force-pushed
the
add-cbf-chain-source
branch
from
3de4777 to
9692ff4
Compare
|
🔔 1st Reminder Hey @tnull! This PR has been waiting for your review. |
febyeji
force-pushed
the
add-cbf-chain-source
branch
6 times, most recently
from
388d7d2 to
fdf6710
Compare
Collaborator
|
@febyeji @randomlogin Please let me know when you consider merging of the two PRs completed and this is ready for a first round of review! |
Fixes the problem when the funding tx is not registered for CBF node, for example during splicing. Added macros to skip incompatible with CBF backend tests (which require mempool).
Decrease timeouts to not falsely conclude some tests as passing
Previously we tracked synced height and also had dirty flag for rescan which was made from the genesis block. This was removed in favor of block heights from channel manager and onchain wallet. Safety reorg interval of 7 blocks (look-back) is added. Force tests use CBF config with 1 required peers, as otherwise it would die if one of two required peers went offline (which made impossible testing of scenarios with 2 nodes on which goes offline).
add auto-restart with exponential backoff (#10) * refactor(cbf): extract build_cbf_node helper and add wallet reference * feat(cbf): auto-restart bip157 node with exponential backoff * feat(cbf): add liveness check before returning requester * fix(cbf): clean up scan state on filter scan failure * fix(cbf): add per-block-fetch timeout to wallet sync
The old wait_for_cbf_sync only checked that the onchain sync timestamp advanced, which could false-pass when the timestamp updated but wallet state was still stale. The new version verifies both onchain and lightning wallet syncs completed, and accepts a check closure so callers can assert concrete wallet state (e.g. balance, payment existence). Also simplifies CbfSyncConfig in test setup to use Default (which already sets required_peers: 1 and reasonable timeouts).
febyeji
force-pushed
the
add-cbf-chain-source
branch
from
22cb2c2 to
817e764
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add chain syncing via compact block filters (BIP 157/158) using the
bip157crate. Closes #767.Introduces
CbfChainSource: a new chain backend that syncs wallet and Lightning state through compact block filters over the Bitcoin P2P network.Commits
1. Add BIP 157 compact block filter chain source
Core implementation, built up layer by layer:
bip157dependency andCbfSyncConfigconfigurationCbfChainSourcelifecycle (start/stop) with P2P broadcastConfirmtrait integrationCbfSyncConfig,set_chain_source_cbf)TestChainSource::Cbf,wait_for_cbf_sync)2. Add CBF integration tests and documentation
--test-threads=1(exclusive bitcoind P2P port)LDK_TEST_CHAIN_SOURCEenv var for test configurabilityDesign decisions
bip157crate #767 discussion). Uses block-level averages, which can underestimate during high congestion. (Percentile selection partially mitigates this?)sync_wallets()) + optional background sync viaBackgroundSyncConfig--test-threads=1(exclusive bitcoind P2P port access)