9507819776

Committed 13 Jun 2024 10:52PM UTC coverage: 83.064% (-0.05%) from 83.114%

Build # 9507819776

Build Type

push

github

Committed by

notmandatory

Commit Message

Merge bitcoindevkit/bdk#1454: Refactor wallet and persist mod, remove bdk_persist crate

ec36c7ecc feat(example): use changeset staging with rpc polling example (志宇)
19328d499 feat(wallet)!: change persist API to use `StageExt` and `StageExtAsync` (志宇)
2e40b0118 feat(chain): reintroduce a way to stage changesets before persisting (志宇)
36e82ec68 chore(chain): relax `miniscript` feature flag scope (志宇)
9e97ac033 refactor(persist): update file_store, sqlite, wallet to use bdk_chain::persist (Steve Myers)
54b0c11cb feat(persist): add PersistAsync trait and StagedPersistAsync struct (Steve Myers)
aa640ab27 refactor(persist): rename PersistBackend to Persist, move to chain crate (Steve Myers)

Pull request description:

  ### Description

  Sorry to submit another refactor PR for the persist related stuff, but I think it's worth revisiting. My primary motivations are:

  1. remove `db` from `Wallet` so users have the ability to use `async` storage crates, for example using `sqlx`. I updated docs and examples to let users know they are responsible for persisting changes.
  2. remove the `anyhow` dependency everywhere (except as a dev test dependency). It really doesn't belong in a lib and by removing persistence from `Wallet` it isn't needed.
  3. remove the `bdk_persist` crate and revert back to the original design with generic error types. I kept the `Debug` and `Display` constrains on persist errors so they could still be used with the `anyhow!` macro.

  ### Notes to the reviewers

  I also replaced/renamed old `Persist` with `StagedPersist` struct inspired by #1453, it is only used in examples. The `Wallet` handles it's own staging.

  ### Changelog notice

  Changed

  - Removed `db` from `Wallet`, users are now responsible for persisting ... (continued)

Run Details

176 of 248 new or added lines in 6 files covered. (70.97%)

3 existing lines in 2 files now uncovered.

11153 of 13427 relevant lines covered (83.06%)

16771.13 hits per line

Source File
Press 'n' to go to next uncovered line, 'b' for previous

58.24

/crates/chain/src/persist.rs

//! This module is home to the [`PersistBackend`] trait which defines the behavior of a data store
//! required to persist changes made to BDK data structures.
//!
//! The [`CombinedChangeSet`] type encapsulates a combination of [`crate`] structures that are
//! typically persisted together.

#[cfg(feature = "async")]
use alloc::boxed::Box;
#[cfg(feature = "async")]
use async_trait::async_trait;
use core::convert::Infallible;
use core::fmt::{Debug, Display};

use crate::Append;

/// A changeset containing [`crate`] structures typically persisted together.
#[derive(Debug, Clone, PartialEq)]
#[cfg(feature = "miniscript")]
#[cfg_attr(
    feature = "serde",
    derive(crate::serde::Deserialize, crate::serde::Serialize),
    serde(
        crate = "crate::serde",
        bound(
            deserialize = "A: Ord + crate::serde::Deserialize<'de>, K: Ord + crate::serde::Deserialize<'de>",
            serialize = "A: Ord + crate::serde::Serialize, K: Ord + crate::serde::Serialize",
        ),
    )
)]
pub struct CombinedChangeSet<K, A> {
    /// Changes to the [`LocalChain`](crate::local_chain::LocalChain).
    pub chain: crate::local_chain::ChangeSet,
    /// Changes to [`IndexedTxGraph`](crate::indexed_tx_graph::IndexedTxGraph).
    pub indexed_tx_graph: crate::indexed_tx_graph::ChangeSet<A, crate::keychain::ChangeSet<K>>,
    /// Stores the network type of the transaction data.
    pub network: Option<bitcoin::Network>,
}

#[cfg(feature = "miniscript")]
impl<K, A> core::default::Default for CombinedChangeSet<K, A> {
    fn default() -> Self {
        Self {
            chain: core::default::Default::default(),
            indexed_tx_graph: core::default::Default::default(),
            network: None,
        }
    }
}

#[cfg(feature = "miniscript")]
impl<K: Ord, A: crate::Anchor> crate::Append for CombinedChangeSet<K, A> {
    fn append(&mut self, other: Self) {
        crate::Append::append(&mut self.chain, other.chain);
        crate::Append::append(&mut self.indexed_tx_graph, other.indexed_tx_graph);
        if other.network.is_some() {
            debug_assert!(
                self.network.is_none() || self.network == other.network,
                "network type must either be just introduced or remain the same"
            );
            self.network = other.network;
        }
    }

    fn is_empty(&self) -> bool {
        self.chain.is_empty() && self.indexed_tx_graph.is_empty() && self.network.is_none()
    }
}

#[cfg(feature = "miniscript")]
impl<K, A> From<crate::local_chain::ChangeSet> for CombinedChangeSet<K, A> {
    fn from(chain: crate::local_chain::ChangeSet) -> Self {
        Self {
            chain,
            ..Default::default()
        }
    }
}

#[cfg(feature = "miniscript")]
impl<K, A> From<crate::indexed_tx_graph::ChangeSet<A, crate::keychain::ChangeSet<K>>>
    for CombinedChangeSet<K, A>
{
    fn from(
        indexed_tx_graph: crate::indexed_tx_graph::ChangeSet<A, crate::keychain::ChangeSet<K>>,
    ) -> Self {
        Self {
            indexed_tx_graph,
            ..Default::default()
        }
    }
}

#[cfg(feature = "miniscript")]
impl<K, A> From<crate::keychain::ChangeSet<K>> for CombinedChangeSet<K, A> {
    fn from(indexer: crate::keychain::ChangeSet<K>) -> Self {
        Self {
            indexed_tx_graph: crate::indexed_tx_graph::ChangeSet {
                indexer,
                ..Default::default()
            },
            ..Default::default()
        }
    }
}

/// A persistence backend for writing and loading changesets.
///
/// `C` represents the changeset; a datatype that records changes made to in-memory data structures
/// that are to be persisted, or retrieved from persistence.
pub trait PersistBackend<C> {
    /// The error the backend returns when it fails to write.
    type WriteError: Debug + Display;

    /// The error the backend returns when it fails to load changesets `C`.
    type LoadError: Debug + Display;

    /// Writes a changeset to the persistence backend.
    ///
    /// It is up to the backend what it does with this. It could store every changeset in a list or
    /// it inserts the actual changes into a more structured database. All it needs to guarantee is
    /// that [`load_from_persistence`] restores a keychain tracker to what it should be if all
    /// changesets had been applied sequentially.
    ///
    /// [`load_from_persistence`]: Self::load_changes
    fn write_changes(&mut self, changeset: &C) -> Result<(), Self::WriteError>;

    /// Return the aggregate changeset `C` from persistence.
    fn load_changes(&mut self) -> Result<Option<C>, Self::LoadError>;
}

impl<C> PersistBackend<C> for () {
    type WriteError = Infallible;
    type LoadError = Infallible;

    fn write_changes(&mut self, _changeset: &C) -> Result<(), Self::WriteError> {
        Ok(())
    }

    fn load_changes(&mut self) -> Result<Option<C>, Self::LoadError> {
        Ok(None)
    }
}

#[cfg(feature = "async")]
/// An async persistence backend for writing and loading changesets.
///
/// `C` represents the changeset; a datatype that records changes made to in-memory data structures
/// that are to be persisted, or retrieved from persistence.
#[async_trait]
pub trait PersistBackendAsync<C> {
    /// The error the backend returns when it fails to write.
    type WriteError: Debug + Display;

    /// The error the backend returns when it fails to load changesets `C`.
    type LoadError: Debug + Display;

    /// Writes a changeset to the persistence backend.
    ///
    /// It is up to the backend what it does with this. It could store every changeset in a list or
    /// it inserts the actual changes into a more structured database. All it needs to guarantee is
    /// that [`load_from_persistence`] restores a keychain tracker to what it should be if all
    /// changesets had been applied sequentially.
    ///
    /// [`load_from_persistence`]: Self::load_changes
    async fn write_changes(&mut self, changeset: &C) -> Result<(), Self::WriteError>;

    /// Return the aggregate changeset `C` from persistence.
    async fn load_changes(&mut self) -> Result<Option<C>, Self::LoadError>;
}

#[cfg(feature = "async")]
#[async_trait]
impl<C> PersistBackendAsync<C> for () {
    type WriteError = Infallible;
    type LoadError = Infallible;

    async fn write_changes(&mut self, _changeset: &C) -> Result<(), Self::WriteError> {
        Ok(())
    }

    async fn load_changes(&mut self) -> Result<Option<C>, Self::LoadError> {
        Ok(None)
    }
}

/// Extends a changeset so that it acts as a convenient staging area for any [`PersistBackend`].
///
/// Not all changes to the in-memory representation needs to be written to disk right away.
/// [`Append::append`] can be used to *stage* changes first and then [`StageExt::commit_to`] can be
/// used to write changes to disk.
pub trait StageExt: Append + Default + Sized {
    /// Commit the staged changes to the persistence `backend`.
    ///
    /// Changes that are committed (if any) are returned.
    ///
    /// # Error
    ///
    /// Returns a backend-defined error if this fails.
    fn commit_to<B>(&mut self, backend: &mut B) -> Result<Option<Self>, B::WriteError>
    where
        B: PersistBackend<Self>,
    {
        // do not do anything if changeset is empty
        if self.is_empty() {
            return Ok(None);
        }
        backend.write_changes(&*self)?;
        // only clear if changes are written successfully to backend
        Ok(Some(core::mem::take(self)))
    }

    /// Stages a new `changeset` and commits it (alongside any other previously staged changes) to
    /// the persistence `backend`.
    ///
    /// Convenience method for calling [`Append::append`] and then [`StageExt::commit_to`].
    fn append_and_commit_to<B>(
        &mut self,
        changeset: Self,
        backend: &mut B,
    ) -> Result<Option<Self>, B::WriteError>
    where
        B: PersistBackend<Self>,
    {
        Append::append(self, changeset);
        self.commit_to(backend)
    }
}

impl<C: Append + Default> StageExt for C {}

/// Extends a changeset so that it acts as a convenient staging area for any
/// [`PersistBackendAsync`].
///
/// Not all changes to the in-memory representation needs to be written to disk right away.
/// [`Append::append`] can be used to *stage* changes first and then [`StageExtAsync::commit_to`]
/// can be used to write changes to disk.
#[cfg(feature = "async")]
#[async_trait]
pub trait StageExtAsync: Append + Default + Sized + Send + Sync {
    /// Commit the staged changes to the persistence `backend`.
    ///
    /// Changes that are committed (if any) are returned.
    ///
    /// # Error
    ///
    /// Returns a backend-defined error if this fails.
    async fn commit_to<B>(&mut self, backend: &mut B) -> Result<Option<Self>, B::WriteError>
    where
        B: PersistBackendAsync<Self> + Send + Sync,
    {
        // do not do anything if changeset is empty
        if self.is_empty() {
            return Ok(None);
        }
        backend.write_changes(&*self).await?;
        // only clear if changes are written successfully to backend
        Ok(Some(core::mem::take(self)))
    }

    /// Stages a new `changeset` and commits it (alongside any other previously staged changes) to
    /// the persistence `backend`.
    ///
    /// Convenience method for calling [`Append::append`] and then [`StageExtAsync::commit_to`].
    async fn append_and_commit_to<B>(
        &mut self,
        changeset: Self,
        backend: &mut B,
    ) -> Result<Option<Self>, B::WriteError>
    where
        B: PersistBackendAsync<Self> + Send + Sync,
    {
        Append::append(self, changeset);
        self.commit_to(backend).await
    }
}

#[cfg(feature = "async")]
#[async_trait]
impl<C: Append + Default + Send + Sync> StageExtAsync for C {}

1	//! This module is home to the [`PersistBackend`] trait which defines the behavior of a data store
2	//! required to persist changes made to BDK data structures.
3	//!
4	//! The [`CombinedChangeSet`] type encapsulates a combination of [`crate`] structures that are
5	//! typically persisted together.
6
7	#[cfg(feature = "async")]
8	use alloc::boxed::Box;
9	#[cfg(feature = "async")]
10	use async_trait::async_trait;
11	use core::convert::Infallible;
12	use core::fmt::{Debug, Display};
13
14	use crate::Append;
15
16	/// A changeset containing [`crate`] structures typically persisted together.
17	#[derive(Debug, Clone, PartialEq)]
18	#[cfg(feature = "miniscript")]
19	#[cfg_attr(
20	feature = "serde",
21	derive(crate::serde::Deserialize, crate::serde::Serialize),	12✔
22	serde(
23	crate = "crate::serde",
24	bound(
25	deserialize = "A: Ord + crate::serde::Deserialize<'de>, K: Ord + crate::serde::Deserialize<'de>",
26	serialize = "A: Ord + crate::serde::Serialize, K: Ord + crate::serde::Serialize",
27	),
28	)
29	)]
30	pub struct CombinedChangeSet<K, A> {
31	/// Changes to the [`LocalChain`](crate::local_chain::LocalChain).
32	pub chain: crate::local_chain::ChangeSet,
33	/// Changes to [`IndexedTxGraph`](crate::indexed_tx_graph::IndexedTxGraph).
34	pub indexed_tx_graph: crate::indexed_tx_graph::ChangeSet<A, crate::keychain::ChangeSet<K>>,
35	/// Stores the network type of the transaction data.
36	pub network: Option<bitcoin::Network>,
37	}
38
39	#[cfg(feature = "miniscript")]
40	impl<K, A> core::default::Default for CombinedChangeSet<K, A> {
41	fn default() -> Self {	11,283✔
42	Self {	11,283✔
43	chain: core::default::Default::default(),	11,283✔
44	indexed_tx_graph: core::default::Default::default(),	11,283✔
45	network: None,	11,283✔
46	}	11,283✔
47	}	11,283✔
48	}
49
50	#[cfg(feature = "miniscript")]
51	impl<K: Ord, A: crate::Anchor> crate::Append for CombinedChangeSet<K, A> {
52	fn append(&mut self, other: Self) {	11,161✔
53	crate::Append::append(&mut self.chain, other.chain);	11,161✔
54	crate::Append::append(&mut self.indexed_tx_graph, other.indexed_tx_graph);	11,161✔
55	if other.network.is_some() {	11,161✔
56	debug_assert!(	3✔
57	self.network.is_none() \|\| self.network == other.network,	3✔
NEW 58	"network type must either be just introduced or remain the same"	×
59	);
60	self.network = other.network;	3✔
61	}	11,158✔
62	}	11,161✔
63
64	fn is_empty(&self) -> bool {	2,487✔
65	self.chain.is_empty() && self.indexed_tx_graph.is_empty() && self.network.is_none()	2,487✔
66	}	2,487✔
67	}
68
69	#[cfg(feature = "miniscript")]
70	impl<K, A> From<crate::local_chain::ChangeSet> for CombinedChangeSet<K, A> {
71	fn from(chain: crate::local_chain::ChangeSet) -> Self {	2,158✔
72	Self {	2,158✔
73	chain,	2,158✔
74	..Default::default()	2,158✔
75	}	2,158✔
76	}	2,158✔
77	}
78
79	#[cfg(feature = "miniscript")]
80	impl<K, A> From<crate::indexed_tx_graph::ChangeSet<A, crate::keychain::ChangeSet<K>>>
81	for CombinedChangeSet<K, A>
82	{
83	fn from(	5,860✔
84	indexed_tx_graph: crate::indexed_tx_graph::ChangeSet<A, crate::keychain::ChangeSet<K>>,	5,860✔
85	) -> Self {	5,860✔
86	Self {	5,860✔
87	indexed_tx_graph,	5,860✔
88	..Default::default()	5,860✔
89	}	5,860✔
90	}	5,860✔
91	}
92
93	#[cfg(feature = "miniscript")]
94	impl<K, A> From<crate::keychain::ChangeSet<K>> for CombinedChangeSet<K, A> {
95	fn from(indexer: crate::keychain::ChangeSet<K>) -> Self {	720✔
96	Self {	720✔
97	indexed_tx_graph: crate::indexed_tx_graph::ChangeSet {	720✔
98	indexer,	720✔
99	..Default::default()	720✔
100	},	720✔
101	..Default::default()	720✔
102	}	720✔
103	}	720✔
104	}
105
106	/// A persistence backend for writing and loading changesets.
107	///
108	/// `C` represents the changeset; a datatype that records changes made to in-memory data structures
109	/// that are to be persisted, or retrieved from persistence.
110	pub trait PersistBackend<C> {
111	/// The error the backend returns when it fails to write.
112	type WriteError: Debug + Display;
113
114	/// The error the backend returns when it fails to load changesets `C`.
115	type LoadError: Debug + Display;
116
117	/// Writes a changeset to the persistence backend.
118	///
119	/// It is up to the backend what it does with this. It could store every changeset in a list or
120	/// it inserts the actual changes into a more structured database. All it needs to guarantee is
121	/// that [`load_from_persistence`] restores a keychain tracker to what it should be if all
122	/// changesets had been applied sequentially.
123	///
124	/// [`load_from_persistence`]: Self::load_changes
125	fn write_changes(&mut self, changeset: &C) -> Result<(), Self::WriteError>;
126
127	/// Return the aggregate changeset `C` from persistence.
128	fn load_changes(&mut self) -> Result<Option<C>, Self::LoadError>;
129	}
130
131	impl<C> PersistBackend<C> for () {
132	type WriteError = Infallible;
133	type LoadError = Infallible;
134
NEW 135	fn write_changes(&mut self, _changeset: &C) -> Result<(), Self::WriteError> {	×
NEW 136	Ok(())	×
NEW 137	}	×
138
NEW 139	fn load_changes(&mut self) -> Result<Option<C>, Self::LoadError> {	×
NEW 140	Ok(None)	×
NEW 141	}	×
142	}
143
144	#[cfg(feature = "async")]
145	/// An async persistence backend for writing and loading changesets.
146	///
147	/// `C` represents the changeset; a datatype that records changes made to in-memory data structures
148	/// that are to be persisted, or retrieved from persistence.
149	#[async_trait]
150	pub trait PersistBackendAsync<C> {
151	/// The error the backend returns when it fails to write.
152	type WriteError: Debug + Display;
153
154	/// The error the backend returns when it fails to load changesets `C`.
155	type LoadError: Debug + Display;
156
157	/// Writes a changeset to the persistence backend.
158	///
159	/// It is up to the backend what it does with this. It could store every changeset in a list or
160	/// it inserts the actual changes into a more structured database. All it needs to guarantee is
161	/// that [`load_from_persistence`] restores a keychain tracker to what it should be if all
162	/// changesets had been applied sequentially.
163	///
164	/// [`load_from_persistence`]: Self::load_changes
165	async fn write_changes(&mut self, changeset: &C) -> Result<(), Self::WriteError>;
166
167	/// Return the aggregate changeset `C` from persistence.
168	async fn load_changes(&mut self) -> Result<Option<C>, Self::LoadError>;
169	}
170
171	#[cfg(feature = "async")]
172	#[async_trait]
173	impl<C> PersistBackendAsync<C> for () {
174	type WriteError = Infallible;
175	type LoadError = Infallible;
176
NEW 177	async fn write_changes(&mut self, _changeset: &C) -> Result<(), Self::WriteError> {	×
NEW 178	Ok(())	×
NEW 179	}	×
180
NEW 181	async fn load_changes(&mut self) -> Result<Option<C>, Self::LoadError> {	×
NEW 182	Ok(None)	×
NEW 183	}	×
184	}
185
186	/// Extends a changeset so that it acts as a convenient staging area for any [`PersistBackend`].
187	///
188	/// Not all changes to the in-memory representation needs to be written to disk right away.
189	/// [`Append::append`] can be used to stage changes first and then [`StageExt::commit_to`] can be
190	/// used to write changes to disk.
191	pub trait StageExt: Append + Default + Sized {
192	/// Commit the staged changes to the persistence `backend`.
193	///
194	/// Changes that are committed (if any) are returned.
195	///
196	/// # Error
197	///
198	/// Returns a backend-defined error if this fails.
199	fn commit_to<B>(&mut self, backend: &mut B) -> Result<Option<Self>, B::WriteError>	4✔
200	where	4✔
201	B: PersistBackend<Self>,	4✔
202	{	4✔
203	// do not do anything if changeset is empty	4✔
204	if self.is_empty() {	4✔
NEW 205	return Ok(None);	×
206	}	4✔
207	backend.write_changes(&*self)?;	4✔
208	// only clear if changes are written successfully to backend
209	Ok(Some(core::mem::take(self)))	4✔
210	}	4✔
211
212	/// Stages a new `changeset` and commits it (alongside any other previously staged changes) to
213	/// the persistence `backend`.
214	///
215	/// Convenience method for calling [`Append::append`] and then [`StageExt::commit_to`].
NEW 216	fn append_and_commit_to<B>(	×
NEW 217	&mut self,	×
NEW 218	changeset: Self,	×
NEW 219	backend: &mut B,	×
NEW 220	) -> Result<Option<Self>, B::WriteError>	×
NEW 221	where	×
NEW 222	B: PersistBackend<Self>,	×
NEW 223	{	×
NEW 224	Append::append(self, changeset);	×
NEW 225	self.commit_to(backend)	×
NEW 226	}	×
227	}
228
229	impl<C: Append + Default> StageExt for C {}
230
231	/// Extends a changeset so that it acts as a convenient staging area for any
232	/// [`PersistBackendAsync`].
233	///
234	/// Not all changes to the in-memory representation needs to be written to disk right away.
235	/// [`Append::append`] can be used to stage changes first and then [`StageExtAsync::commit_to`]
236	/// can be used to write changes to disk.
237	#[cfg(feature = "async")]
238	#[async_trait]
239	pub trait StageExtAsync: Append + Default + Sized + Send + Sync {
240	/// Commit the staged changes to the persistence `backend`.
241	///
242	/// Changes that are committed (if any) are returned.
243	///
244	/// # Error
245	///
246	/// Returns a backend-defined error if this fails.
247	async fn commit_to<B>(&mut self, backend: &mut B) -> Result<Option<Self>, B::WriteError>
248	where
249	B: PersistBackendAsync<Self> + Send + Sync,
NEW 250	{	×
NEW 251	// do not do anything if changeset is empty	×
NEW 252	if self.is_empty() {	×
NEW 253	return Ok(None);	×
NEW 254	}	×
NEW 255	backend.write_changes(&*self).await?;	×
NEW 256	// only clear if changes are written successfully to backend	×
NEW 257	Ok(Some(core::mem::take(self)))	×
NEW 258	}	×
259
260	/// Stages a new `changeset` and commits it (alongside any other previously staged changes) to
261	/// the persistence `backend`.
262	///
263	/// Convenience method for calling [`Append::append`] and then [`StageExtAsync::commit_to`].
264	async fn append_and_commit_to<B>(
265	&mut self,
266	changeset: Self,
267	backend: &mut B,
268	) -> Result<Option<Self>, B::WriteError>
269	where
270	B: PersistBackendAsync<Self> + Send + Sync,
NEW 271	{	×
NEW 272	Append::append(self, changeset);	×
NEW 273	self.commit_to(backend).await	×
NEW 274	}	×
275	}
276
277	#[cfg(feature = "async")]
278	#[async_trait]
279	impl<C: Append + Default + Send + Sync> StageExtAsync for C {}

bitcoindevkit / bdk / 9507819776

Source File Press 'n' to go to next uncovered line, 'b' for previous

Source File
Press 'n' to go to next uncovered line, 'b' for previous