This adds code to migrate the existing block index in ffldb to the new
format managed by the blockchain package and updates the code to use the
new infrastructure.
This adds infrastructure necessary for blockchain to independently
manage and persist a full block index in a bucket. Note that the
new infrastructure is not used yet as it will be integrated in future
commits after migrating existing databases has been done.
The purpose of this infrastructure is to significantly speed up startup
time which currently has to load a bunch of the most recent blocks in
order to rebuild the stake state and also to pave the way towards being
able to have the full block index in memory allows for a lot of
optimizations and greatly simplified code.
This also includes a full suite of tests to ensure proper serialization
and deserialization functionality.
This modifies the database versioning to use separate keys within the
version bucket for each component in order to simplify upgrade logic.
Having a single key for the various versions of the database components
makes it difficult to use because the version isn't known until the data
is read, however, modifying the format necessarily means the version is
needed before that point to know which format to read.
While here, it uses a uint64 to store the date instead of a uint32 so
the timestamp continues to work past 2106.
Also, it removes the upgrade started logic since it should be possible
to stop in the middle of an upgrade and resume later. Forcing a
completely new database to be created because an upgrade was stopped in
the middle is not ideal.
This merge commit adds the following code from the
github.com/decred/dcrutil package into a new
github.com/decred/dcrd/dcrutil package:
* Address handling
* Amount type
* AppDataDir func
* bitflags functions
* Block wrapper type
* Hash160 func
* Tx wrapper type
* WIF type
as well as all tests for this code.
The old github.com/decred/dcrutil/hdkeychain package has also been
merged and moved to github.com/decred/dcrd/dcrutil/hdkeychain.
dcrd packages have been updated to use the new packages and the dep
files have been updated for this change.
The legacy ticket database, which was GOB serialized and stored on
shut down, has been removed. Ticket state information is now held in
a stake node, which acts as a modularized "black box" to contain all
information about the state of the stake system. Stake nodes are now
a component of the blockchain blockNode struct, and are updated with
them.
Stake nodes, like their internal treap primitives, are immutable
objects that are created with their connect and disconnect node
functions. The blockchain database now stores all information about
the stake state of the best node in the block database. The blockchain
makes the assumption that the stake state of the best node is known at
any given time. If the states of former blocks or sidechains must be
evaluated, this can be achieved by iterating backwards along the
blockchain from the best node, and then connecting stake nodes
iteratively if necessary.
Performance improvements with this new module are dramatic. The long
delays on start up and shut down are removed. Blockchain
synchronization time is improved approximately 5-10x on the mainnet
chain. The state of the database is atomic, so unexpected shut downs
should no longer have the ability to disrupt the chain state.
An upgrade path has been added for version 1 blockchain databases.
Users with this blockchain database will automatically update when
they start their clients.
