This adds a directive to instruct gometalinter to ignore a false
positive produced by vet. The code in question intentionally makes use
of the property vet is warning about.
Change tickettreap to allow for efficient queries by min blockheight,
allowing for O(1) ticket expiry.
This allows for a very significant performance increase as previously
the same query required an iteration of 40,000 or so items while
processing each block.
Remove tickettreap.Mutable - it is not used, and it is difficult to
maintain with the other changes made to the treapNode class used by
Immutable.
Fix O(n) lookups-by-position into being O(ln n) in liveTickets.
This reduces the time to handle a new block by around a third.
Update memory-usage calculations to be more accurate, and have
extra testing.
Contains the following commits:
- 711f33450c
- b6b1e55d1e
- Reverted because Travis is already at a more recent version
- bd4e64d1d4
Also, the merge commit contains the necessary decred-specific
alterations, converts all other references to sha to hash to keep with
the spirit of the merged commits, and various other cleanup intended to
bring the code bases more in line with one another.
The legacy ticket database, which was GOB serialized and stored on
shut down, has been removed. Ticket state information is now held in
a stake node, which acts as a modularized "black box" to contain all
information about the state of the stake system. Stake nodes are now
a component of the blockchain blockNode struct, and are updated with
them.
Stake nodes, like their internal treap primitives, are immutable
objects that are created with their connect and disconnect node
functions. The blockchain database now stores all information about
the stake state of the best node in the block database. The blockchain
makes the assumption that the stake state of the best node is known at
any given time. If the states of former blocks or sidechains must be
evaluated, this can be achieved by iterating backwards along the
blockchain from the best node, and then connecting stake nodes
iteratively if necessary.
Performance improvements with this new module are dramatic. The long
delays on start up and shut down are removed. Blockchain
synchronization time is improved approximately 5-10x on the mainnet
chain. The state of the database is atomic, so unexpected shut downs
should no longer have the ability to disrupt the chain state.
An upgrade path has been added for version 1 blockchain databases.
Users with this blockchain database will automatically update when
they start their clients.
Ticket voting selection and validation require fast access to a
lexicographically sorted list of all available tickets. The current
implementation stores these tickets in bucketized maps which are
unordered. This means the data has to be sorted every time it is needed
in order to select a ticket and validate a block. On my hardware, the
process of removing tickets that have been voted and adding new tickets
takes ~27ms with the current implementation.
In order to speed this up, this introduces a new internal package named
tickettreap that provides a tailored treap data structure that is
intended to be used for the in-memory live ticket pool when the new
database and blockchain enhancements have been synced from upstream.
Since it maintains the data in sorted order, it provides a significant
speedup over the current map-based implementation. On my hardware, this
implementation is able to remove the tickets that have voted and add the
new tickets in ~0.083ms and iterate the entire structure for selecting the
tickets in ~1.88ms for a total of ~1.96ms. That equates to roughly 14x
faster access.
In addition, it would also be possible for the stake code to keep a map
for height-based lookups (while still relying on this new data structure
for keeping the lexicographic sort ordering requirements) for an even
greater speedup since it would eliminate the need to iterate the
structure to remove tickets at a given height.
Finally, when evaluating forks, the current code is quite inefficient
because it needs to make a copy of all of the live ticket maps. This
new structure provides O(1) snapshot/copy capabilities so it is much
more efficient in that regard as well.
Benchmarks:
ImmutableCopy 30000000 40.7 ns/op 0 B/op 0 allocs/op
ImmutableIterate 1000 1881738 ns/op 0 B/op 0 allocs/op
