dcrd/blockchain/stakeversion.go
Dave Collins 57b59b4e26
blockchain: Decouple processing and download logic.
This completely reworks the way block index and processing works to use
headers-first semantics and support out of order processing of the
associated block data.

This will ultimately provide a wide variety of benefits as it means the
entire shape of the block tree can be determined from the headers alone
which in turn will allow much more informed decisions to be made,
provides better information regarding sync status and warning
conditions, and allows future work to add invalidation and
reconsideration of arbitrary blocks.

It must be noted that this is a major change to the way the block index
and block handling is done and is deeply integrated with the consensus
rules, so it will need a significant amount of testing and extremely
careful review.

Also of note is that there are a lot of assumptions made in the calling
code in regards to expected error attribution and the state the chain
has at the moment notifications are processed, so all of those semantics
have intentionally been retained, even though they might seem a bit out
of place now, in order to limit the amount of changes introduced at a
time and thus better ensures correctness.

For example, in the future it would probably make more sense to make the
notifications entirely asynchronous and for validation failures to be
reported via those asynchronous notifications.  However, before that can
happen, all callers would first need to be updated to ensure they do not
rely on the chain being in the same state it was at the moment the
notification was generated.

In addition to all of the existing full block tests, this introduces
a comprehensive set of processing order tests which exercise all of the
new logic.

High level overview of the changes:

- Introduce tracking for whether a block is fully linked, meaning it
  builds on a branch that has block data for all of its ancestors
- Add received order tracking to ensure miners are not able to gain an
  advantage in terms of chain selection by only advertising a header
- Add several new pieces of information to the block index:
  - Header with the most cumulative work that is not known to be invalid
  - Header with the most cumulative work that is known to be invalid
  - Map of best chain candidates to aid in efficient selection
  - Map of unlinked children for more efficient linking
  - Prunable cached chain tips to significantly reduce potential search
    space during invalidation
- Introduce a compare function which determines which of two nodes
  should be considered better for the purposes of best chain selection
- Add chain tip iteration capabilities with potential filtering
- Mark all descendants invalid due to known invalid ancestor when a
  block is marked invalid
- Add ability to determine if a block can currently be validated
- Rework the chain reorganization func to work with an arbitrary target
- Modify the overall chain reorg func to reorg to the block with the
  most cumulative work that is valid in the case it is not possible to
  reorg to the target block
- Add a new MultiError type for keeping track of multiple errors in the
  same call
- Add a new ErrNoBlockData error kind
- Update all cases that only require all of the ancestor block data to
  be available to check for that condition instead of the more strict
  condition of already being fully validated
- Introduce a new processing lock separate from the overall chain lock
- Add new method and ability to independently accept block headers
- Rework the block processing func to accept blocks out of order so long
  as their header is already known valid
- Keep a cache of blocks that recently passed contextual validation
  checks
- Cleanup and correct various comments to match reality
- Add comprehensive tests to exercise the new processing logic and best
  header tracking (for both invalid and not known invalid)
2021-01-07 10:43:49 -06:00

323 lines
11 KiB
Go

// Copyright (c) 2016-2020 The Decred developers
// Use of this source code is governed by an ISC
// license that can be found in the LICENSE file.
package blockchain
import (
"encoding/binary"
"errors"
"fmt"
"github.com/decred/dcrd/chaincfg/chainhash"
)
var (
errVoterVersionMajorityNotFound = errors.New("voter version majority " +
"not found")
errStakeVersionMajorityNotFound = errors.New("stake version majority " +
"not found")
)
// stakeMajorityCacheVersionKey creates a map key that is comprised of a stake
// version and a hash. This is used for caches that require a version in
// addition to a simple hash.
func stakeMajorityCacheVersionKey(version uint32, hash *chainhash.Hash) [stakeMajorityCacheKeySize]byte {
var key [stakeMajorityCacheKeySize]byte
binary.LittleEndian.PutUint32(key[0:], version)
copy(key[4:], hash[:])
return key
}
// calcWantHeight calculates the height of the final block of the previous
// interval given a stake validation height, stake validation interval, and
// block height.
func calcWantHeight(stakeValidationHeight, interval, height int64) int64 {
// The adjusted height accounts for the fact the starting validation
// height does not necessarily start on an interval and thus the
// intervals might not be zero-based.
intervalOffset := stakeValidationHeight % interval
adjustedHeight := height - intervalOffset - 1
return (adjustedHeight - ((adjustedHeight + 1) % interval)) +
intervalOffset
}
// CalcWantHeight calculates the height of the final block of the previous
// interval given a block height.
func (b *BlockChain) CalcWantHeight(interval, height int64) int64 {
return calcWantHeight(b.chainParams.StakeValidationHeight, interval,
height)
}
// findStakeVersionPriorNode walks the chain backwards from prevNode until it
// reaches the final block of the previous stake version interval and returns
// that node. The returned node will be nil when the provided prevNode is too
// low such that there is no previous stake version interval due to falling
// prior to the stake validation interval.
//
// This function MUST be called with the chain state lock held (for writes).
func (b *BlockChain) findStakeVersionPriorNode(prevNode *blockNode) *blockNode {
// Check to see if the blockchain is high enough to begin accounting
// stake versions.
svh := b.chainParams.StakeValidationHeight
svi := b.chainParams.StakeVersionInterval
nextHeight := prevNode.height + 1
if nextHeight < svh+svi {
return nil
}
wantHeight := calcWantHeight(svh, svi, nextHeight)
return prevNode.Ancestor(wantHeight)
}
// isStakeMajorityVersion determines if minVer requirement is met based on
// prevNode. The function always uses the stake versions of the prior window.
// For example, if StakeVersionInterval = 11 and StakeValidationHeight = 13 the
// windows start at 13 + (11 * 2) 25 and are as follows: 24-34, 35-45, 46-56 ...
// If height comes in at 35 we use the 24-34 window, up to height 45.
// If height comes in at 46 we use the 35-45 window, up to height 56 etc.
//
// This function MUST be called with the chain state lock held (for writes).
func (b *BlockChain) isStakeMajorityVersion(minVer uint32, prevNode *blockNode) bool {
// Walk blockchain backwards to calculate version.
node := b.findStakeVersionPriorNode(prevNode)
if node == nil {
return 0 == minVer
}
// Generate map key and look up cached result.
key := stakeMajorityCacheVersionKey(minVer, &node.hash)
if result, ok := b.isStakeMajorityVersionCache[key]; ok {
return result
}
// Tally how many of the block headers in the previous stake version
// validation interval have their stake version set to at least the
// requested minimum version.
versionCount := int32(0)
iterNode := node
for i := int64(0); i < b.chainParams.StakeVersionInterval && iterNode != nil; i++ {
if iterNode.stakeVersion >= minVer {
versionCount++
}
iterNode = iterNode.parent
}
// Determine the required amount of votes to reach supermajority.
numRequired := int32(b.chainParams.StakeVersionInterval) *
b.chainParams.StakeMajorityMultiplier /
b.chainParams.StakeMajorityDivisor
// Cache result.
result := versionCount >= numRequired
b.isStakeMajorityVersionCache[key] = result
return result
}
// calcPriorStakeVersion calculates the header stake version of the prior
// interval. The function walks the chain backwards by one interval and then
// it performs a standard majority calculation.
//
// This function MUST be called with the chain state lock held (for writes).
func (b *BlockChain) calcPriorStakeVersion(prevNode *blockNode) (uint32, error) {
// Walk blockchain backwards to calculate version.
node := b.findStakeVersionPriorNode(prevNode)
if node == nil {
return 0, nil
}
// Check cache.
if result, ok := b.calcPriorStakeVersionCache[node.hash]; ok {
return result, nil
}
// Tally how many of each stake version the block headers in the previous stake
// version validation interval have.
versions := make(map[uint32]int32) // [version][count]
iterNode := node
for i := int64(0); i < b.chainParams.StakeVersionInterval && iterNode != nil; i++ {
versions[iterNode.stakeVersion]++
iterNode = iterNode.parent
}
// Determine the required amount of votes to reach supermajority.
numRequired := int32(b.chainParams.StakeVersionInterval) *
b.chainParams.StakeMajorityMultiplier /
b.chainParams.StakeMajorityDivisor
for version, count := range versions {
if count >= numRequired {
b.calcPriorStakeVersionCache[node.hash] = version
return version, nil
}
}
return 0, errStakeVersionMajorityNotFound
}
// calcVoterVersionInterval tallies all voter versions in an interval and
// returns a version that has reached 75% majority. This function MUST be
// called with a node that is the final node in a valid stake version interval
// and greater than or equal to the stake validation height or it will result in
// an assertion error.
//
// This function is really meant to be called internally only from this file.
//
// This function MUST be called with the chain state lock held (for writes).
func (b *BlockChain) calcVoterVersionInterval(prevNode *blockNode) (uint32, error) {
// Ensure the provided node is the final node in a valid stake version
// interval and is greater than or equal to the stake validation height
// since the logic below relies on these assumptions.
svh := b.chainParams.StakeValidationHeight
svi := b.chainParams.StakeVersionInterval
expectedHeight := calcWantHeight(svh, svi, prevNode.height+1)
if prevNode.height != expectedHeight || expectedHeight < svh {
return 0, AssertError(fmt.Sprintf("calcVoterVersionInterval "+
"must be called with a node that is the final node "+
"in a stake version interval -- called with node %s "+
"(height %d)", prevNode.hash, prevNode.height))
}
// See if we have cached results.
if result, ok := b.calcVoterVersionIntervalCache[prevNode.hash]; ok {
return result, nil
}
// Tally both the total number of votes in the previous stake version validation
// interval and how many of each version those votes have.
versions := make(map[uint32]int32) // [version][count]
totalVotesFound := int32(0)
iterNode := prevNode
for i := int64(0); i < svi && iterNode != nil; i++ {
totalVotesFound += int32(len(iterNode.votes))
for _, v := range iterNode.votes {
versions[v.Version]++
}
iterNode = iterNode.parent
}
// Determine the required amount of votes to reach supermajority.
numRequired := totalVotesFound * b.chainParams.StakeMajorityMultiplier /
b.chainParams.StakeMajorityDivisor
for version, count := range versions {
if count >= numRequired {
b.calcVoterVersionIntervalCache[prevNode.hash] = version
return version, nil
}
}
return 0, errVoterVersionMajorityNotFound
}
// calcVoterVersion calculates the last prior valid majority stake version. If
// the current interval does not have a majority stake version it'll go back to
// the prior interval. It'll keep going back up to the minimum height at which
// point we know the version was 0.
//
// This function MUST be called with the chain state lock held (for writes).
func (b *BlockChain) calcVoterVersion(prevNode *blockNode) (uint32, *blockNode) {
// Walk blockchain backwards to find interval.
node := b.findStakeVersionPriorNode(prevNode)
// Iterate over versions until a majority is found. Don't try to count
// votes before the stake validation height since there could not
// possibly have been any.
for node != nil && node.height >= b.chainParams.StakeValidationHeight {
version, err := b.calcVoterVersionInterval(node)
if err == nil {
return version, node
}
if !errors.Is(err, errVoterVersionMajorityNotFound) {
break
}
node = node.RelativeAncestor(b.chainParams.StakeVersionInterval)
}
// No majority version found.
return 0, nil
}
// calcStakeVersion calculates the header stake version based on voter versions.
// If there is a majority of voter versions it uses the header stake version to
// prevent reverting to a prior version.
//
// This function MUST be called with the chain state lock held (for writes).
func (b *BlockChain) calcStakeVersion(prevNode *blockNode) uint32 {
version, node := b.calcVoterVersion(prevNode)
if version == 0 || node == nil {
// Short circuit.
return 0
}
// Check cache.
if result, ok := b.calcStakeVersionCache[node.hash]; ok {
return result
}
// Walk chain backwards to start of node interval (start of current
// period) Note that calcWantHeight returns the LAST height of the
// prior interval; hence the + 1.
startIntervalHeight := calcWantHeight(b.chainParams.StakeValidationHeight,
b.chainParams.StakeVersionInterval, node.height) + 1
startNode := node.Ancestor(startIntervalHeight)
if startNode == nil {
// Note that should this not be possible to hit because a
// majority voter version was obtained above, which means there
// is at least an interval of nodes. However, be paranoid.
b.calcStakeVersionCache[node.hash] = 0
}
// See if we are enforcing V3 blocks yet. Just return V0 since it
// wasn't enforced and therefore irrelevant.
if !b.isMajorityVersion(3, startNode,
b.chainParams.BlockRejectNumRequired) {
b.calcStakeVersionCache[node.hash] = 0
return 0
}
// Don't allow the stake version to go backwards once it has been locked
// in by a previous majority, even if the majority of votes are now a
// lower version.
if b.isStakeMajorityVersion(version, node) {
priorVersion, _ := b.calcPriorStakeVersion(node)
if priorVersion > version {
version = priorVersion
}
}
b.calcStakeVersionCache[node.hash] = version
return version
}
// calcStakeVersionByHash calculates the last prior valid majority stake
// version. If the current interval does not have a majority stake version
// it'll go back to the prior interval. It'll keep going back up to the
// minimum height at which point we know the version was 0.
//
// This function MUST be called with the chain state lock held (for writes).
func (b *BlockChain) calcStakeVersionByHash(hash *chainhash.Hash) (uint32, error) {
prevNode := b.index.LookupNode(hash)
if prevNode == nil || !b.index.CanValidate(prevNode) {
return 0, fmt.Errorf("block %s is not known", hash)
}
return b.calcStakeVersion(prevNode), nil
}
// CalcStakeVersionByHash calculates the expected stake version for the block
// AFTER the provided block hash.
//
// This function is safe for concurrent access.
func (b *BlockChain) CalcStakeVersionByHash(hash *chainhash.Hash) (uint32, error) {
b.chainLock.Lock()
version, err := b.calcStakeVersionByHash(hash)
b.chainLock.Unlock()
return version, err
}