57b59b4e26
This completely reworks the way block index and processing works to use
headers-first semantics and support out of order processing of the
associated block data.
This will ultimately provide a wide variety of benefits as it means the
entire shape of the block tree can be determined from the headers alone
which in turn will allow much more informed decisions to be made,
provides better information regarding sync status and warning
conditions, and allows future work to add invalidation and
reconsideration of arbitrary blocks.
It must be noted that this is a major change to the way the block index
and block handling is done and is deeply integrated with the consensus
rules, so it will need a significant amount of testing and extremely
careful review.
Also of note is that there are a lot of assumptions made in the calling
code in regards to expected error attribution and the state the chain
has at the moment notifications are processed, so all of those semantics
have intentionally been retained, even though they might seem a bit out
of place now, in order to limit the amount of changes introduced at a
time and thus better ensures correctness.
For example, in the future it would probably make more sense to make the
notifications entirely asynchronous and for validation failures to be
reported via those asynchronous notifications. However, before that can
happen, all callers would first need to be updated to ensure they do not
rely on the chain being in the same state it was at the moment the
notification was generated.
In addition to all of the existing full block tests, this introduces
a comprehensive set of processing order tests which exercise all of the
new logic.
High level overview of the changes:
- Introduce tracking for whether a block is fully linked, meaning it
builds on a branch that has block data for all of its ancestors
- Add received order tracking to ensure miners are not able to gain an
advantage in terms of chain selection by only advertising a header
- Add several new pieces of information to the block index:
- Header with the most cumulative work that is not known to be invalid
- Header with the most cumulative work that is known to be invalid
- Map of best chain candidates to aid in efficient selection
- Map of unlinked children for more efficient linking
- Prunable cached chain tips to significantly reduce potential search
space during invalidation
- Introduce a compare function which determines which of two nodes
should be considered better for the purposes of best chain selection
- Add chain tip iteration capabilities with potential filtering
- Mark all descendants invalid due to known invalid ancestor when a
block is marked invalid
- Add ability to determine if a block can currently be validated
- Rework the chain reorganization func to work with an arbitrary target
- Modify the overall chain reorg func to reorg to the block with the
most cumulative work that is valid in the case it is not possible to
reorg to the target block
- Add a new MultiError type for keeping track of multiple errors in the
same call
- Add a new ErrNoBlockData error kind
- Update all cases that only require all of the ancestor block data to
be available to check for that condition instead of the more strict
condition of already being fully validated
- Introduce a new processing lock separate from the overall chain lock
- Add new method and ability to independently accept block headers
- Rework the block processing func to accept blocks out of order so long
as their header is already known valid
- Keep a cache of blocks that recently passed contextual validation
checks
- Cleanup and correct various comments to match reality
- Add comprehensive tests to exercise the new processing logic and best
header tracking (for both invalid and not known invalid)
91 lines
3.1 KiB
Go
91 lines
3.1 KiB
Go
// Copyright (c) 2015-2020 The Decred developers
|
|
// Use of this source code is governed by an ISC
|
|
// license that can be found in the LICENSE file.
|
|
|
|
package blockchain
|
|
|
|
import (
|
|
"math"
|
|
"time"
|
|
)
|
|
|
|
// poissonConfidenceSecs returns the number of seconds it will take to produce
|
|
// an event at the provided confidence level given a Poisson distribution with 1
|
|
// event ocurring in the given target interval.
|
|
func poissonConfidenceSecs(targetIntervalSecs int64, confidence float64) int64 {
|
|
// The waiting times between events in a Poisson distribution are
|
|
// exponentially distributed and the CDF for an exponential distribution is:
|
|
//
|
|
// x = 1 - e^-λt
|
|
//
|
|
// Since the goal rate is 1 event per target and time is in terms of
|
|
// targetIntervalSecs, let λ = 1, i = targetIntervalSecs, t = s/i, and x =
|
|
// confidence:
|
|
// => x = 1 - e^-(s/i)
|
|
//
|
|
// Solve for s:
|
|
// => e^-(s/i) = 1 - x
|
|
// => -(s/i) = ln(1 - x)
|
|
// => s = ln(1 - x) * -i
|
|
// => s = ln1p(-confidence) * -targetIntervalSecs
|
|
//
|
|
// Extra 0.5 to round up.
|
|
return int64(math.Log1p(-confidence)*-float64(targetIntervalSecs) + 0.5)
|
|
}
|
|
|
|
// chainPruner is used to occasionally prune the blockchain of old nodes that
|
|
// can be freed to the garbage collector.
|
|
type chainPruner struct {
|
|
chain *BlockChain
|
|
lastPruneTime time.Time
|
|
pruningInterval time.Duration
|
|
|
|
// prunedPerIntervalHint is the maximum expected number of nodes that will
|
|
// be pruned per pruning interval with a high degree of confidence.
|
|
prunedPerIntervalHint int64
|
|
}
|
|
|
|
// newChainPruner returns a new chain pruner.
|
|
func newChainPruner(chain *BlockChain) *chainPruner {
|
|
// Set the pruning interval to match the target time per block.
|
|
targetTimePerBlock := chain.chainParams.TargetTimePerBlock
|
|
pruningInterval := targetTimePerBlock
|
|
pruningIntervalSecs := int64(pruningInterval.Seconds())
|
|
|
|
// Calculate the maximum expected number of nodes that will be pruned per
|
|
// interval with a 99% confidence level to use as a hint for reducing the
|
|
// number of allocations.
|
|
//
|
|
// Note that as long as the pruning interval is the same as the target block
|
|
// interval, this will always result in the same value for all networks,
|
|
// but it's better to calculate it properly so it remains accurate if the
|
|
// pruning interval is changed.
|
|
const confidence = 0.99
|
|
targetTimePerBlockSecs := int64(targetTimePerBlock.Seconds())
|
|
confidenceSecs := poissonConfidenceSecs(targetTimePerBlockSecs, confidence)
|
|
pruneHintFloat := float64(confidenceSecs) / float64(pruningIntervalSecs)
|
|
pruneHint := int64(math.Round(pruneHintFloat))
|
|
|
|
return &chainPruner{
|
|
chain: chain,
|
|
lastPruneTime: time.Now(),
|
|
pruningInterval: pruningInterval,
|
|
prunedPerIntervalHint: pruneHint,
|
|
}
|
|
}
|
|
|
|
// pruneChainIfNeeded removes references to old information that should no
|
|
// longer be held in memory if the pruning interval has elapsed.
|
|
//
|
|
// This function MUST be called with the chain lock held (for writes).
|
|
func (c *chainPruner) pruneChainIfNeeded() {
|
|
now := time.Now()
|
|
duration := now.Sub(c.lastPruneTime)
|
|
if duration < c.pruningInterval {
|
|
return
|
|
}
|
|
|
|
c.lastPruneTime = now
|
|
c.chain.pruneStakeNodes()
|
|
}
|