This adds infrastructure to track missed votes and automatically
generate revocations for them along with a new assertion helper named
AssertTipNumRevocations which, as it name suggests, allows the caller to
assert the header commits to the specified number of revocations.
This will be useful for porting the rest of the block validation tests
to the fullblocktests framework.
Currently, the stake difficulty estimation function for algorithm
defined in DCP0001 only works with mainnet due to a limitation of only
working when the interval size is less than or equal to the ticket
maturity.
This generalizes the function to work with all values of interval size
and ticket maturity and adds a full suite of tests using the testnet
parameters to ensure proper functionality.
Of particular note is the case when the current height is before the
ticket maturity floor since that means some of the tickets being
estimated will mature during the interval and that there can be no
non-estimated immature tickets.
This also implies that, unlike when the ticket maturity is greater than
or equal to the interval size, the estimate may not exactly match the
real final value when given the exact same number of tickets to estimate
as real purchased tickets in the remainder of the interval depending on
whether or not the current height is before the ticket maturity floor.
This is the case because the number of real tickets that end up being
purchased before the maturity floor may not match the estimated
assumption of max tickets per block.
This finishes separating the mining code from the mempool that was
partially done in commit 9031d85 by extending the TxSource interface to
include the new functionality required by Decred.
The following is an overview of the changes to accomplish this:
- Move the VoteTx struct out of the mempool package in the mining
package and rename it to VoteDesc instead to signify it is a vote
descriptor and be consistent with the naming of TxDesc
- Update mempool to use the new mining.VoteDesc struct
- Rename CheckIfTxsExist to HaveAllTransactions to be more consistent
with HaveTransaction in the TxSource interface as it now lives
alongside it
- Add VoteHashesForBlock, VotesForBlocks, and IsTxTreeKnownInvalid
to the TxSource interface
- Switch the mining code to call the functions on the TxSource interface
instead and remove the reference to the concrete mempool
This replaces the mempool IsTxTreeValid function with
IsTxTreeKnownInvalid instead which more accurately describes its
semantics and purpose.
The actual purpose of this function is related to mempool acceptance and
building block templates for mining where the assumption is that the
previous block's regular transaction tree will be valid and is only
invalid specifically if there are enough known votes that vote against
it.
This change is desirable because the function provides semantics that
are actually opposite to that of the consensus rules where a tree is
only valid if it has a majority of yes votes and thus claiming it's
valid when that is not actually known for certain yet can easily cause
confusion.
The current code creates block templates with coinbase transactions that
have 4 uint64 extra nonces, but only ever uses one of them. Rather than
wasting the extra space, this modifies the code so it only uses a single
uint64 extra nonce.
It should be noted that realistically there isn't even a real need for
an extra nonce in the coinbase at all because there is extra space in
the header specifically for that purpose and miners can't modify the
coinbase via getwork anyways.
However, it is still necessary to insert for the current code in order
to ensure every block template has a unique merkle root since that is
what is used to uniquely identify the block template.
This modifies all files in the main package so that function definitions
are on the a single line as is the style used in the upstream code as
well as throughout its code base. This helps minimize the differences
between them and the upstream code in order to facilitate easier syncs
due to less merge conflicts as a result of superfluous changes.
This optimizes the way in which the mempool oprhan map is limited in the
same way the server block manager maps were previously optimized.
Previously the code would read a cryptographically random value large
enough to construct a hash, find the first entry larger than that value,
and evict it.
That approach is quite inefficient and could easily become a
bottleneck when processing transactions due to the need to read from a
source such as /dev/urandom and all of the subsequent hash comparisons.
Luckily, strong cryptographic randomness is not needed here. The primary
intent of limiting the maps is to control memory usage with a secondary
concern of making it difficult for adversaries to force eviction of
specific entries.
Consequently, this changes the code to make use of the pseudorandom
iteration order of Go's maps along with the preimage resistance of the
hashing function to provide the desired functionality. It has
previously been discussed that the specific pseudorandom iteration order
is not guaranteed by the Go spec even though in practice that is how it
is implemented. This is not a concern however because even if the
specific compiler doesn't implement that, the preimage resistance of the
hashing function alone is enough.
The following is a before and after comparison of the function for both
speed and memory allocations:
benchmark old ns/op new ns/op delta
----------------------------------------------------------------
BenchmarkLimitNumOrphans 3727 243 -93.48%
benchmark old allocs new allocs delta
-----------------------------------------------------------------
BenchmarkLimitNumOrphans 4 0 -100.00%
This moves the priority-related code from the mempool package to the
mining package and also exports a new constant named UnminedHeight which
takes the place of the old unexported mempoolHeight.
Even though the mempool makes use of the priority code to make decisions
about what it will accept, priority really has to do with mining since
it influences which transactions will end up into a block. This change
also has the side effect of being a step towards enabling separation of
the mining code into its own package which, as previously mentioned,
needs access to the priority calculation code as well.
Finally, the mempoolHeight variable was poorly named since what it
really represents is a transaction that has not been mined into a block
yet. Renaming the variable to more accurately reflect its purpose makes
it clear that it belongs in the mining package which also needs the
definition now as well since the priority calculation code relies on it.
This will also benefit an outstanding PR which needs access to the same
value.
This fixes a bug where a transaction would lose reference to other
transactions dependant on it when being considered for inclusion in a
new block template. The issue only occurs when the transaction being
considered triggers a change of priority queue ordering to ordering by
fee. It results in none of the dependant transactions being considered
for inclusion in the new block template.
This prevents the ability for duplicate transactions to be added to the
mining priority heap when a transaction spends multiple outputs from the
same input transaction by converting the dependency tracking to use a
map keyed by the transaction hash instead of a linked list.
Make sure the peers.json file is actually written and read in unit test.
Also some minor edits:
- Capitalize Tor.
- Capitalize JSON.
- Proper capitalization at the beginning of a sentence.
Bite the bullet now and remove carriage return (`\r`) from all
documentation files to allow further edits without messy diffs.
No further changes have been made besides `sed -i 's/\r//g' docs/*`.
This adds the dcrd version info as an additional key to the JSON-RPC
getversion response to accompany the existing RPC API version
information and bumps the JSON-RPC minor version to account for the
backwards-compatible change.
This modifies the mining code to request the desired blocks from chain
by hash as opposed to using a separate function to get the top block and
removes the now unused plubming through block manager.
Not only is this faster because it avoids the need to go through block
manager, it is also more robust since it does not rely on the tip of the
chain not having changing for correctness.
This implements the getchaintips JSON-RPC and updates the associated
JSON-RPC API documentation accordingly.
It should be noted that until the entire block index is loaded into
memory, the chain tips tracking currently only works with tips that have
been seen since the daemon was started.
This modifies the startup logic to cache the tip and parent blocks in
the main chain block cache at initialization time which is desirable
because all of the transactions which come in to the mempool need access
to the these blocks to construct their utxo views due to the potential
for invalidation of the outputs in the previous block through voting.
Without caching them, the blocks end up being loaded from the database
again every time a new transaction enters the mempool until the blocks
eventually get cached when a new block is connected which is majorly
inefficient.
This PR moves the following block validation rule tests to
fullblocktests.
- `ErrBlockOneOutputs`
- Create a premine block with one premine output removed.
- Create a premine block with a bad spend script.
- Create a premine block with an incorrect pay to amount.
- `ErrVotesMismatch`
- Create block with a header that commits to more votes than the block
actually contains.
- `ErrIncongruentVotebit`
- Attempt to add block with incorrect votebits set.
- 4x Voters, 2x Nay 2x Yea, but block header says Yea
- 3x Voters, 2x Nay 1x Yea, but block header says Yea
- 3x Voters, 1x Nay 2x Yea, but block header says Nay
- `ErrSStxCommitment`
- Attempt to add block with a bad ticket purchase commitment.
- `ErrSSGenPayeeOuts`
- Attempt to add a block with a bad vote payee output.
- Attempt to add a block with an incorrect vote payee output amount.
- `ErrBadCoinbaseFraudProof`
- Create block with an invalid coinbase transaction.
- `ErrFraudBlockIndex`
- Create block with an invalid transaction block height.
- `ErrFraudAmountIn`
- Create block with an invalid transaction amount.
- `ErrExpiredTx`
- Create block with an expired transaction.
- `ErrBadBlockHeight`
- Create block with an invalid block height.
- `ErrPoolSize`
- Create block with an invalid ticket pool size.
- `ErrInvalidFinalState`
- Create block with an invalid final state.
- `ErrScriptMalformed`
- Create block with a malformed spend script.
This simplifies the recently added chain tip tracking by allowing the
block index to handle it when a node is added to it.
This is possible since the block index does not support nodes that do
not connect to an existing node (with the exception of the genesis
block), and thus all new nodes are either extending an existing chain or
are on a side chain, but in either case, are a new chain tip. In the
case the node is extending a chain, the parent is no longer a tip so it
can be removed.
This removes the unused threshold state serialization code and
associated tests that appear to be left over from initial development
but were never needed due to other caches used.
This simplifies, optimizes, and improves the robustness of the voter
version calculation as follows:
- Modifies calcVoterVersion to make use of the AncestorNode function to directly
iterate back to the end of the prevoius stake version interval instead
of manually adjusting and calling findStakeVersionPriorNode.
- Adds an optimization to skip any attempts to count votes before stake
validation height since there can't possibly be any votes prior to
that point.
- Modifies calcVoterVersionInterval to explicitly assert the assumptions
at the beginning of the function to prevent invalid behavior instead
of only commenting them and then indirectly asserting it via the total
number of found votes.
This modifies the TipGeneration function to use the new chain tip
functionality in order to determine the entire generation of blocks
stemming from the current tip as opposed to the children.
This adds logic to keep track of all known chain tips. Currently, this
will only work with chain tips that have been seen since the daemon was
loaded, however, when moving to the full block index in memory, it will
pick end up tracking all chain tips in the block index.
Also, note that nothing makes use of the functionality as of this commit
yet. It will be useful in the future to allow things such as removal of
the more expensive per-node children tracking, mining code improvements,
and enhanced fork-related interrogation ability.
This replaces the GetGeneration function which allowed getting the
entire generation (all children stemming from the same parent) of an
arbitrary bock with TipGeneration which only returns the same
information for the tip block.
This is being done because the function is only used for mining purposes
to get the generation of the current tip. The code is simplified by
reducing its scope to its actual purpose as an initial benefit. It also
provides much better optimization opportunities later.
This adds a new exported function named IsSolved which allows consumers
of chaingen to easily check if a block is solved according to the target
difficulty specified by the bits in its header.
This improves the test which checks for votes on the wrong block by
creating a new commitment script and testing 3 different scenarios:
- Ticket that commits to the parent of the correct block to ensure that
the code under test is not just failing due to a hash that doesn't
exist
- Ticket that commits to the correct block hash and wrong block height
- Ticket the commits to the correct block height and wrong block hash
Putting the test code in the same package makes it easier for forks
since they don't have to change the import paths as much and it also
gets rid of the need for internal_test.go to bridge.
This copies the minor bits of code referenced by chaingen from
blockchain to break the dependency on the blockchain package.
The intent of chaingen is that it is totally separate code from
the blockchain package so that accidental consensus changes are much
more difficult to slip through.
This modifies the chaingen test harness to error when a duplicate block
name is provided to NextBlock to help prevent misuse.
It also orrects a couple of comment typos.
This removes the calcStakeVersionByNode in favor of just using
calcStakeVersion directly since the latter already takes the node and
the former is a simple pass through with no additional handling.
This switches block template proposal checks over to use
CheckConnectBlock now that all of the core checks needed for block
templates is included there, and, more importantly, removes the the
BFDryRun flag since it was only used to test block template proposals
and it is much simpler to allow the main chain processing paths to run
without the complication of selectively processing code for dry run
behavior.
