This optimizes the way in which the mempool oprhan map is limited in the
same way the server block manager maps were previously optimized.
Previously the code would read a cryptographically random value large
enough to construct a hash, find the first entry larger than that value,
and evict it.
That approach is quite inefficient and could easily become a
bottleneck when processing transactions due to the need to read from a
source such as /dev/urandom and all of the subsequent hash comparisons.
Luckily, strong cryptographic randomness is not needed here. The primary
intent of limiting the maps is to control memory usage with a secondary
concern of making it difficult for adversaries to force eviction of
specific entries.
Consequently, this changes the code to make use of the pseudorandom
iteration order of Go's maps along with the preimage resistance of the
hashing function to provide the desired functionality. It has
previously been discussed that the specific pseudorandom iteration order
is not guaranteed by the Go spec even though in practice that is how it
is implemented. This is not a concern however because even if the
specific compiler doesn't implement that, the preimage resistance of the
hashing function alone is enough.
The following is a before and after comparison of the function for both
speed and memory allocations:
benchmark old ns/op new ns/op delta
----------------------------------------------------------------
BenchmarkLimitNumOrphans 3727 243 -93.48%
benchmark old allocs new allocs delta
-----------------------------------------------------------------
BenchmarkLimitNumOrphans 4 0 -100.00%
This moves the priority-related code from the mempool package to the
mining package and also exports a new constant named UnminedHeight which
takes the place of the old unexported mempoolHeight.
Even though the mempool makes use of the priority code to make decisions
about what it will accept, priority really has to do with mining since
it influences which transactions will end up into a block. This change
also has the side effect of being a step towards enabling separation of
the mining code into its own package which, as previously mentioned,
needs access to the priority calculation code as well.
Finally, the mempoolHeight variable was poorly named since what it
really represents is a transaction that has not been mined into a block
yet. Renaming the variable to more accurately reflect its purpose makes
it clear that it belongs in the mining package which also needs the
definition now as well since the priority calculation code relies on it.
This will also benefit an outstanding PR which needs access to the same
value.
This fixes a bug where a transaction would lose reference to other
transactions dependant on it when being considered for inclusion in a
new block template. The issue only occurs when the transaction being
considered triggers a change of priority queue ordering to ordering by
fee. It results in none of the dependant transactions being considered
for inclusion in the new block template.
This prevents the ability for duplicate transactions to be added to the
mining priority heap when a transaction spends multiple outputs from the
same input transaction by converting the dependency tracking to use a
map keyed by the transaction hash instead of a linked list.
Make sure the peers.json file is actually written and read in unit test.
Also some minor edits:
- Capitalize Tor.
- Capitalize JSON.
- Proper capitalization at the beginning of a sentence.
Bite the bullet now and remove carriage return (`\r`) from all
documentation files to allow further edits without messy diffs.
No further changes have been made besides `sed -i 's/\r//g' docs/*`.
This adds the dcrd version info as an additional key to the JSON-RPC
getversion response to accompany the existing RPC API version
information and bumps the JSON-RPC minor version to account for the
backwards-compatible change.
This modifies the mining code to request the desired blocks from chain
by hash as opposed to using a separate function to get the top block and
removes the now unused plubming through block manager.
Not only is this faster because it avoids the need to go through block
manager, it is also more robust since it does not rely on the tip of the
chain not having changing for correctness.
This implements the getchaintips JSON-RPC and updates the associated
JSON-RPC API documentation accordingly.
It should be noted that until the entire block index is loaded into
memory, the chain tips tracking currently only works with tips that have
been seen since the daemon was started.
This modifies the startup logic to cache the tip and parent blocks in
the main chain block cache at initialization time which is desirable
because all of the transactions which come in to the mempool need access
to the these blocks to construct their utxo views due to the potential
for invalidation of the outputs in the previous block through voting.
Without caching them, the blocks end up being loaded from the database
again every time a new transaction enters the mempool until the blocks
eventually get cached when a new block is connected which is majorly
inefficient.
This PR moves the following block validation rule tests to
fullblocktests.
- `ErrBlockOneOutputs`
- Create a premine block with one premine output removed.
- Create a premine block with a bad spend script.
- Create a premine block with an incorrect pay to amount.
- `ErrVotesMismatch`
- Create block with a header that commits to more votes than the block
actually contains.
- `ErrIncongruentVotebit`
- Attempt to add block with incorrect votebits set.
- 4x Voters, 2x Nay 2x Yea, but block header says Yea
- 3x Voters, 2x Nay 1x Yea, but block header says Yea
- 3x Voters, 1x Nay 2x Yea, but block header says Nay
- `ErrSStxCommitment`
- Attempt to add block with a bad ticket purchase commitment.
- `ErrSSGenPayeeOuts`
- Attempt to add a block with a bad vote payee output.
- Attempt to add a block with an incorrect vote payee output amount.
- `ErrBadCoinbaseFraudProof`
- Create block with an invalid coinbase transaction.
- `ErrFraudBlockIndex`
- Create block with an invalid transaction block height.
- `ErrFraudAmountIn`
- Create block with an invalid transaction amount.
- `ErrExpiredTx`
- Create block with an expired transaction.
- `ErrBadBlockHeight`
- Create block with an invalid block height.
- `ErrPoolSize`
- Create block with an invalid ticket pool size.
- `ErrInvalidFinalState`
- Create block with an invalid final state.
- `ErrScriptMalformed`
- Create block with a malformed spend script.
This simplifies the recently added chain tip tracking by allowing the
block index to handle it when a node is added to it.
This is possible since the block index does not support nodes that do
not connect to an existing node (with the exception of the genesis
block), and thus all new nodes are either extending an existing chain or
are on a side chain, but in either case, are a new chain tip. In the
case the node is extending a chain, the parent is no longer a tip so it
can be removed.
This removes the unused threshold state serialization code and
associated tests that appear to be left over from initial development
but were never needed due to other caches used.
This simplifies, optimizes, and improves the robustness of the voter
version calculation as follows:
- Modifies calcVoterVersion to make use of the AncestorNode function to directly
iterate back to the end of the prevoius stake version interval instead
of manually adjusting and calling findStakeVersionPriorNode.
- Adds an optimization to skip any attempts to count votes before stake
validation height since there can't possibly be any votes prior to
that point.
- Modifies calcVoterVersionInterval to explicitly assert the assumptions
at the beginning of the function to prevent invalid behavior instead
of only commenting them and then indirectly asserting it via the total
number of found votes.
This modifies the TipGeneration function to use the new chain tip
functionality in order to determine the entire generation of blocks
stemming from the current tip as opposed to the children.
This adds logic to keep track of all known chain tips. Currently, this
will only work with chain tips that have been seen since the daemon was
loaded, however, when moving to the full block index in memory, it will
pick end up tracking all chain tips in the block index.
Also, note that nothing makes use of the functionality as of this commit
yet. It will be useful in the future to allow things such as removal of
the more expensive per-node children tracking, mining code improvements,
and enhanced fork-related interrogation ability.
This replaces the GetGeneration function which allowed getting the
entire generation (all children stemming from the same parent) of an
arbitrary bock with TipGeneration which only returns the same
information for the tip block.
This is being done because the function is only used for mining purposes
to get the generation of the current tip. The code is simplified by
reducing its scope to its actual purpose as an initial benefit. It also
provides much better optimization opportunities later.
This adds a new exported function named IsSolved which allows consumers
of chaingen to easily check if a block is solved according to the target
difficulty specified by the bits in its header.
This improves the test which checks for votes on the wrong block by
creating a new commitment script and testing 3 different scenarios:
- Ticket that commits to the parent of the correct block to ensure that
the code under test is not just failing due to a hash that doesn't
exist
- Ticket that commits to the correct block hash and wrong block height
- Ticket the commits to the correct block height and wrong block hash
Putting the test code in the same package makes it easier for forks
since they don't have to change the import paths as much and it also
gets rid of the need for internal_test.go to bridge.
This copies the minor bits of code referenced by chaingen from
blockchain to break the dependency on the blockchain package.
The intent of chaingen is that it is totally separate code from
the blockchain package so that accidental consensus changes are much
more difficult to slip through.
This modifies the chaingen test harness to error when a duplicate block
name is provided to NextBlock to help prevent misuse.
It also orrects a couple of comment typos.
This removes the calcStakeVersionByNode in favor of just using
calcStakeVersion directly since the latter already takes the node and
the former is a simple pass through with no additional handling.
This switches block template proposal checks over to use
CheckConnectBlock now that all of the core checks needed for block
templates is included there, and, more importantly, removes the the
BFDryRun flag since it was only used to test block template proposals
and it is much simpler to allow the main chain processing paths to run
without the complication of selectively processing code for dry run
behavior.
This replaces and improves the tests for the ForceHeadReorganization
function with new tests based on the dynamic chaingen infrastructure
versus uses static test data that gets out of date as data structures
change and is much more difficult to update.
It also removes the now unused static reorgto180.bz2 test data.
This removes a couple of the reorganization tests that rely on static
test data since that logic is much better handled by the dynamic full
block tests and thus is no longer needed.
This modifies the newly added block index serialization code to use a
separate structure as opposed to working with block nodes directly.
This approach is more desirable because it provides better separation
and more robust code against changes to the block node structure itself
and it avoids complications dealing with non-serialized fields such as
parent and child pointers.
This modifies the forceHeadReorganization function to lookup the child
node directly from the index instead of looping through all of the
children looking for it.
This fixes the behavior of expanding ~ on Windows and macOS, as well
as adding support for ~otheruser to expand to the home directory of
otheruser.
Modify the default sample config to reflect the change in the meaning
of ~ on Windows, and the unintended meaning of ~ on macOS. Include
defaults for all 3 of the major operating system classes since
uncommenting the default Unix option on macOS or Windows would change
the blockchain data directory to a non-default location.
This adds code to migrate the existing block index in ffldb to the new
format managed by the blockchain package and updates the code to use the
new infrastructure.
