This replaces the ErrDoubleSpend and ErrMissingTx error codes with a
single error code named ErrMissingTxOut and updates the relevant errors
and expected test results accordingly.
Once upon a time, the code relied on a transaction index, so it was able
to definitively differentiate between a transaction output that
legitimately did not exist and one that had already been spent.
However, since the code now uses a pruned utxoset, it is no longer
possible to reliably differentiate since once all outputs of a
transaction are spent, it is removed from the utxoset completely.
Consequently, a missing transaction could be either because the
transaction never existed or because it is fully spent.
Also, while here, consistently use the LookupEntry function on the
UtxoView instead of directly accessing the internal map as intended.
This modifies the newBlockNode function to accept the parent as an
argument to automatically connect the newly created node. When it is
not nil, the work sum will automatically be summed and the parent of the
new node will be set accordingly.
This simplifies the block node construction a bit and allows some
redundant code to be removed. It also paves the way for easier simpler
full block index construction in the future.
This separates the logic for populating the stake information in a block
node from the construction of the node in order to better delineate the
difference between the two and to pave the way for that information to
be stored separately in the database versus needing to load full blocks
to retrieve it.
This modifies the blockchain code to store all blocks that have passed
proof-of-work and contextual validity tests in the database even if they
may ultimately fail to connect.
This eliminates the need to store those blocks in memory, allows them to
be available as orphans later even if they were never part of the main
chain, and helps pave the way toward being able to separate the download
logic from the connection logic.
Note that it also updates the blockExists function since the code base
is currently in the process of changing over to decouple download and
connection logic, but not all of the necessary parts are updated yet,
to ensure blocks that are in the database, but do not have an associated
main chain block index entry, are treated as if they do not exist for
the purposes of chain connection and selection logic.
This moves the test for validating the revocations in the block are
actually eligible to be revoked per the results of the ticket lottery
into the checkBlockContext function where it more naturally belongs
since it is only dependent on the block and its contextual position
within the chain.
While here, it also refactors the check into a separate function named
checkAllowedRevocations to improve readability. Also, since this was
the final check in the CheckBlockStakeSanity function, that entire
function has been removed along with its call site.
This moves the test for validating the votes in the block are eligible
votes per the results of the ticket lottery into the checkBlockContext
function where it more naturally belongs since it is only dependent on
the block and its contextual position within the chain.
While here, it also refactors the check into a separate function named
checkAllowedVotes to improve readability.
This moves the check for validating that only ticket purchases, votes,
and revocations are included in the stake tree to the checkBlockSanity
function where it more naturally belongs since it does not require
access to any previous chain context.
This moves the check for validating that the number of revocations does
not exceed the maximum allowed to the checkBlockSanity function where it
more naturally belongs since it does not require access to any previous
chain context.
It should be noted that this check previously used math.MaxUint8 and was
rather implicitly limited by the fact the header uses a uint8 for the
field, so this makes the requirement more explicit by defining a
constant.
This moves the test for validating the final state specified by the
header matches the calculated final state into the
checkBlockHeaderContext function where it more naturally belongs since
it is only dependent on the header and its contextual position within
the chain.
This modifies the checkBlockHeaderSanity function to include an
additional check which ensures the final state is zero prior to stake
validation height.
It should be noted that this is technically a hard fork since before
this change it was possible to use any value for the final state prior
to stake validation height. However, no blocks on mainnet nor testnet
ever used non-zero final states for blocks in that range and they are
buried behind checkpoints in the minimum required version of the
software due to the previous consensus votes, so it isn't possible to
trigger a reorg that would fork old nodes off the network anyways, so it
is safe to retroactively apply this check.
This moves the checks for validating that votes commit to the parent of
the block they are contained in and the vote bits specified by the
header match the result of the actual votes included in the block to the
checkBlockSanity function where they more naturally belong since they do
not require access to any previous chain context.
This moves the test for validating the pool size specified by the header
matches the calculated size into the checkBlockHeaderContext function
where it more naturally belongs since it is only dependent on the header
and its contextual position within the chain.
This modifies the exported CheckConnectBlock function to call the
checkBlockSanity and checkBlockContext functions to ensure all
validation rules are enforced from the mining code.
Since both the mining code and tests typically work with unsolved
blocks, this also introduces a new parameter on CheckConnectBlock to
pass behavior flags which allows the caller to skip the proof of work
check.
It also modifies forceHeadReorganization to call checkBlockContext for
the same reason.
Finally, all tests and call sites are updated with the appropriate
flags and various tests are updated for the changes accordingly.
This introduces two new functions to the blockchain package named
headerApprovesParent and voteBitsApproveParent and updates all call
sites in the package in order to improve the readability and more
clearly describe the intention of the code.
This modifies the SSGenBlockVotedOn function to directly copy the hash
the vote commits to into a statically sized array versus calling the
hash function and thus can remove the potential runtime error.
This is preferred because the code will fail to compile if the size
of chainhash.HashSize is every changed which is desirable because it
would mean the assumptions the code makes would be invalidated.
It also updates all callers in the repository accordingly.
This moves the check for validating that there are only ticket purchases
included in blocks before stake validation height to checkBlockSanity
where is more naturally belongs since it does not require access to any
previous chain context.
This removes a duplicate check which ensures ticket purchases pay the
required amount. It is already checked by checkProofOfStake via
checkBlockSanity.
This cleans up the checkBlockSanity and make it more consistent with
the rest of the code base.
The following is an overview of the changes:
- Use the local variable throughout
- Check the header values against the expected value instead of the
reverse
- Make the errors more consistent and verbose in some cases
- Use an int64 for the type so consensus doesn't different between
computer architecture
- Correct and add some comments
This moves the checkBlockContext function to back to validate.go next to
checkBlockHeaderContext where it belongs to help minimize the
differences between the upstream code in order to facilitate easier
syncs due to less merge conflicts as a result of superfluous changes.
This moves the test for validating the block height specified by the
header matches the height it actually connects to the chain into the
checkBlockHeaderContext function where it more naturally belongs since
it is only dependent on the header and its contextual position within
the chain.
This moves the test for validating that the vote bits specified by the
header are a specific value before reaching stake validation height into
the checkBlockHeaderSanity function where it more naturally belongs
since it is solely dependent on the header.
This changes the validation that the block does not exceed the maximum
number of ticket purchases to make use of the value committed to by the
header and thus moves the test into checkBlockHeaderSanity accordingly.
This is acceptable since the code also validates the header commitment
for the number of ticket purchases later and therefore implies
correctness.
This changes the validation that the block does not contain any votes or
revocations before stake enabled height and stake validation height to
make use of the values committed to by the header and thus moves the
test into checkBlockHeaderSanity accordingly.
It also combines the two checks to only check before stake validation
height since that always comes after stake enabled height and adds an
assertion to ensure that optimization assumption holds.
This is acceptable since the code also validates the header commitments
for the number of votes and revocations later and therefore implies
correctness.
This moves the test for validating the stake difficulty specified by the
header matches the calculated difficulty into the
checkBlockHeaderContext function where it more naturally belongs since
it is only dependent on the header and its contextual position within
the chain.
Also, while here, it updates the error message to be more consistent
with existing error messages and does not incorrectly create a rule
error from an internal error in the unlikely case the difficulty fails
to be calculated.
This changes the validation that the block does not exceed the maximum
number of votes to make use of the value committed to by the header and
thus moves the test into checkBlockHeaderSanity accordingly.
This is acceptable since the code also validates the header commitment
for the number of votes later and therefore implies correctness.
This moves the test for validating the number of votes specified by the
header is at least the required minimum into the checkBlockHeaderSanity
function where it more naturally belongs since it is solely dependent on
information the header.
Also, remove the redundant check for the same condition from
checkBlockSanity since it has now already been checked according to the
value the header commits to and therefore validating the header
commitment for the number of votes implies correctness.
This modifies the checkBlockHeaderSanity function to accept the header
as intended so that it is only performing context-free checks. Since
checking the stake difficulty of all of the ticket purchases necessarily
requires the block, that check is moved into checkBlockSanity.
Also, while here, improve the comments on the {c,C}checkProofOfStake
functions. Note that having comments on an exported function that refer
to an unexported one is not very helpful since documentation generation
tools don't see unexported funcs.
This makes the semantics of block fetching much more explicit by
introducing a new function named fetchMainChainBlockByHash which only
attempts to the load a block from the main chain block cache and then
falls back to the database. While currently only blocks in the main
chain are in the database, the intention is for future commits to
allow the database to house side chain blocks as well, so having clearly
defined semantics will help make that transition easier to verify.
While here, it also renames {f,F}etchBlockFromHash to {f,F}etchBlockByHash
to be consistent with the naming used by the other existing functions
and updates the comments to better describe the function semantics.
This modifies the block connection code paths to accept the parent block
in addition to the block as a parameter versus loading it on demand in
the called functions. This makes the dependency more explicit and also
ensures it doesn't potentially get loaded multiple times in the reorg
case.
This refactors the block index logic into a separate struct and
introduces an individual lock for it so it can be queried independent of
the chain lock.
It also modifies the `newBlockNode` function to accept nil for the
ticket spend information parameter and updates all of the test code that
doesn't require it to use nil.
This modifies the IsSStx, IsSSGen, and IsSSRtx functions to only return
a bool and introduces CheckSStx, CheckSSGen, and CheckSSRtx to return
the actual error as needed by consensus.
This is being done because "is" functions are much nicer to use when
they don't return an error and the callers that use them almost never
care why they aren't of the type, they just want to determine if they
are. In the few cases where the caller does care, they can use of the
new check functions.
While here, also update the comments to call out what the more common
names for the transaction types are and to add comments to the test
functions for consistency.
Finally, it updates all callers in the repo accordingly.
This modifies the block node structure to include only the specifically
used fields, some of which in a more compact format, as opposed to
copying the entire header and updates all code and tests accordingly.
Not only is this a more efficient approach that helps pave the way for
future optimizations, it is also consistent with the upstream code which
helps minimize the differences to facilitate easier syncs due to less
merge conflicts.
In particular, since the merkle and stake roots, number of revocations,
size, nonce, and extradata fields aren't used currently, they are no
longer copied into the block node. Also, the block node already had a
height field, which is also in the header, so this change also removes
that duplication.
Another change is that the block node now stores the timestamp as an
int64 unix-style timestamp which is only 8 bytes versus the old
timestamp that was in the header which is a time.Time and thus 24 bytes.
It should be noted that future optimizations will very likely end up
adding most of the omitted header fields back to the block node as
individual fields so the headers can be efficiently reconstructed from
memory, however, these changes are still beneficial due to the ability
to decouple the block node storage format from the header struct which
allows more compact representations and reording of the fields for
optimal struct packing.
Ultimately, the need for the parent hash can also be removed, which will
save an additional 32 bytes which would not be possible without this
decoupling.
This adds a new function FindSpentTicketsInBlock that extracts all spent and
revoked tickets, plus the vote bits of a given block and uses that function
instead of the individual functions when loading a new node into the blockchain.
This improves startup time a bit, as the isSSGen, isSSRtx and determineTxType
are somewhat slow, as they need to decode the output script of the transactions.
Looping over the transactions once and doing a single test is faster.
Numbers for my computer:
```
| variation | mainnet | testnet |
| with patch | 23.7s | 42.6s |
| without patch | 35.9s | 118.0s |
```
This modifies the CheckProofOfWork function to accept a block header
instead of an entire block since there is no reason to force the caller
to have access to a full block when only the header is required.
This merge commit adds the following code from the
github.com/decred/dcrutil package into a new
github.com/decred/dcrd/dcrutil package:
* Address handling
* Amount type
* AppDataDir func
* bitflags functions
* Block wrapper type
* Hash160 func
* Tx wrapper type
* WIF type
as well as all tests for this code.
The old github.com/decred/dcrutil/hdkeychain package has also been
merged and moved to github.com/decred/dcrd/dcrutil/hdkeychain.
dcrd packages have been updated to use the new packages and the dep
files have been updated for this change.
This implements the agenda for voting on the LN features as defined in
DCP0002 and DCP0003 along with consensus tests to ensure its correctness.
The following is an overview of the changes:
- Generate new version blocks and reject old version blocks after a
super majority has been reached
- New block version on mainnet is version 5
- New block version on testnet is version 6
- Introduce a convenience function for determining if the vote passed
and is now active
- Introduce a new function for getting the consensus script verification
flags and setting the appropriate flags to enforce the new consensus
semantics in accordance with the state of the vote
- Enforce transaction finality checks based on the past median time in
accordance with the state of the vote
- Enforce relative time locks via sequence numbers in accordance with
the state of the vote
- Modify the more strict standardness checks (acceptance to the mempool
and relay) to enforce DCP0002 in accordance with the state of the vote
- It should be noted that the flag for DCP0003 is always set for
the more strict standardness checks because it is a soft fork while
the DCP0002 enforcement must depend on the result of the vote
because it is a hard fork
- Add tests for determining consensus script verification flags for both
mainnet and testnet
- Add tests for determining if the agenda is active for both mainnet and
testnet
This repurposes the sequence number of version 2 transaction inputs to
provide consensus-enforced relative lock-time semantics so that a
transaction can require inputs to have a specified relative age before
they are allowed to be included in a block. Each relative time lock can
either specify a relative number of seconds (with a granularity of 512
seconds and a maximum value of 33,553,920) or a specific number of
blocks (max 65535).
The number of seconds is calculated relative to the past median time of
the block before the one that contains the referenced output. This is
done because, due to other changes that will also be included in the
same agenda vote, said time will become the earliest possible time the
block that contains the referenced output could have been (technically
it will be one second after that, but that complexity is ignored since
there is already a granularity involved anyways).
It is also possible to disable the behavior by setting bit 31 of the
sequence number (which all transactions currently do by default since
they are set to the max).
In order for the transaction to be permitted to the mempool, relayed,
considered for inclusion into block templates, and allowed into a block,
the specified relative time locks for all of its inputs must be
satisfied.
This only implements the required logic and tests to enforce the new
behavior. Code to enforce the new behavior when considering candidate
transactions for acceptance to the mempool, relaying, and inclusion into
block templates will be added in a separate commit.
A consensus vote is required in order to reject blocks which contain
transactions that violate the new rules at a consensus level. Code to
selectively enable consensus enforcement based on the result of an
agenda vote will be added in a separate commit.
In order to accomplish this new behavior, the concept of a sequence lock
is introduced which allows the minimum possible time and height at which
a transaction can be included into a block to be calculated from all
inputs with non-disabled relative time locks, and functions to calculate
and evaluate the sequence lock are added.
The following is an overview of the changes:
- Introduce a new struct named SequenceLock to represent the previously
described sequence lock
- Define new constants related to sequence numbers named
SequenceLockTimeDisabled, SequenceLockTimeIsSeconds,
SequenceLockTimeMask, and SequenceLockTimeGranularity
- Add a new function named calcSequenceLock to calculate the sequence
lock for a given transaction
- Add a new function named SequenceLockActive to determine if a given
sequence lock is satisfied for a given block height and past median
time
- Add a convenience function named LockTimeToSequence which can be used
to convert a relative lock time to a sequence number
- Add a comprehensive set of tests for all of the new funcs
This implements a new stake difficulty algorithm along with a voting
agenda for all networks to change and a comprehensive set of tests. It
also implements estimation using the new algorithm for the
estimatestakediff RPC.
The following is an overview of the changes:
- Add new agenda to vote on changing the stake difficulty algorithm to
all networks
- The version on mainnet is version 4
- The version on testnet and simnet is version 5
- Modifies the stake difficulty calculation function to calculate the
difficulty based on the result of the vote
- Modifies the stake difficulty estimation function to calculate the
difficulty based on the result of the vote
- Makes the stake difficulty estimation function concurrent safe
- Calls it directly from the RPC server instead of going through block
manager
- Removes no longer needed code from the block manager
- Generate new version blocks and reject old version blocks after a
super majority has been reached
- New block version on mainnet is version 4
- New block version on testnet and simnet is version 5
- Add tests for the supply estimation used in the new algorithm
- Add tests for the new algorithm calculations
- Add tests for the estimation based on the new algorithm
This implements a new voting agenda for the testnet and simnet networks
for increasing the maximum block size from 1MiB to 1.25MB.
The following is an overview of the changes:
- Bump the maximum protocol block size limit to 1.25MB
- Bump the protocol version to 4
- Add wire tests for v3 protocol sizes and update tests for latest
- Update all wire values that are defined in terms of the max block size
to respect the protocol version
- Update the MaxSigOpsPerBlock constant to maintain existing value
regardless of the raised max protocol block size
- Decouple the maximum tx size from the block size by creating a chain
parameter for it with the current sizes so it is not a part of the
hard fork vote
- Add definition for new version 4 stake vote along with agenda to vote
on block size to the testnet and simnet networks
- Convert the MaximumBlockSize chain parameter to a slice in order to
hold the different possible sizes
- Adds a new function that returns the maximum block size based upon the
result of the vote
- Change the existing context-free block size check to enforce the max
protocol size and introduce a context-specific check which enforces a
restricted size based upon the network consensus parameters and the
result of the vote
- Set the max allowed size in generated block templates based upon the
network params and result of the vote
- Generate version 4 blocks and reject version 3 blocks after a super
majority has been reached
Port BIP0009 over from btcd. The following is an overview of the
changes:
- Add new configuration options to the chaincfg package which allows the
rule deployments to be defined per chain
- Implement code to calculate the threshold state as required by voting
- Use threshold state caches that are stored to the database in order
to accelerate startup time
- Remove caches that are invalid due to definition changes in the
params including additions, deletions, and changes to existing
entries
- Verify that PoW and PoS majorities have been reached.
- Add tests for new error type
- Add tests for threshold state
- Deployments are per stake version.
- Add human readable vote/choice infrastructure.
- Add RPC command to interrogate vote status (getvoteinfo).
This removes a couple of checks that are completely impossible to hit.
These are impossible to hit because there is a maximum protocol limit of
1MiB and the absolute minimum possible transaction size that will
successfully deserialize with no inputs and no outputs is 15 bytes
(which is itself an invalid transaction anyways), which means a maximum
of 66,666 of these invalid transactions could fit into a 1MiB block.
Since the check is comparing against 393,216 on mainnet and 1,000,000 on
testnet and simnet, it can easily be seen it simply isn't possible to
hit.
Technically, if it were possible to hit these conditions, changing them
would be a hard fork, however, since they're impossible to hit, this is
effectively a noop change in regards to consensus.
