This refactors the consensus code which extracts the null data from the
coinbase from txscript.ExtractCoinbaseNullData so that it is performed
directly in the validation code where it more properly belongs.
The only reason the extraction was previously done in txscript is
because it was not possible to parse scripts outside of it, but that is
no longer the case now that txscript offers an exported tokenizer for
that purpose.
The extraction code is ever so slightly more efficient now that it no
longer needs to be as generic since it now has direct knowledge of the
conditions that need to be handled.
Great care was taken to ensure the semantics are not changed while
refactoring the code and no additional tests are added in this commit
because all of the conditions and code paths are covered by the tests
recently added to the full block tests.
While here, also perform some related code cleanup in the function and
improve the error messages .
Since the txscript.ExtractCoinbaseNullData is no longer necessary, this
deprecates the function and releated error code and constant so they can
be removed in the next major version of txscript.
Finally, since this relies on the script tokenizer which is not yet in a
released version of the txscript module, bump the requirement to include
an as yet unreleased version of txscript to ensure the next time the
blockchain module is released, it will require a newer version of
txscript to be released first.
This converts the ExtractCoinbaseNullData function to make use of the
new tokenizer instead of the far less efficient parseScript thereby
significantly optimizing the function.
The following is a before and after comparison of analyzing a typical
coinbase script:
benchmark old ns/op new ns/op delta
-------------------------------------------------------------------
BenchmarkExactCoinbaseNullData 227 31.0 -86.34%
benchmark old allocs new allocs delta
-------------------------------------------------------------------
BenchmarkExactCoinbaseNullData 1 0 -100.00%
benchmark old bytes new bytes delta
-------------------------------------------------------------------
BenchmarkExactCoinbaseNullData 448 0 -100.00%
This converts the majority of script errors from generic errors created
via errors.New and fmt.Errorf to use a concrete type that implements the
error interface with an error code and description.
This allows callers to programmatically detect the type of error via
type assertions and an error code while still allowing the errors to
provide more context.
For example, instead of just having an error the reads "disabled opcode"
as would happen prior to these changes when a disabled opcode is
encountered, the error will now read "attempt to execute disabled opcode
OP_FOO".
While it was previously possible to programmatically detect many errors
due to them being exported, they provided no additional context and
there were also various instances that were just returning errors
created on the spot which callers could not reliably detect without
resorting to looking at the actual error message, which is nearly always
bad practice.
Also, while here, export the MaxStackSize and MaxScriptSize constants
since they can be useful for consumers of the package and perform some
minor cleanup of some of the tests.
Standardness rules are not the same as consensus rules and it is
important to keep clear separation between them, because standardness
rules can and do change, while the consensus rules must not change
without a specific vote, and even then, the old rules must be kept
around for validation of all blocks prior to any changes introduced by a
successful vote.
Prior to this commit, the blockchain code which enforces the consensus
rule that requires the second output of the coinbase transaction to
contain the height of the block in a provably pruneable nulldata script
push was relying on code in txscript/standard.go, which, as the name of
the file suggests, is specifically intended for code related to
standardness checks.
This introduces a new function in the txscript package named
ExtractCoinbaseNullData which does not rely on any of the code related
to standardness checks and modifies the blockchain code accordingly to
make use of the new function instead. It also removes the
standardness-dependent GetNullDataContent function which is no longer
used by anything.
Finally, it adds tests in the txscript package to ensure the new
function has the required semantics.
This modifies the script engine to replace OP_NOP3 with
OP_CHECKSEQUENCEVERIFY and adds a flag to selectively enable its
enforcement.
The new opcode examines the top item on the stack and compares it
against the sequence number of the associated transaction input in order
to allow scripts to conditionally enforce the inclusion of relative time
locks to the transaction.
The following is an overview of the changes:
- Introduce a new flag named ScriptVerifyCheckSequenceVerify to
provide conditional enforcement of the new opcode
- Introduce a constant named OP_CHECKSEQUENCEVERIFY which has the same
value as OP_NOP3 since it is replacing it
- Update opcode to name mappings accordingly
- Abstract the logic that deals with time lock verification since it is
the same for both the new opcode and OP_CHECKLOCKTIMEVERIFY
- Implement the required opcode semantics
- Add tests to ensure the opcode works as expected including when used
both correctly and incorrectly
This reverts the changes related to the CheckSequenceVerify opcode that
were merged from upstream since additional changes are needed and it's
much cleaner to implement all of code related to the sequence locks in
the same PR which will be referenced by the DCP as opposed to being
split up in multiple.
Upstream commit a6bf1d9850.
The merge commit modifies all of the encoded transactions in the test
data to use Decred native format and contains some other minor
modifications necessary to integrate with Decred.
This commit is the first stage of several that are planned to convert
the blockchain package into a concurrent safe package that will
ultimately allow support for multi-peer download and concurrent chain
processing. The goal is to update btcd proper after each step so it can
take advantage of the enhancements as they are developed.
In addition to the aforementioned benefit, this staged approach has been
chosen since it is absolutely critical to maintain consensus.
Separating the changes into several stages makes it easier for reviewers
to logically follow what is happening and therefore helps prevent
consensus bugs. Naturally there are significant automated tests to help
prevent consensus issues as well.
The main focus of this stage is to convert the blockchain package to use
the new database interface and implement the chain-related functionality
which it no longer handles. It also aims to improve efficiency in
various areas by making use of the new database and chain capabilities.
The following is an overview of the chain changes:
- Update to use the new database interface
- Add chain-related functionality that the old database used to handle
- Main chain structure and state
- Transaction spend tracking
- Implement a new pruned unspent transaction output (utxo) set
- Provides efficient direct access to the unspent transaction outputs
- Uses a domain specific compression algorithm that understands the
standard transaction scripts in order to significantly compress them
- Removes reliance on the transaction index and paves the way toward
eventually enabling block pruning
- Modify the New function to accept a Config struct instead of
inidividual parameters
- Replace the old TxStore type with a new UtxoViewpoint type that makes
use of the new pruned utxo set
- Convert code to treat the new UtxoViewpoint as a rolling view that is
used between connects and disconnects to improve efficiency
- Make best chain state always set when the chain instance is created
- Remove now unnecessary logic for dealing with unset best state
- Make all exported functions concurrent safe
- Currently using a single chain state lock as it provides a straight
forward and easy to review path forward however this can be improved
with more fine grained locking
- Optimize various cases where full blocks were being loaded when only
the header is needed to help reduce the I/O load
- Add the ability for callers to get a snapshot of the current best
chain stats in a concurrent safe fashion
- Does not block callers while new blocks are being processed
- Make error messages that reference transaction outputs consistently
use <transaction hash>:<output index>
- Introduce a new AssertError type an convert internal consistency
checks to use it
- Update tests and examples to reflect the changes
- Add a full suite of tests to ensure correct functionality of the new
code
The following is an overview of the btcd changes:
- Update to use the new database and chain interfaces
- Temporarily remove all code related to the transaction index
- Temporarily remove all code related to the address index
- Convert all code that uses transaction stores to use the new utxo
view
- Rework several calls that required the block manager for safe
concurrency to use the chain package directly now that it is
concurrent safe
- Change all calls to obtain the best hash to use the new best state
snapshot capability from the chain package
- Remove workaround for limits on fetching height ranges since the new
database interface no longer imposes them
- Correct the gettxout RPC handler to return the best chain hash as
opposed the hash the txout was found in
- Optimize various RPC handlers:
- Change several of the RPC handlers to use the new chain snapshot
capability to avoid needlessly loading data
- Update several handlers to use new functionality to avoid accessing
the block manager so they are able to return the data without
blocking when the server is busy processing blocks
- Update non-verbose getblock to avoid deserialization and
serialization overhead
- Update getblockheader to request the block height directly from
chain and only load the header
- Update getdifficulty to use the new cached data from chain
- Update getmininginfo to use the new cached data from chain
- Update non-verbose getrawtransaction to avoid deserialization and
serialization overhead
- Update gettxout to use the new utxo store versus loading
full transactions using the transaction index
The following is an overview of the utility changes:
- Update addblock to use the new database and chain interfaces
- Update findcheckpoint to use the new database and chain interfaces
- Remove the dropafter utility which is no longer supported
NOTE: The transaction index and address index will be reimplemented in
another commit.
This change moves IsFinalizedTransaction to txscript and also changes
the first argument to take a wire.MsgTx instead of btcutil.Tx. This
is needed for an upcoming diff in which txscript will require
IsFinalizedTransaction and we do not want to import the btcd/blockchain.
