This udpates the txscript module to use v2 of the dcrutil module and v2
of the chaincfg module since dcrutil/v2 requires it.
While here, it also modifies the signing functions to accept the new
dcrutil.AddressParams instead of a pointer to a chaincfg.Params struct
in order to remove the tight coupling between txscript and chaincfg at
the API boundary.
It also updates the tests accordingly.
This removes the DefaultScriptVersion constant and updates the code and
tests to use version 0 scripts accordingly.
It is being removed because it is highly error prone since anything that
is working with scripts needs to understand the version it is dealing
with and having a constant that could change that version out from under
them could easily lead to buggy behavior.
This converts the CalcMultiSigStats function to make use of the new
extractMultisigScriptDetails function instead of the far less efficient
parseScript thereby significantly optimizing the function.
The tests are also updated accordingly.
The following is a before and after comparison of analyzing a standard
multisig script:
benchmark old ns/op new ns/op delta
---------------------------------------------------------------
BenchmarkCalcMultiSigStats 972 79.5 -91.82%
benchmark old allocs new allocs delta
---------------------------------------------------------------
BenchmarkCalcMultiSigStats 1 0 -100.00%
benchmark old bytes new bytes delta
---------------------------------------------------------------
BenchmarkCalcMultiSigStats 2304 0 -100.00%
This adds tests to ensure the isAnyKindOfScriptHash function properly
identifies the four stake-tagged pay-to-script-hash possibilities in
addition to ensuring they are not misidentified as standard
pay-to-script-hash scripts.
This tightens the multisig and pay-to-pubkey standard script
identification functions to use the same strict pubkey requirements as
the consensus rules since standardness rules are generally intended to
be more restrictive than the consensus rules which implies they are at a
minimum at least as restrictive.
The tests are also updated to deal with the additional restriction
accordingly.
This modifies the majority of the tests that make use of chain
parameters and the RPC tests to use the resurrected regression test
network.
It also bumps the affected module versions as follows:
- github.com/decred/dcrd/txscript@v1.0.2
- github.com/decred/dcrd/blockchain/stake@v1.0.3
- github.com/decred/dcrd/mempool@v1.0.2
The blockchain and dcrutil modules are also affected, but since their
version has already been bumped since their last release tags, they are
not bumped again.
This modifies the test code for several unit tests to prefer using
another network, such as mainnet or simnet, when not specifically
testing something that requires testnet.
In the cases where where testnet is required, it also now prefers a
local reference over directly accessing the testnet parameters directly
from the chaincfg package.
This is being done because the test network undergoes periodic resets
and by reducing the number of direct references, it simplifies the
process of resetting it.
This converts the majority of script errors from generic errors created
via errors.New and fmt.Errorf to use a concrete type that implements the
error interface with an error code and description.
This allows callers to programmatically detect the type of error via
type assertions and an error code while still allowing the errors to
provide more context.
For example, instead of just having an error the reads "disabled opcode"
as would happen prior to these changes when a disabled opcode is
encountered, the error will now read "attempt to execute disabled opcode
OP_FOO".
While it was previously possible to programmatically detect many errors
due to them being exported, they provided no additional context and
there were also various instances that were just returning errors
created on the spot which callers could not reliably detect without
resorting to looking at the actual error message, which is nearly always
bad practice.
Also, while here, export the MaxStackSize and MaxScriptSize constants
since they can be useful for consumers of the package and perform some
minor cleanup of some of the tests.
Putting the test code in the same package makes it easier for forks
since they don't have to change the import paths as much and it also
gets rid of the need for internal_test.go to bridge.
Also, do some light cleanup on a few tests while here.
This is a follow-up to a previous change which removed the ability to
create a dcrutil.Address from a hybrid pubkey. It removes all
secp256k1 hybrid pubkey support from the chainec/dcrec packages since
these pubkeys are not usable when evaluated by OP_CHECKSIG or
OP_CHECKMULTISIG.
While ParsePubKey is called by consensus code and its behavior has
been changed to error when encountering a hybrid secp256k1 pubkey,
this is not a consensus change as every use of ParsePubKey in
consensus code either checks beforehand that the pubkey is compressed
or uncompressed, requires a parsed address to be P2PKH or P2SH, or
checks an address string against the premine ledger (which only
contains P2PKH addresses).
This modifies the recently-added NullDataScript function in several
ways in an effort to make them more consistent with the tests in the
rest of the code base and improve/correct the logic:
- Use the hexToBytes and mustParseShortForm functions
- Consistently format the test errors
- Replace the valid bool flag with an expected error and test against it
- Ensure the returned script type is the expected type in all cases
This merge commit adds the following code from the
github.com/decred/dcrutil package into a new
github.com/decred/dcrd/dcrutil package:
* Address handling
* Amount type
* AppDataDir func
* bitflags functions
* Block wrapper type
* Hash160 func
* Tx wrapper type
* WIF type
as well as all tests for this code.
The old github.com/decred/dcrutil/hdkeychain package has also been
merged and moved to github.com/decred/dcrd/dcrutil/hdkeychain.
dcrd packages have been updated to use the new packages and the dep
files have been updated for this change.
decred's consensus rules require address generation
to use a compressed pubkey format, this was hacked
into NewAddressSecpPubKey. This commit removes the
hack and provides NewAddressSecpPubKeyCompressed to
enforce the consensus prerequisite
This modifies the recently-added NullDataScript function in several
ways in an effort to make them more consistent with the tests in the
rest of the code base and improve/correct the logic:
- Use the hexToBytes and mustParseShortForm functions
- Consistently format the test errors
- Replace the valid bool flag with an expected error and test against it
- Ensure the returned script type is the expected type in all cases
This adds a new function named NullDataScript to the txscript package that returns a provably-pruneable OP_RETURN script with the provided data. The function will return an error if the provided data is larger than the maximum allowed length for a nulldata script to be be considered standard.
Putting the test code in the same package makes it easier for forks
since they don't have to change the import paths as much and it also
gets rid of the need for internal_test.go to bridge.
Also, do some light cleanup on a few tests while here.
This corrects the isNullData standard transaction type test to work
properly with canonically-encoded data pushes. In particular, single
byte data pushes that are small integers (0-16) are converted to the
equivalent numeric opcodes when canonically encoded and the code failed
to detect them properly.
It also adds several tests to ensure that both canonical and
non-canonical nulldata scripts are recognized properly and modifies the
test failure print to include the script that failed.
This does not affect consensus since it is just a standardness check.
This commit is the first stage of several that are planned to convert
the blockchain package into a concurrent safe package that will
ultimately allow support for multi-peer download and concurrent chain
processing. The goal is to update btcd proper after each step so it can
take advantage of the enhancements as they are developed.
In addition to the aforementioned benefit, this staged approach has been
chosen since it is absolutely critical to maintain consensus.
Separating the changes into several stages makes it easier for reviewers
to logically follow what is happening and therefore helps prevent
consensus bugs. Naturally there are significant automated tests to help
prevent consensus issues as well.
The main focus of this stage is to convert the blockchain package to use
the new database interface and implement the chain-related functionality
which it no longer handles. It also aims to improve efficiency in
various areas by making use of the new database and chain capabilities.
The following is an overview of the chain changes:
- Update to use the new database interface
- Add chain-related functionality that the old database used to handle
- Main chain structure and state
- Transaction spend tracking
- Implement a new pruned unspent transaction output (utxo) set
- Provides efficient direct access to the unspent transaction outputs
- Uses a domain specific compression algorithm that understands the
standard transaction scripts in order to significantly compress them
- Removes reliance on the transaction index and paves the way toward
eventually enabling block pruning
- Modify the New function to accept a Config struct instead of
inidividual parameters
- Replace the old TxStore type with a new UtxoViewpoint type that makes
use of the new pruned utxo set
- Convert code to treat the new UtxoViewpoint as a rolling view that is
used between connects and disconnects to improve efficiency
- Make best chain state always set when the chain instance is created
- Remove now unnecessary logic for dealing with unset best state
- Make all exported functions concurrent safe
- Currently using a single chain state lock as it provides a straight
forward and easy to review path forward however this can be improved
with more fine grained locking
- Optimize various cases where full blocks were being loaded when only
the header is needed to help reduce the I/O load
- Add the ability for callers to get a snapshot of the current best
chain stats in a concurrent safe fashion
- Does not block callers while new blocks are being processed
- Make error messages that reference transaction outputs consistently
use <transaction hash>:<output index>
- Introduce a new AssertError type an convert internal consistency
checks to use it
- Update tests and examples to reflect the changes
- Add a full suite of tests to ensure correct functionality of the new
code
The following is an overview of the btcd changes:
- Update to use the new database and chain interfaces
- Temporarily remove all code related to the transaction index
- Temporarily remove all code related to the address index
- Convert all code that uses transaction stores to use the new utxo
view
- Rework several calls that required the block manager for safe
concurrency to use the chain package directly now that it is
concurrent safe
- Change all calls to obtain the best hash to use the new best state
snapshot capability from the chain package
- Remove workaround for limits on fetching height ranges since the new
database interface no longer imposes them
- Correct the gettxout RPC handler to return the best chain hash as
opposed the hash the txout was found in
- Optimize various RPC handlers:
- Change several of the RPC handlers to use the new chain snapshot
capability to avoid needlessly loading data
- Update several handlers to use new functionality to avoid accessing
the block manager so they are able to return the data without
blocking when the server is busy processing blocks
- Update non-verbose getblock to avoid deserialization and
serialization overhead
- Update getblockheader to request the block height directly from
chain and only load the header
- Update getdifficulty to use the new cached data from chain
- Update getmininginfo to use the new cached data from chain
- Update non-verbose getrawtransaction to avoid deserialization and
serialization overhead
- Update gettxout to use the new utxo store versus loading
full transactions using the transaction index
The following is an overview of the utility changes:
- Update addblock to use the new database and chain interfaces
- Update findcheckpoint to use the new database and chain interfaces
- Remove the dropafter utility which is no longer supported
NOTE: The transaction index and address index will be reimplemented in
another commit.
isMultiSig was not verifying the number of pubkeys specified matched
the number of pubkeys provided. This caused certain non-standard
scripts to be considered multisig scripts.
However, the script still would have failed during execution.
NOTE: This only affects whether or not the script is considered
standard and does NOT affect consensus.
Also, add a test for this check.
- Move reference tests to test package since they are intended to
exercise the engine as callers would
- Improve the short form script parsing to allow additional opcodes:
DATA_#, OP_#, FALSE, TRUE
- Make use of a function to decode hex strings rather than manually
defining byte slices
- Update the tests to make use of the short form script parsing logic
rather than manually defining byte slices
- Consistently replace all []byte{} and [][]byte{} with nil
- Define tests only used in a specific function inside that func
- Move invalid flag combination test to engine_test since that is what
it is testing
- Remove all redundant script tests in favor of the JSON-based tests in
the data directory.
- Move several functions from internal_test.go to the test files
associated with what the tests are checking
This commit moves all code related to standard scripts into a separate
file named standard.go as well as the associated tests into
standard_test.go. Since the code in address.go and address_test.go is
only related to standard scripts, it has been combined into the new
files and the old files deleted.
The intent here is to make it clear that the code in standard.go is not
related to consensus.
