This updates the txscript module dependencies, the copyright year in the
files modified since the previous release, and serves as a base for
txscript/v4.0.0.
The updated direct dependencies in this commit are as follows:
- github.com/decred/dcrd/chaincfg/chainhash@v1.0.3
- github.com/decred/dcrd/chaincfg/v3@v3.1.0
- github.com/decred/dcrd/dcrec/edwards/v2@v2.0.2
- github.com/decred/dcrd/dcrec/secp256k1/v4@v4.0.1
- github.com/decred/dcrd/wire@v1.5.0
- github.com/decred/slog@v1.2.0
The full list of updated direct dependencies since the previous
txscript/v3.0.0 release are as follows:
- github.com/dchest/siphash@v1.2.2
- github.com/decred/base58@v1.0.3
- github.com/decred/dcrd/chaincfg/chainhash@v1.0.3
- github.com/decred/dcrd/chaincfg/v3@v3.1.0
- github.com/decred/dcrd/crypto/blake256@v1.0.0
- github.com/decred/dcrd/dcrec/edwards/v2@v2.0.2
- github.com/decred/dcrd/dcrec/secp256k1/v4@v4.0.1
- github.com/decred/dcrd/wire@v1.5.0
- github.com/decred/slog@v1.2.0
The following direct dependencies are no longer required as compared to
the previous txscript/v3.0.0 release:
- github.com/decred/dcrd/dcrutil/v3
Finally, all modules in the repository that depend on txscript are
tidied to ensure they are updated to use the latest versions hoisted
forward as a result.
This removes ErrTooMuchNullData and its associated stringer test since
it is no longer used by anything in the repository.
This is part of a series of commits to remove all code related to
standard scripts from txscript.
This removes ErrTooManyRequiredSigs and its associated stringer test
since it is no longer used by anything in the repository.
This is part of a series of commits to remove all code related to
standard scripts from txscript.
This removes ErrNotMultisigScript and its associated stringer test since
it is no longer used by anything in the repository.
This is part of a series of commits to remove all code related to
standard scripts from txscript.
This updates the index subscriber catchup function to update
subscriptions directly with notification instead of going through the
Run lifecycle process which avoids a case of receiving out of sync
index notifications while catch up is in progress.
Multiple typos in error type tests and review issues have also been
fixed in this commit.
This is based on https://proposals.decred.org/proposals/c96290a but was
modified in order to deal with realities that were unknown at the time
of the specification draft.
It is large and could not really be broken apart due to the pervasive
use of the isTreasuryEnabled flag. It was primarily authored by
* Marco Peereboom <marco@peereboom.us>
* Dave Collins <davec@conformal.com>
* Matheus Degiovani <opensource@matheusd.com>
With additional contributions from
* Donald Adu-Poku <donald.adu@gmail.com>
* Jamie Holdstock <jholdstock@decred.org>
Major changes:
* Add decentralized treasury agenda, as specified in DCP0006, to all supported
nets.
* Add functions to determine if the decentralized treasury agenda is active at
given block.
* Add new opcode OP_TADD that is a nop in txscript but is used to tag scripts
that credit the treasury account. This opcode is overloaded for treasurybase
and for normal transactions.
* Add new opcode OP_TSPEND that is a nop in txscript but is used to tag scripts
that debit the treasury account.
* Add new opcode OP_TGEN that is a nop in txscript but is used to tag P2PKH and
P2SH outputs in a TSpend transaction.
* Add functions that detect if a transaction is a valid TAdd, TSpend
or treasurybase transaction.
* Add error codes that return specific treasurybase/TAdd/TSpend consensus
violations.
* Modify countSpentOutputs to deal with treasury opcodes accordingly.
* Modify indexBlock to skip treasury transactions that do not have inputs.
* Add IsTreasuryEnabled call to ChainQueryer interface.
* Add treasury logger for debugging and logging the decentralized treasury
subsystem.
* Add IsTreasuryActive flag to BlockConnectedNtfnsData and
BlockDisconnectedNtfnsData.
* Modify OP_SSGEN to allow an optional output that contains votes for a TSpend
transaction hash.
* Add function that returns TSpend votes from an SSGen transaction.
* Modify CalcStakeVoteSubsidy so that treasurybase, unlike coinbase, is always
awarded the full percentage of the assigned block reward.
* Add helper functions to do all TSpend math so that callers don't roll their
own.
* Modify IsCoinBaseTx to not mistake a TSpend transaction as a coinbase.
* Add checkTreasuryBase function that verifies that a treasurybase is properly
constructed and pays the right amount to the treasury account.
* Add functions to calculate treasury balance for the provided block hash/node.
* Add function that verifies if a TSpend has a valid signature.
* Add functions to determine if a TSpend is not overspending.
* Add function to determine if a TSpend has been mined on the provided chain.
* Add functions that count and verifies treasury spend votes.
* Modify connectTransaction and disconnectTransactions to deal with the various
treasury transactions.
* Split CheckTransactionSanity in two functions
checkTransactionSanityContextFree and checkTransactionSanityContextual. This
is done in order to keep the decentralized treasury, which is always
contextual, from infecting the context free checks.
* Modify checkTransactionSanityContextual to recognize and verify treasury
transactions.
* Modify CheckTransactionSanity to deal with treasury transactions.
* Split checkBlockSanity in two functions checkBlockSanityContextFree and
checkBlockSanityContextual. This is done in order to keep the decentralized
treasury, which is always contextual, from infecting the context free checks.
* Modify checkBlockSanityContextual to enforce treasurybase and TAdd consensus
checks.
* Modify checkBlockPositional by unindenting it and adding TSpend consensus
enforcement.
* Modify checkCoinbaseUniqueHeightWithAddress to deal with the removal of the
project subsidy from output 0.
* Add checkCoinbaseUniqueHeightWithTreasuryBase that verifies coinbase and
treasurybase in the provided block.
* Unindent checkBlockContext.
* Modify checkTicketRedeemerCommitments and checkVoteInputs to deal with
potential tspend votes.
* Modify CheckTransactionInputs to skip treasurybase transactions.
* Modify CheckTransactionInputs to deal with TSpend transactions. Ensure the
provided Pi key is valid and that the signature is valid for the transaction.
Ensure that treasury TAdd and TSpend transaction utxo can only be spent after
coinbase maturity.
* Modify CountSigOps to deal with treasury transactions.
* Modify CountP2SHSigOps to deal with treasury transactions.
* Modify getStakeTreeFees to skip treasury transactions. Modify
totalOutputs to subtract ValueIn 0 for TSpend and treasurybase transactions.
* Modify checkTransactionsAndConnect to deal with modified amounts.
* Add tspendChecks function that verifies an entire TSpend transaction
validity at the point of the provided block. It ensures a TSpend is on a TVI.
It ensures the TSpend is in the valid window. It verifies that a TSpend In
and Out amounts match. It ensures a TSpend has the ValueIn amount encoded in
the OP_RETURN in Out 0. It ensures a TSpend has not been mined before on this
chain. It ensures a TSpend has the requisite votes. It ensures a TSpend is
not overspending.
* Modify checkConnectBlock to call checkTreasuryBase and tspendChecks when
treasury agenda is active.
* Add two tables to the database. Table "treasury" records the balance as of
this block and balance changes that occurred in this block which will become
active in CoinbaseMaturity blocks. Table "tspend" records all block hashes
where a TSpend has been mined this is to detect forks and prevent a Tspend
from being mined more than once.
* Modify handleBlockchainNotification to communicate if the treasury agenda is
active and skip treasurybase transaction when needed.
* Add various Treasury parameters to chaincfg params.
* Add hardcoded Tspend signatures in dcr_tmux_simnet_setup.sh.
* Add notifytspend and stoptspend calls to the RPC server. notifytspend
notifies the mempool when a TSpend transaction arrives.
* Modify commit filters V2 to recognize TAdd and TSpend transactions. It was
possible to modify V2 instead of introducing V3 because nothing changes from
the viewpoint of the wallet and treasury opcodes are disallowed prior to
agenda activation.
* Modify AddMemPoolTransaction to skip TSpend transactions that would throw the
fee estimator off.
* Add IsTreasuryAgendaActive, OnTSpendReceived and TSpendMinedOnAncestor to
mempool.Config in order to reject/accept TSpends in the mempool.
* Modify checkPoolDoubleSpend to ignore treasurybase.
* Modify mempool.maybeAcceptTransaction to enforce treasury standardness rules.
Don't allow TSpend transactions prior to stake validation height. Skip
treasurybase and tspend transactions in the orphan test. Ensure a tspend is
in a valid window. Ensure not more than 7 TSpends are active in the mempool.
Ensure TSpend has a well-known Pi key. Ensure The provided Pi key was used to
sign the transaction. Ensure TSpend was not mined in an ancestor block.
Notify subscribers that a valid TSpend was received.
* Add standardCoinbaseOpReturn and standardTreasurybaseOpReturn to create an
OP_RETURN followed by a data push that little endian encodes the height of
the block. Then there are a number of random bytes to ensure that the
transaction hash is always random.
* Modify createCoinbaseTx to create a coinbase that is valid when treasury is
enabled or not. Additionally, alter the transaction version if treasury is
enabled.
* Add createTreasuryBaseTx that creates a standard treasurybase.
* Modify maybeInsertStakeTx to recognize treasurybase and TSpend transactions.
* Modify handleTooFewVoters to call createTreasuryBaseTx when the treasury
agenda is active. Skip copying treasurybase.
* Modify NewBlockTemplate to recognize and deal with treasury transactions.
Skip TSpend transaction if block is not a TVI. Skip TSpend transaction if it
is not in the proper window. Skip TSpend transaction if a TSpend does not
have enough yes votes. Skip TSpend transaction if it overspends the treasury
account. Skip TAdd if there are more than 20 TAdds in the block. Create
treasurybase if required. Insert valid TAdd/TSpend transactions into stake
tree.
* Add TreasuryBalance and IsTreasuryAgendaActive to rpcserver Chain interface.
* Add gettreasurybalance, sendfromtreasury and sendtotreasury calls to RPC
server.
* Add notifytspend and stopnotifytspend to RPC websocket commands.
* Add simnet miner to generate large number of blocks during rpctests without
triggering PoW difficulty increases. This is used to verify various treasury
and tspend conditions during CI/CT.
* Modify RPC voting wallet to also vote on TSpends.
* Add json tests to verify all new opcodes and corner cases in the script
engine.
* Modify isStakeOpcode to recognize treasury opcodes.
* Modify countSigOpsV0 to count TSpends.
* Modify handleStakeOutSign to deal with TSpends.
* Modify SignTxOutput to recognize TSpends.
* Add TSpendSignatureScript that signs a TSpend transaction.
* Add TreasuryAddTy and TreasurySpendTy types to the standard scripts.
* Add isTreasuryAddScript and isTreasurySpendScript functions that recognize
a form of TAdd and TSpend transactions.
* Modify ExtractPkScriptAddrs to deal with TAdd and TSpend outputs.
* Add TxVersionSeqLock = 2 and TxVersionTreasury = 3 to wire. This is
used to discriminate between treasury and non-treasury scripts.
* Rig up all functions that need the isTreasuryEnabledflag directly or
indirectly.
* Shuffle various functions around and export them when they were needed to be
called from other packages.
* Added and modified numerous tests to verify (hopefully) all corner cases that
the decentralized treasury agenda has added.
This updates the txscript package to provide support for the errors.Is
and errors.As functions introduced in Go 1.13 per the recently
documented best practices.
The following is an overview of the changes:
- Rename the ErrorCode type to ErrorKind and make it a string
- Implement the error interface on ErrorKind
- Update all error code definitions to the new type
- Remove error code string map that is no longer required
- Remove unused ErrInternal definition
- Change the error code field of the Error type to Err
- Implement Unwrap on the Error type to support unwrapping via
errors.Is/As
- Remove the IsErrorCode function since it is no longer needed due to
errors.Is
- Update IsDERSigError to cope with the changes
- Modify various comments to refer to error kinds instead of codes
- Update all test code to use the new ErrorKind directly along with
errors.Is for detecting the expect errors
- Add full test coverage to ensure the new error definitions work as
intended
- Update the doc.go errors section accordingly
This implements an efficient and zero-allocation script tokenizer that
is exported to both provide a new capability to tokenize scripts to
external consumers of the API as well as to serve as a base for
refactoring the existing highly inefficient internal code.
It is important to note that this tokenizer is intended to be used in
consensus critical code in the future, so it must exactly follow the
existing semantics.
The current script parsing mechanism used throughout the txscript module
is to fully tokenize the scripts into an array of internal parsed
opcodes which are then examined and passed around in order to implement
virtually everything related to scripts.
While that approach does simplify the analysis of certain scripts and
thus provide some nice properties in that regard, it is both extremely
inefficient in many cases, and makes it impossible for external
consumers of the API to implement any form of custom script analysis
without manually implementing a bunch of error prone tokenizing code or,
alternatively, the script engine exposing internal structures.
For example, as shown by profiling the total memory allocations of an
initial sync, the existing script parsing code allocates a total of
around 295.12GB, which equates to around 50% of all allocations
performed. The zero-alloc tokenizer this introduces will allow that to
be reduced to virtually zero.
The following is a before and after comparison of tokenizing a large
script with a high opcode count using the existing code versus the
tokenizer this introduces for both speed and memory allocations:
benchmark old ns/op new ns/op delta
------------------------------------------------------------
BenchmarkScriptParsing 153099 961 -99.37%
benchmark old allocs new allocs delta
------------------------------------------------------------
BenchmarkScriptParsing 1 0 -100.00%
benchmark old bytes new bytes delta
------------------------------------------------------------
BenchmarkScriptParsing 466945 0 -100.00%
The following is an overview of the changes:
- Introduce new error code ErrUnsupportedScriptVersion
- Implement zero-allocation script tokenizer
- Add a full suite of tests to ensure the tokenizer works as intended
and follows the required consensus semantics
- Add an example of using the new tokenizer to count the number of
opcodes in a script
- Update README.md to include the new example
- Update script parsing benchmark to use the new tokenizer
This introduces a new error named ErrCheckSigAltVerify and modifies the
opcodeCheckSigAltVerify handler to use the abstractVerify function along
with the new error. This makes the handling consistent with all other
signature checking verification opcode handlers and ensures the error
both can be programmatically detected as well as be uniquely identified
as compared to a generic verify failure.
This removes the ScriptBip16 flag from the txscript package, changes the
default semantics to always enforce its behavior, and updates all
callers in the repository accordingly.
This change is being made to simplify the script engine code since the
flag has always been active and required by consensus in Decred, so there is
no need to require a flag to conditionally toggle it.
Also, since it is no longer possible to invoke the script engine without
the flag with the clean stack flag, it removes the now unused
ErrInvalidFlags error and associated tests.
It should be noted that the test removed from script_tests.json
specifically dealt with ensuring a signature script that contained
non-data-pushing opcodes was successful when neither the ScriptBip16 or
ScriptVerifySigPushOnly flags were set. Therefore, it is no longer
necessary.
Finally, the P2SH indicator to enable the flag in the test data has been
retained for now in order to keep the logic changes separate.
This converts the majority of script errors from generic errors created
via errors.New and fmt.Errorf to use a concrete type that implements the
error interface with an error code and description.
This allows callers to programmatically detect the type of error via
type assertions and an error code while still allowing the errors to
provide more context.
For example, instead of just having an error the reads "disabled opcode"
as would happen prior to these changes when a disabled opcode is
encountered, the error will now read "attempt to execute disabled opcode
OP_FOO".
While it was previously possible to programmatically detect many errors
due to them being exported, they provided no additional context and
there were also various instances that were just returning errors
created on the spot which callers could not reliably detect without
resorting to looking at the actual error message, which is nearly always
bad practice.
Also, while here, export the MaxStackSize and MaxScriptSize constants
since they can be useful for consumers of the package and perform some
minor cleanup of some of the tests.
