This udpates the txscript module to use v2 of the dcrutil module and v2
of the chaincfg module since dcrutil/v2 requires it.
While here, it also modifies the signing functions to accept the new
dcrutil.AddressParams instead of a pointer to a chaincfg.Params struct
in order to remove the tight coupling between txscript and chaincfg at
the API boundary.
It also updates the tests accordingly.
This completes the process of converting the typeOfScript function to
use a combination of raw script analysis and the new tokenizer instead
of the far less efficient parsed opcodes.
In particular, it converts the detection of stake change scripts to use
raw script analysis by introducing a new function named
isStakeChangeScript which makes use of the recently added
extractStakePubKeyHash and extractStakeScriptHash functions and removes
the script parsing fallback from the typeOfScript function since this is
the final case.
The following is a before and after comparison of analyzing a large
script for both the stake change script change and the overall
GetScriptClass function which relies on the now fully converted
typeOfScript function:
benchmark old ns/op new ns/op delta
-----------------------------------------------------------------
BenchmarkIsStakeChangeScript 133810 4.39 -100.00%
BenchmarkGetScriptClass 145001 62.9 -99.96%
benchmark old allocs new allocs delta
-----------------------------------------------------------------
BenchmarkIsStakeChangeScript 1 0 -100.00%
BenchmarkGetScriptClass 1 0 -100.00%
benchmark old bytes new bytes delta
-----------------------------------------------------------------
BenchmarkIsStakeChangeScript 466944 0 -100.00%
BenchmarkGetScriptClass 466944 0 -100.00%
This continues the process of converting the typeOfScript function to
use a combination of raw script analysis and the new tokenizer instead
of the far less efficient parsed opcodes.
In particular, it converts the detection of stake revocation scripts to
use raw script analysis.
In order to accomplish this, it introduces a new function named
isStakeGenScript which makes of the recently added
extractStakePubKeyHash and extractStakeScriptHash functions.
The following is a before and after comparison of analyzing a large
script:
benchmark old ns/op new ns/op delta
---------------------------------------------------------------------
BenchmarkIsStakeRevocationScript 117699 4.58 -100.00%
benchmark old allocs new allocs delta
---------------------------------------------------------------------
BenchmarkIsStakeRevocationScript 1 0 -100.00%
benchmark old bytes new bytes delta
---------------------------------------------------------------------
BenchmarkIsStakeRevocationScript 466944 0 -100.00%
This continues the process of converting the typeOfScript function to
use a combination of raw script analysis and the new tokenizer instead
of the far less efficient parsed opcodes.
In particular, it converts the detection of stake generation scripts to
use raw script analysis.
In order to accomplish this, it introduces a new function named
isStakeGenScript which makes of the recently added
extractStakePubKeyHash and extractStakeScriptHash functions.
The following is a before and after comparison of analyzing a large
script:
benchmark old ns/op new ns/op delta
---------------------------------------------------------------------
BenchmarkIsStakeGenerationScript 121043 4.26 -100.00%
benchmark old allocs new allocs delta
---------------------------------------------------------------------
BenchmarkIsStakeGenerationScript 1 0 -100.00%
benchmark old bytes new bytes delta
---------------------------------------------------------------------
BenchmarkIsStakeGenerationScript 466944 0 -100.00%
This continues the process of converting the typeOfScript function to
use a combination of raw script analysis and the new tokenizer instead
of the far less efficient parsed opcodes.
In particular, it converts the detection of stake submission scripts to
use raw script analysis.
In order to accomplish this, it introduces three new functions. The first
one is named extractStakePubKeyHash and works with the raw script bytes
to simultaneously determine if the script is a stake-tagged
pay-to-pubkey-hash script tagged with a specified stake opcode, and in
the case it is, extract and return the hash. The second new function,
named extractStakeScriptHash, is similar except it detect a stake-tagged
pay-to-script-hash script tagged with a specified stake opcode.
Finally, the third function is named isStakeSubmissionScript and is
defined in terms of the former two functions.
The extract function approach was chosen because it is common for
callers to want to only extract relevant details from a script if the
script is of the specific type. Extracting those details requires
performing the exact same checks to ensure the script is of the correct
type, so it is more efficient to combine the two into one and define the
type determination in terms of the result so long as the extraction does
not require allocations.
The following is a before and after comparison of analyzing a large
script:
benchmark old ns/op new ns/op delta
---------------------------------------------------------------------
BenchmarkIsStakeSubmissionScript 140308 4.20 -100.00%
benchmark old allocs new allocs delta
---------------------------------------------------------------------
BenchmarkIsStakeSubmissionScript 1 0 -100.00%
benchmark old bytes new bytes delta
---------------------------------------------------------------------
BenchmarkIsStakeSubmissionScript 466944 0 -100.00%
This continues the process of converting the typeOfScript function to
use a combination of raw script analysis and the new tokenizer instead
of the far less efficient parsed opcodes.
In particular, it converts the detection of nulldata scripts to use both raw
script analysis and the new tokenizer.
The following is a before and after comparison of analyzing a large
script:
benchmark old ns/op new ns/op delta
--------------------------------------------------------------
BenchmarkIsNullDataScript 120800 3.81 -100.00%
benchmark old allocs new allocs delta
--------------------------------------------------------------
BenchmarkIsNullDataScript 1 0 -100.00%
benchmark old bytes new bytes delta
--------------------------------------------------------------
BenchmarkIsNullDataScript 466944 0 -100.00%
This continues the process of converting the typeOfScript function to
use a combination of raw script analysis and the new tokenizer instead
of the far less efficient parsed opcodes.
In particular, it converts the detection of pay-to-alt-pubkey-hash
scripts to use raw script analysis.
In order to accomplish this, it introduces two new functions. The first
one is named extractPubKeyHashAltDetails and works with the raw script
bytes to simultaneously determine if the script is a
pay-to-alt-pubkey-hash script, and in the case it is, extract and return
the hash and signature type. The second new function is named
isPubKeyHashAltScript and is defined in terms of the former.
The extract function approach was chosen because it is common for
callers to want to only extract relevant details from a script if the
script is of the specific type. Extracting those details requires
performing the exact same checks to ensure the script is of the correct
type, so it is more efficient to combine the two into one and define the
type determination in terms of the result so long as the extraction does
not require allocations.
It is important to note that this new implementation intentionally has a
semantic difference from the existing implementation in that it will now
only pass when one of two signature types currently supported by
consensus are specified whereas previously it would allow any single
byte data push.
The following is a before and after comparison of analyzing a large
script:
benchmark old ns/op new ns/op delta
-------------------------------------------------------------------
BenchmarkIsAltPubKeyHashScript 107100 2.63 -100.00%
benchmark old allocs new allocs delta
-------------------------------------------------------------------
BenchmarkIsAltPubKeyHashScript 1 0 -100.00%
benchmark old bytes new bytes delta
-------------------------------------------------------------------
BenchmarkIsAltPubKeyHashScript 466944 0 -100.00%
This continues the process of converting the typeOfScript function to
use a combination of raw script analysis and the new tokenizer instead
of the far less efficient parsed opcodes.
In particular, it converts the detection of pay-to-pubkey-hash scripts
to use raw script analysis.
In order to accomplish this, it introduces two new functions. The first
one is named extractPubKeyHash and works with the raw script bytes
to simultaneously determine if the script is a pay-to-pubkey-hash script,
and in the case it is, extract and return the hash. The second new
function is named isPubKeyHashScript and is defined in terms of the
former.
The extract function approach was chosen because it is common for
callers to want to only extract relevant details from a script if the
script is of the specific type. Extracting those details requires
performing the exact same checks to ensure the script is of the correct
type, so it is more efficient to combine the two into one and define the
type determination in terms of the result so long as the extraction does
not require allocations.
The following is a before and after comparison of analyzing a large
script:
benchmark old ns/op new ns/op delta
----------------------------------------------------------------
BenchmarkIsPubKeyHashScript 165903 0.64 -100.00%
benchmark old allocs new allocs delta
----------------------------------------------------------------
BenchmarkIsPubKeyHashScript 1 0 -100.00%
benchmark old bytes new bytes delta
----------------------------------------------------------------
BenchmarkIsPubKeyHashScript 466945 0 -100.00%
This continues the process of converting the typeOfScript function to
use a combination of raw script analysis and the new tokenizer instead
of the far less efficient parsed opcodes.
In particular, it converts the detection of pay-to-alt-pubkey scripts to
use raw script analysis.
In order to accomplish this, it introduces two new functions. The first
one is named extractPubKeyAltDetails and works with the raw script bytes
to simultaneously determine if the script is a pay-to-alt-pubkey script,
and in the case it is, extract and return the relevant details. The
second new function is named isPubKeyAltScript and is defined in terms
of the former.
The extract function approach was chosen because it is common for
callers to want to only extract relevant details from a script if the
script is of the specific type. Extracting those details requires
performing the exact same checks to ensure the script is of the correct
type, so it is more efficient to combine the two into one and define the
type determination in terms of the result so long as the extraction does
not require allocations.
It is important to note that this new implementation intentionally
tightens the following semantics as compared to the existing
implementation:
- The signature type must now be one of the two supported types versus
allowing any single byte data push
- The public key must now be of the correct length for the given
signature type versus allowing any size up to 512 bytes
- The public key for schnorr secp256k1 pubkeys must now be a compressed
public key and adhere to the strict encoding requirements for them
The following is a before and after comparison of analyzing a large
script:
benchmark old ns/op new ns/op delta
---------------------------------------------------------------
BenchmarkIsAltPubKeyScript 143449 2.99 -100.00%
benchmark old allocs new allocs delta
---------------------------------------------------------------
BenchmarkIsAltPubKeyScript 1 0 -100.00%
benchmark old bytes new bytes delta
---------------------------------------------------------------
BenchmarkIsAltPubKeyScript 466944 0 -100.00%
This continues the process of converting the typeOfScript function to
use a combination of raw script analysis and the new tokenizer instead
of the far less efficient parsed opcodes.
In particular, it converts the detection of pay-to-pubkey scripts to use
raw script analysis.
In order to accomplish this, it introduces four new functions:
extractCompressedPubKey, extractUncompressedPubKey, extractPubKey, and
isPubKeyScript. The extractPubKey function makes use of
extractCompressedPubKey and extractUncompressedPubKey to combine their
functionality as a convenience and isPubKeyScript is defined in terms of
extractPubKey.
The extractCompressedPubKey works with the raw script bytes to
simultaneously determine if the script is a pay-to-compressed-pubkey
script, and in the case it is, extract and return the raw compressed
pubkey bytes.
Similarly, the extractUncompressedPubKey works in the same way except it
determines if the script is a pay-to-uncompressed-pubkey script and
returns the raw uncompressed pubkey bytes in the case it is.
The extract function approach was chosen because it is common for
callers to want to only extract relevant details from a script if the
script is of the specific type. Extracting those details requires
performing the exact same checks to ensure the script is of the correct
type, so it is more efficient to combine the two into one and define the
type determination in terms of the result so long as the extraction does
not require allocations.
The following is a before and after comparison of analyzing a large
script:
benchmark old ns/op new ns/op delta
------------------------------------------------------------
BenchmarkIsPubKeyScript 124749 4.01 -100.00%
benchmark old allocs new allocs delta
------------------------------------------------------------
BenchmarkIsPubKeyScript 1 0 -100.00%
benchmark old bytes new bytes delta
------------------------------------------------------------
BenchmarkIsPubKeyScript 466944 0 -100.00%
This converts the isAnyKindOfScriptHash function to analyze the raw
script instead of requiring far less efficient parsed opcodes thereby
significantly optimizing the function.
Since the function relies on isStakeScriptHash to identify a stake
tagged pay-to-script-hash, and is the only consumer of it, this also
converts that function to analyze the raw script and renames it to
isStakeScriptHashScript for more consistent naming.
Finally, the tests are updated accordingly.
The following is a before and after comparison of analyzing a large
script:
benchmark old ns/op new ns/op delta
-------------------------------------------------------------------
BenchmarkIsAnyKindOfScriptHash 101249 3.83 -100.00%
benchmark old allocs new allocs delta
-------------------------------------------------------------------
BenchmarkIsAnyKindOfScriptHash 1 0 -100.00%
benchmark old bytes new bytes delta
-------------------------------------------------------------------
BenchmarkIsAnyKindOfScriptHash 466944 0 -100.00%
This implements an efficient and zero-allocation script tokenizer that
is exported to both provide a new capability to tokenize scripts to
external consumers of the API as well as to serve as a base for
refactoring the existing highly inefficient internal code.
It is important to note that this tokenizer is intended to be used in
consensus critical code in the future, so it must exactly follow the
existing semantics.
The current script parsing mechanism used throughout the txscript module
is to fully tokenize the scripts into an array of internal parsed
opcodes which are then examined and passed around in order to implement
virtually everything related to scripts.
While that approach does simplify the analysis of certain scripts and
thus provide some nice properties in that regard, it is both extremely
inefficient in many cases, and makes it impossible for external
consumers of the API to implement any form of custom script analysis
without manually implementing a bunch of error prone tokenizing code or,
alternatively, the script engine exposing internal structures.
For example, as shown by profiling the total memory allocations of an
initial sync, the existing script parsing code allocates a total of
around 295.12GB, which equates to around 50% of all allocations
performed. The zero-alloc tokenizer this introduces will allow that to
be reduced to virtually zero.
The following is a before and after comparison of tokenizing a large
script with a high opcode count using the existing code versus the
tokenizer this introduces for both speed and memory allocations:
benchmark old ns/op new ns/op delta
------------------------------------------------------------
BenchmarkScriptParsing 153099 961 -99.37%
benchmark old allocs new allocs delta
------------------------------------------------------------
BenchmarkScriptParsing 1 0 -100.00%
benchmark old bytes new bytes delta
------------------------------------------------------------
BenchmarkScriptParsing 466945 0 -100.00%
The following is an overview of the changes:
- Introduce new error code ErrUnsupportedScriptVersion
- Implement zero-allocation script tokenizer
- Add a full suite of tests to ensure the tokenizer works as intended
and follows the required consensus semantics
- Add an example of using the new tokenizer to count the number of
opcodes in a script
- Update README.md to include the new example
- Update script parsing benchmark to use the new tokenizer
