This modifies the core logic for decompressing a y coordinate from an x
coordinate, which is primarily used when parsing public keys, but can
also be used when recovering public keys from compact signatures in the
future, to make use of field vals and their recently-added optimized
modular square root calculation capability and also adds comprehensive
tests to ensure the decompression works properly independent of public
key parsing.
Since the public key type is still defined in terms of big ints, the
parsing function still relies on them and so the existing
decompressPoint function was updated to make use of the new function and
perform the necessary conversion to and from big ints. Ultimately, the
goal is make that unnecessary.
This is work towards eventually using the new more efficient mod n
scalar throughout.
The following benchmark shows a before and after comparison of
decompressing a public key:
benchmark old ns/op new ns/op delta
-------------------------------------------------------------
BenchmarkPubKeyDecompress 43574 10961 -74.85%
benchmark old allocs new allocs delta
-------------------------------------------------------------
BenchmarkPubKeyDecompress 28 0 -100.00%
benchmark old bytes new bytes delta
-------------------------------------------------------------
BenchmarkPubKeyDecompress 2586 0 -100.00%
This introduces the ability to determine if a specialized field value is
greater than or equal to the field prime minus the group order in
constant time. It is also significantly faster than performing the
calculation via big integers since it would require a conversion from
the specialized type in practice.
Comprehensive tests with 100% coverage and benchmarks are included.
For now, this merely introduces the method without modifying any of the
code to make use of it, but it will be useful when working with
signatures and public key recovery in the future.
The following benchmark shows a comparison between determining the
condition via the new specialized method and via generic big ints:
benchmark old ns/op new ns/op delta
--------------------------------------------------------------------
BenchmarkIsGtOrEqPrimeMinusOrder 123 10.6 -91.38%
benchmark old allocs new allocs delta
--------------------------------------------------------------------
BenchmarkIsGtOrEqPrimeMinusOrder 2 0 -100.00%
benchmark old bytes new bytes delta
--------------------------------------------------------------------
BenchmarkIsGtOrEqPrimeMinusOrder 96 0 -100.00%
This introduces the ability to calculate the square root modulo the
field characteristic, when it exists, via the specialized field value
type. It is significantly faster than performing the calculation via
big integers and it is also constant time.
The motivation for adding this is that it is needed for point
decompression which currently makes use of big ints.
Comprehensive tests with 100% coverage and a full suite of benchmarks
are included.
Like other code in this area, the code relies on techniques that are not
the easiest to review for correctness, so significant effort has been
put into attempting to clearly explain the details via comments to help
with review.
For now, this merely introduces the method without modifying any of the
code to make use of it. The intention is to do so in future commits.
The following benchmark shows a comparison between calculating the
square root via the new specialized method and via generic big ints.
As can be seen, the new method is roughly twice as fast and reduces
allocations by 100%.
benchmark old ns/op new ns/op delta
--------------------------------------------------
BenchmarkSqrt 24605 11087 -54.94%
benchmark old allocs new allocs delta
--------------------------------------------------
BenchmarkSqrt 91 0 -100.00%
benchmark old bytes new bytes delta
--------------------------------------------------
BenchmarkSqrt 1016 0 -100.00%
This modifies the function to serialize signatures to use the new
ModNScalar type instead of big ints and takes the opportunity to clean
up the relevant code so that it is more readable and avoid unnecessary
extra allocations.
It also improves the comments to reference the DER specification and
include the specific details of the format inline.
This is work towards eventually using the new more efficient mod n
scalar throughout.
While the primary focus of this change is not optimization since
signature serialization happens infrequently enough that it is not in a
performance hot path, as can be seen from the benchmark below, it does
have the effect of making the serialization faster and significantly
reducing the number of allocations.
The following benchmark shows a before and after comparison of
serializing a typical signature:
benchmark old ns/op new ns/op delta
---------------------------------------------------------
BenchmarkSigSerialize 373 316 -15.28%
benchmark old allocs new allocs delta
---------------------------------------------------------
BenchmarkSigSerialize 5 3 -40.00%
benchmark old bytes new bytes delta
---------------------------------------------------------
BenchmarkSigSerialize 256 144 -43.75%
This modifies the function that produces signatures to use the new
ModNScalar type instead of big ints and also fleshes out the comments to
include a reference to original signing algorithm in GECC, a paraphrased
version of it for those without access to the book, and the
modifications made so that it is deterministic per RFC6979 and conforms
to BIP0062.
This is work towards eventually using the new more efficient mod n
scalar throughout.
While the primary focus of this change is not optimization since signing
happens infrequently enough that it is not in a performance hot path, as
can be seen from the benchmark below, it does have the effect of making
signing a bit faster.
The following benchmark shows a before and after comparison of producing
a typical signature:
benchmark old ns/op new ns/op delta
-------------------------------------------------
BenchmarkSign 59788 54905 -8.17%
benchmark old allocs new allocs delta
-------------------------------------------------
BenchmarkSign 43 36 -16.28%
benchmark old bytes new bytes delta
-------------------------------------------------
BenchmarkSign 2338 1776 -24.04%
This modifies the Signature type to be opaque in order to facilitate
eventually using the new more efficient mod n scalar. For now, it makes
no other changes beyond making the type opaque.
This modifies the signature verification function use the new ModNScalar
type instead of big ints.
This is work towards eventually using the new more efficient mod n
scalar throughout.
The following benchmark shows a before and after comparison of typical
signature verification:
benchmark old ns/op new ns/op delta
------------------------------------------------------
BenchmarkSigVerify 219346 180815 -17.57%
benchmark old allocs new allocs delta
------------------------------------------------------
BenchmarkSigVerify 57 36 -36.84%
benchmark old bytes new bytes delta
------------------------------------------------------
BenchmarkSigVerify 2818 1697 -39.78%
This adds a function to find the modular multiplicative inverse in
non-constant time by making use big ints along with the associated tests
and benchmarks.
Ideally this will be replaced with an implementation that does not rely
on big integers, as well as ultimately have a constant-time
implementation. However, this version is being introduced to avoid
blocking the ongoing work towards using the new more efficient mod n
scalar throughout.
This modifies the internal scalarMultJacobian and scalarBaseMultJacobian
functions to accept the new ModNScalar type instead of a raw byte slices
and updates the code accordingly.
It also moves the moduloReduce function from curve.go to
ellipticadaptor.go since it is only required in the adaptor code now.
This is work towards eventually using the new more efficient mod n
scalar throughout.
This modifies the NonceRFC6979 function to return the new ModNScalar
type instead of a big.Int to move closer towards big integer removal in
the primary code paths.
While here, this also adds a few more explicit zeros of memory when
signing which is good practice when dealing with key material.
This modifies the NonceRFC6979 function to follow the RFC for data that
is larger than the expected size. In particular, section 2.3.2
specifically says that the sequence is first _truncated_ or expanded to
the expected length. The exact wording is:
if qlen < blen, then the qlen leftmost bits are kept, and subsequent
bits are discarded
The current code is dropping the leftmost bits (most significant in big
endian) instead of the rightmost.
While here, this also switches to a byte slice for the key param instead
of a big integer to avoid requiring big integers which will be going
away in the future.
This also adds a bunch of additional tests to ensure the expected
behavior in all scenarios with malformed data as well as including tests
for the extra data and version portion which were previously lacking.
In practice, nothing in the code base is calling the function with too
many bytes, so this has no realistic effect on anything, but the
function should actually follow the spec.
This modifies the PrivateKey type to be opaque and updates the internal
implementation of it to use the new ModNScalar type instead of a
big.Int.
It also modifies NewPrivateKey to accept the new ModNScalar type instead
of a big.Int and improves the PrivKeyFromBytes comment to explicitly
call out the semantics that realistically already existed when using the
value in calculations.
It is also worth noting that there are still a couple of internal
conversions to big.Int when dealing with signatures that are required
since that code still relies on them for now. The goal is to eventually
update them to use the more efficient specialized types and this change
help facilitate that process.
This introduces a specialized type for handling arithmetic modulo the
group order which is significantly faster than using big integers from
the standard library
The implementation is also constant time, whereas big integers from the
standard library are not, which helps protect against side channel
attacks in certain scenarios.
Comprehensive tests with 100% coverage and a full suite of benchmarks
are included.
Due to the fact the code relies on a lot of low-level bit manipulations
to achieve the speed and constant time characteristics, it is not the
easiest code to review for correctness, so significant effort has been
put into attempting to clearly explain the details via comments to help
with review.
For now, this merely introduces the type without modifying any of the
code to make use of it, and it also does not yet include the ability to
perform inversions which will be required in order to switch over to it.
That capability will be added in a future commit.
The following benchmarks show a comparison between the new specialized
type and generic big ints for various operations:
benchmark old ns/op new ns/op delta
------------------------------------------------------------
BenchmarkModN 267 18.9 -92.92%
BenchmarkZero 7.28 0.64 -91.25%
BenchmarkIsZero 0.32 0.32 +0.00%
BenchmarkEquals 9.14 5.58 -38.95%
BenchmarkAddModN 305 23.8 -92.20%
BenchmarkMulModN 457 237 -48.14%
BenchmarkSquareModN 459 238 -48.15%
BenchmarkNegateModN 295 9.82 -96.67%
BenchmarkIsOverHalfOrder 9.25 8.55 -7.57%
benchmark old allocs new allocs delta
------------------------------------------------------------
BenchmarkModN 2 0 -100.00%
BenchmarkZero 0 0 +0.00%
BenchmarkIsZero 0 0 +0.00%
BenchmarkEquals 0 0 +0.00%
BenchmarkAddModN 2 0 -100.00%
BenchmarkMulModN 2 0 -100.00%
BenchmarkSquareModN 2 0 -100.00%
BenchmarkNegateModN 2 0 -100.00%
BenchmarkIsOverHalfOrder 0 0 +0.00%
The comment says this is only exported for test code, but the test
code is able to call the unexported function, and is not used outside
of the package.
stake/v3 is not released yet so a breaking change is permitted here to
remove this API.
This change replaces usage of chainhash (which is intended solely for
transaction and blocks) and dcrutil.Hash160 (used in scripts for
hashing redeem scripts and pubkeys) with direct calls to each hashing
function from dcrd's crypto subpackages. This improves clarity and
shortens the transitive package imports. Notably, dcrutil and
everything it depends on is no longer transitively imported when
importing the hdkeychain package.
This modifies all prelease modules to use the latest commit to work out
more transitive dependency issues. Several of these are needed due to
the dependency chain.
The updated direct dependencies are as follows:
- github.com/decred/dcrd/blockchain/stake/v3@v3.0.0-20200215031403-6b2ce76f0986
- github.com/decred/dcrd/blockchain/v3@v3.0.0-20200215031403-6b2ce76f0986
- github.com/decred/dcrd/chaincfg/v3@v3.0.0-20200215031403-6b2ce76f0986
- github.com/decred/dcrd/connmgr/v3@v3.0.0-20200215031403-6b2ce76f0986
- github.com/decred/dcrd/dcrec/secp256k1/v3@v3.0.0-20200215031403-6b2ce76f0986
- github.com/decred/dcrd/dcrutil/v3@v3.0.0-20200215031403-6b2ce76f0986
- github.com/decred/dcrd/hdkeychain/v3@v3.0.0-20200215031403-6b2ce76f0986
- github.com/decred/dcrd/mempool/v4@v4.0.0-20200215031403-6b2ce76f0986
- github.com/decred/dcrd/mining/v3@v3.0.0-20200215031403-6b2ce76f0986
- github.com/decred/dcrd/txscript/v3@v3.0.0-20200215031403-6b2ce76f0986
This modifies all prelease modules to use the latest commit to work out
more transitive dependency issues. Several of these are needed due to
the dependency chain.
The updated direct dependencies are as follows:
- github.com/decred/dcrd/blockchain/stake/v3@v3.0.0-20200215023918-6247af01d5e3
- github.com/decred/dcrd/blockchain/v3@v3.0.0-20200215023918-6247af01d5e3
- github.com/decred/dcrd/chaincfg/v3@v3.0.0-20200215023918-6247af01d5e3
- github.com/decred/dcrd/connmgr/v3@v3.0.0-20200215023918-6247af01d5e3
- github.com/decred/dcrd/dcrec/secp256k1/v3@v3.0.0-20200215023918-6247af01d5e3
- github.com/decred/dcrd/dcrutil/v3@v3.0.0-20200215023918-6247af01d5e3
- github.com/decred/dcrd/hdkeychain/v3@v3.0.0-20200215023918-6247af01d5e3
- github.com/decred/dcrd/mempool/v4@v4.0.0-20200215023918-6247af01d5e3
- github.com/decred/dcrd/mining/v3@v3.0.0-20200215023918-6247af01d5e3
- github.com/decred/dcrd/txscript/v3@v3.0.0-20200215023918-6247af01d5e3
This modifies all prelease modules to use the latest version to work out
more transitive dependency issues.
The updated direct dependencies are as follows:
- github.com/decred/dcrd/blockchain/stake/v3@v3.0.0-20200215015031-3283587e6add
- github.com/decred/dcrd/blockchain/v3@v3.0.0-20200215015031-3283587e6add
- github.com/decred/dcrd/chaincfg/v3@v3.0.0-20200215015031-3283587e6add
- github.com/decred/dcrd/connmgr/v3@v3.0.0-20200215015031-3283587e6add
- github.com/decred/dcrd/dcrec/secp256k1/v3@v3.0.0-20200215015031-3283587e6add
- github.com/decred/dcrd/dcrutil/v3@v3.0.0-20200215015031-3283587e6add
- github.com/decred/dcrd/hdkeychain/v3@v3.0.0-20200215015031-3283587e6add
- github.com/decred/dcrd/mempool/v4@v4.0.0-20200215015031-3283587e6add
- github.com/decred/dcrd/mining/v3@v3.0.0-20200215015031-3283587e6add
- github.com/decred/dcrd/txscript/v3@v3.0.0-20200215015031-3283587e6add
This modifies all prelease modules to use the latest version so they can
be used in require statements in consumer code that is also under
development without running into transitive dependency issues.
The updated direct dependencies are as follows:
- github.com/decred/dcrd/blockchain/stake/v3@v3.0.0-20200214194519-928737b3e580
- github.com/decred/dcrd/blockchain/v3@v3.0.0-20200214194519-928737b3e580
- github.com/decred/dcrd/connmgr/v3@v3.0.0-20200214194519-928737b3e580
- github.com/decred/dcrd/dcrutil/v3@v3.0.0-20200214194519-928737b3e580
- github.com/decred/dcrd/mempool/v4@v4.0.0-20200214194519-928737b3e580
- github.com/decred/dcrd/mining/v3@v3.0.0-20200214194519-928737b3e580
- github.com/decred/dcrd/txscript/v3@v3.0.0-20200214194519-928737b3e580
This replace the ECPrivKey method with a SerializedPrivKey method to be
consistent with the newly added SerializePubKey method and updates the
tests accordingly.
The primary reason for this change is that it removes the secp256k1 type
from the public API and is more efficient in practice because the
serialized bytes are already available internally and callers are
exceedingly likely to just serialize the returned privkey again, so the
current approach almost always resulted in an additional needless round
trip through parsing the private key.
This removes the ECPubKey method in favor of the newly added
SerializedPubKey method, updates the tests and example to use the new
method instead, and finally updates the README accordingly as well.
The primary reason for this change is that it removes the secp256k1 type
from the public API and is more efficient in practice because the
serialized bytes are already available internally and callers are
exceedingly likely to just serialize the returned pubkey again, so the
current approach almost always resulted in an additional needless round
trip through parsing the public key, which is rather expensive.
This adds a PrivKey method to dcrutil.WIF which returns the serialized
private key.
While here and before a stable release of dcrutil/v3 has been tagged,
the method SerializePubKey is renamed to simply PubKey.
SerializePubKey was not an accurate method name, as it described an
action performed rather than the data returned, and this
implementation was returning already-serialized bytes.
A mention in a doc comment to the unexported field ecType (which is
probably a find & replace artifact) is removed. Because the WIF might
be for a key that is not ECDSA (e.g. ed25519 or schnorr keys), the
ecType field is renamed to scheme.
hdkeychain has received breaking changes since the last release (break
occurred in 85f0c09df2), and further releases must use a bumped major
version. The standard process for introducing major API breaks has
been followed:
- Bump the major version in the go.mod of the affected module if not
already done since the last release tag
- Add a replacement to the go.mod in the main module if not already done
since the last release tag
- Update all imports in the repo to use the new major version as
necessary
- Make necessary modifications to allow all other modules to use the new
version in the same commit
- Repeat the process for any other modules the require a new major as a
result of consuming the new major(s)
This modifies some recently-updated modules to use a valid prerelease
version so they can be used in require statements in consumer code that
is also under development.
The updated direct dependencies are as follows:
- github.com/decred/dcrd/chaincfg/v3@v3.0.0-20200214194519-928737b3e580
- github.com/decred/dcrd/dcrec/secp256k1/v3@v3.0.0-20200214194519-928737b3e580
The SerializedPubKey method returns the internal and memoized bytes
representing the compressed serialization of the HD key's secp256k1
public key. The motivation for providing this method is to avoid an
extra serialization and deserialization through *secp256k1.PublicKey
when creating the pubkey hash. Wallet profiling revealed that roughly
30% of CPU time deriving child addresses from account branch extended
keys was a result of the extra forced serialization steps.
This adds --peeridletimeout config option and updates the peer
connection to enforce idle timeouts using the connection's read
deadline. Associated testsand documentation have also been updated.
This change moves tickets that spend from transactions in
the mempool into a separate data structure, so they are not
considered for block template inclusion or fee estimation.
This modifies the consensus critical function which determines if the
redeem script of a p2sh or stake-tagged p2sh contains stake opcodes to
avoid calling the GetScriptClass function which is only intended
explicitly for working with standard script forms which only apply in
the context of the more restrictive standardness policy rules.
It also renames the function to better indicate its purpose is to
specifically check the redeem script as opposed to the entire signature
script.
This updates the allowed votes range for mainnet, simnet
and regnet to coinbase maturity from tip. Testnet has been
exempted due to longer reorgs that have already happened
in testnet's history.
Add a new file and a test for the rpcserver funcion
handleCreateRawTransaction.
Change createrawsstx documentation to use address instead of desaddr.
Add that map to the help message.
Change createrawssrtx documentation to show that it accepts only one
input in documentation and part of the help message.
For createrawsstx explicitly state that all values are in atoms in help
messages and the documentation.
For createrawssrtx explicitly state that all values are in coins in help
messages and the documentation. Remove range loop for only one input.
For createrawtransaction explicitly state that all values are in coins in
help messages and the documentation. Move conversion to atoms above the
check for max amount and check after converting from coins.
Remove bad white spaces from JSON api documentation.
This adds a specialized function for testing if a field value is one
that is constant time and faster than an equality check along with
comprehensive tests for correct functionality. As the benchmarks below
show, it results in a reduction of roughly 2.5% for addition which
translates to about a 1% reduction in typical signature verification
time.
benchmark old ns/op new ns/op delta
--------------------------------------------------------------------
BenchmarkAddJacobian 580 564 -2.76%
BenchmarkScalarBaseMultJacobian 34629 33885 -2.15%
BenchmarkScalarBaseMultLarge 47468 47021 -0.94%
BenchmarkScalarMult 157832 156503 -0.84%
BenchmarkSigVerify 216043 213725 -1.07%
BenchmarkNonceRFC6979 6447 6370 -1.19%
This implements signature verification directly instead of calling the
version in the standard library in order to ultimately be able to break
the reliance on big ints it requires and thus allow for optimization
specifically for the sec256k1 curve.
It currently is only marginally faster since it still relies on big
ints. However, there is already a 10% reduction in allocations which
reduces pressure on the GC during IBD when there are a ton of signature
verifications happening.
benchmark old ns/op new ns/op delta
-----------------------------------------------------
BenchmarkSigVerify 224238 219603 -2.07%
benchmark old allocs new allocs delta
-----------------------------------------------------
BenchmarkSigVerify 64 57 -10.94%
benchmark old bytes new bytes delta
-----------------------------------------------------
BenchmarkSigVerify 3138 2818 -10.20%
This introduces a new structure for passing around jacobian coordinates
and updates the code to make use of it.
This increases the readability of the code by reducing a lot of extra
parameter clutter and makes it more obvious what type of coordinates the
functions expect (Jacobian vs affine) as well as which parameters are
for passing the results back out. It also makes the internal code less
cumbersome to work with.
The only reason it was originally split into coords was due to early
versions of the Go compiler having issues with escape analysis leading
to slower performance in performance-critical code. However, that is no
longer the case as can be seen by the benchmarks below which show
everything within the margin of error in terms of performance. A few
reductions in allocations were achieved in the scalar multiplication
path, but the focus of these changes are not optimization.
benchmark old ns/op new ns/op delta
------------------------------------------------------------------
BenchmarkAddJacobian 582 577 -0.86%
BenchmarkAddJacobianNotZOne 1091 1093 +0.18%
BenchmarkScalarBaseMult 46389 46681 +0.63%
BenchmarkScalarBaseMultJacobian 34437 34386 -0.15%
BenchmarkScalarBaseMultLarge 47012 47216 +0.43%
BenchmarkScalarMult 155915 156262 +0.22%
BenchmarkNAF 1447 1372 -5.18%
BenchmarkSigVerify 215279 217324 +0.95%
BenchmarkFieldNormalize 21.3 21.5 +0.94%
BenchmarkNonceRFC6979 6402 6393 -0.14%
benchmark old allocs new allocs delta
------------------------------------------------------------------
BenchmarkAddJacobian 0 0 +0.00%
BenchmarkAddJacobianNotZOne 0 0 +0.00%
BenchmarkScalarBaseMult 5 5 +0.00%
BenchmarkScalarBaseMultJacobian 1 1 +0.00%
BenchmarkScalarBaseMultLarge 8 8 +0.00%
BenchmarkScalarMult 24 22 -8.33%
BenchmarkNAF 3 3 +0.00%
BenchmarkSigVerify 64 58 -9.38%
BenchmarkFieldNormalize 0 0 +0.00%
BenchmarkNonceRFC6979 17 17 +0.00%
benchmark old bytes new bytes delta
------------------------------------------------------------------
BenchmarkAddJacobian 0 0 +0.00%
BenchmarkAddJacobianNotZOne 0 0 +0.00%
BenchmarkScalarBaseMult 224 224 +0.00%
BenchmarkScalarBaseMultJacobian 32 32 +0.00%
BenchmarkScalarBaseMultLarge 400 400 +0.00%
BenchmarkScalarMult 1233 1137 -7.79%
BenchmarkNAF 128 128 +0.00%
BenchmarkSigVerify 3138 2850 -9.18%
BenchmarkFieldNormalize 0 0 +0.00%
BenchmarkNonceRFC6979 976 976 +0.00%
This modifies the PublicKey type to be defined as its own struct as
opposed of in terms of an ecdsa.PublicKey and updates the code
accordingly.
For now, it essentially keeps the same format with the exception of
embedding the elliptic.Curve, but being its own type will allow for it
to eventually be made opaque and thus provide greater optimizations.
This is continued work towards making the internals of the package
entirely independent of the crypto/elliptic and crypto/ecdsa stdlib
interfaces.
