This modifies signature verification to perform the final equality check
in Jacobian projective space to avoid an extra expensive field inversion
by taking advantage of the fact the R component of the signature can be
converted to Jacobian projective space and compared directly so long as
the two possible cases that arise from R being mod N and the original x
coordinate used to create it being mod P.
The following is a before and after comparison of verification of a
typical signature:
benchmark old ns/op new ns/op delta
-----------------------------------------------------
BenchmarkSigVerify 189505 177147 -6.52%
benchmark old allocs new allocs delta
-----------------------------------------------------
BenchmarkSigVerify 34 34 +0.00%
benchmark old bytes new bytes delta
-----------------------------------------------------
BenchmarkSigVerify 1633 1633 +0.00%
The MaybeAcceptDependents function was added as a part of the logic to
track tickets with non-approved inputs separately in a staging pool.
Given it modifies the contents of the mempool, it needs to hold a write
mutex as opposed to a read mutex as otherwise it can modify the mempool
out from under readers.
While here, comment the required locking semantics on the other
functions that were added as a part of the staging logic as well.
This removes the GenerateKey function from the schnorr package since
nothing uses it and it is expected that the caller will use
secp256k1.GeneartePrivateKey instead since the package works with that
type as private keys.
This reduces the number of internal copies of private keys that are made
when serializing them and parsing them from bytes in order to help
reduce the attack surface.
This removes the hash func callback parameter for hashing the commitment
data from the internal sign and verify functions in favor of just
calling the real hash function directly.
This is being done for several reasons:
- It is not safe to use just any hash function, rather the hash function
must have certain properties, so it's safer to make it explicit
- There is only a single hash function, blake256, being used and thus
having an extra unneeded parameter is not desirable
- Any modifications to the commitment data invalidate existing
signatures, so the internals of the function have to realistically be
updated when making any changes anyway
This modifies the internal schnorrVerify function to accept the
signature as a type instead of its raw serialized form. This will
ultimately facilitate making it easier to switch over to more efficient
types for the r and s components.
The existing signing and verification tests have several issues:
- They are using data that is not easily independently verifiable by
other implementations
- The known good test data is using a hash function that is, per the
comments, "horribly broken" instead of using the real hash function
- They are not actually testing anything for the real hash function
since they are doing the equivalent of sign(x) == sign(x)
- The randomness aspects are not actually all that random since they
just involve shuffling the order of the test data instead of actually
producing random data each run
- They aren't consistent with the way tests are written in the rest of
the code
Consequently, this reworks the signing and signature verification tests
to resolve all of those issues. The following is a high-level overview
of the changes:
- Create new known good test data that is easily reproducible and
has been independently verified with the Sage computer algebra system
- The test data includes the original messages so the resulting hashes
are independently verifiable and have a known source
- The test data includes variations of signing the same data with
different keys and nonces, both deterministically generated via
RFC6979 and random, and signing different data with the same keys
- Add random tests that generate random keys and messages with each run
from a new random seed and log that seed in the event of failure
- Ensure mutating a random bit in each good signature results in that
mutated signature failing to verify the original message
- Ensure mutating a random bit in each message hash that was
originally signed results in the original good signature failing to
verify the new mutated message
This removes the pubNonceX and Y parameters from the internal signing
function since they're always called with nil and if the tweaking
functionality is desired in the future, it should take a point to
operate on instead of individual big ints.
This moves the code related to benchmarking signing and signature
verification to a separate file and modifies them to be more consistent
with the code in the secp256k1 package.
This moves the signature code from ecdsa.go and ecdsa_test.go to
signature.go and signature_test.go where it more properly belongs since
not only is it dealing with signatures, ECDSA is another algorithm
altogether and thus it doesn't make sense to name the files that.
This modifies the Signature type to be opaque in order to facilitate
eventually using the new more efficient mod n scalar from secp256k1.
For now, it makes no other changes beyond making the type opaque.
This removes the unnecessary generalized Verify function in favor of the
Verify method on the signature to be more consistent with the secp256k1
package.
This removes the code related to public key recovery from a schnorr
signature since it is not used, there are not currently any plans to use
it, and the existing code would require some effort to get it production
ready.
This code will still be in the repository history if it is ever needed
at some point in the future.
This adds a method to the PrivateKey type to clear the memory associated
with it so that consumers can explicitly clear key material for enhanced
security against memory scraping. It also adds tests to ensure it works
as expected.
This modifies the PublicKey type to be opaque in order to facilitate
eventually using the new more efficient mod n scalar. For now, it makes
no other changes beyond making the type opaque.
This updates the internal implementation of the Signature type to use
the new ModNScalar type instead of big.Ints and modifies NewSignature to
accept the new ModNScalar type accordingly.
Signature parsing and signing operations are now completely free of
big.Ints. Verification still relies on them due to their use in public
keys.
This allows the stringer for the DynamicBanScore type to be used without
deadlocking.
This is a port of the same fix in btcd along with improved test
coverage. Credits to Jin <lochjin@gmail.com> for the fix.
This reworks the DER signature parsing tests to ensure 100% test
coverage of all branches in the parsing code and also test for the
explicit reason for a failure to ensure that each test is actually
testing the intended condition.
Compare this to the previous more generic validity testing which only
tested that a signature failed to parse as opposed to _why_ it failed to
parse. The result was that some of the tests were not actually testing
what they claimed.
This reworks the DER signature parsing code to use the majority of the
implementation code that exists in the consensus scripting engine,
updates it to make use of the new specialized mod N scalar, and
introduces a new error type and associated error code for signature
errors which can be used with the errors.Is and errors.As interfaces to
programmatically detect the failure reason.
It includes full tests for the new signature error infrastructure.
The primary motivation behind using the implementation code from the
consensus script engine is that it is already extremely well tested, any
signatures that don't conform to it would be rejected by consensus
anyway, and the code is easier to follow due to the use of constants and
better documentation.
This removes the ability to parse signatures encoded with the more lax
Basic Encoding Rules (BER) since they have never been valid on the
Decred blockchain and the existing parsing code for them really didn't
follow the spec anyway given it enforced additional length restrictions
that the BER spec technically allows.
This modifies the instances in the edwards package that zero out big
ints to use a function to clear the internal memory used by the big int
since merely setting it to 0 does not do that.
This modifies the instances in the schnorr package that zero out big
ints that use a function to clear the internal memory used by the big
int since merely setting it to 0 does not do that.
This overhauls compact signatures to significantly optimize them and to
use the recently-added mod n scalar type along with field vals. It also
significantly improves the comments and readability while overhauling
the related code.
The existing approach to create a compact signature was extremely
inefficient since it involved attempting to recover all of the possible
public key candidates from the signature and comparing them against the
real public key in order to determine which candidate was the correct
one.
This takes a much more efficient approach that consists of modifying the
signing code to produce and return a "pubkey recovery code" that
uniquely identifies which of the possible candidates is correct since
the signing code already has the necessary information. It adds
virtually zero overhead to normal signatures, but has a significant
impact on compact signatures.
It also gains speed advantages by making use of several of the
recently-added optimized field val methods such as y coordinate
decompression and determining if a field val is >= the field prime minus
the group order.
The following benchmarks show a before and after comparison of producing
regular signatures, producing compact signatures, and recovering the
public key from the compact signatures:
benchmark old ns/op new ns/op delta
-----------------------------------------------------------
BenchmarkSign 58268 58478 +0.36%
BenchmarkSignCompact 551174 58171 -89.45%
BenchmarkRecoverCompact 212780 206042 -3.17%
benchmark old allocs new allocs delta
-----------------------------------------------------------
BenchmarkSign 36 36 +0.00%
BenchmarkSignCompact 166 39 -76.51%
BenchmarkRecoverCompact 51 36 -29.41%
benchmark old bytes new bytes delta
-----------------------------------------------------------
BenchmarkSign 1776 1776 +0.00%
BenchmarkSignCompact 7815 1920 -75.43%
BenchmarkRecoverCompact 2610 1745 -33.14%
This modifies the SetBytes method on the ModNScalar type to return the
overflow condition as a uint32 instead of a bool in order to better
facilite constant time code in upcoming changes and updates the callers
as needed.
It also updates the internal fieldToModNScalar convenience function to
return the flag as well.
This modifies the core logic for decompressing a y coordinate from an x
coordinate, which is primarily used when parsing public keys, but can
also be used when recovering public keys from compact signatures in the
future, to make use of field vals and their recently-added optimized
modular square root calculation capability and also adds comprehensive
tests to ensure the decompression works properly independent of public
key parsing.
Since the public key type is still defined in terms of big ints, the
parsing function still relies on them and so the existing
decompressPoint function was updated to make use of the new function and
perform the necessary conversion to and from big ints. Ultimately, the
goal is make that unnecessary.
This is work towards eventually using the new more efficient mod n
scalar throughout.
The following benchmark shows a before and after comparison of
decompressing a public key:
benchmark old ns/op new ns/op delta
-------------------------------------------------------------
BenchmarkPubKeyDecompress 43574 10961 -74.85%
benchmark old allocs new allocs delta
-------------------------------------------------------------
BenchmarkPubKeyDecompress 28 0 -100.00%
benchmark old bytes new bytes delta
-------------------------------------------------------------
BenchmarkPubKeyDecompress 2586 0 -100.00%
This introduces the ability to determine if a specialized field value is
greater than or equal to the field prime minus the group order in
constant time. It is also significantly faster than performing the
calculation via big integers since it would require a conversion from
the specialized type in practice.
Comprehensive tests with 100% coverage and benchmarks are included.
For now, this merely introduces the method without modifying any of the
code to make use of it, but it will be useful when working with
signatures and public key recovery in the future.
The following benchmark shows a comparison between determining the
condition via the new specialized method and via generic big ints:
benchmark old ns/op new ns/op delta
--------------------------------------------------------------------
BenchmarkIsGtOrEqPrimeMinusOrder 123 10.6 -91.38%
benchmark old allocs new allocs delta
--------------------------------------------------------------------
BenchmarkIsGtOrEqPrimeMinusOrder 2 0 -100.00%
benchmark old bytes new bytes delta
--------------------------------------------------------------------
BenchmarkIsGtOrEqPrimeMinusOrder 96 0 -100.00%
This introduces the ability to calculate the square root modulo the
field characteristic, when it exists, via the specialized field value
type. It is significantly faster than performing the calculation via
big integers and it is also constant time.
The motivation for adding this is that it is needed for point
decompression which currently makes use of big ints.
Comprehensive tests with 100% coverage and a full suite of benchmarks
are included.
Like other code in this area, the code relies on techniques that are not
the easiest to review for correctness, so significant effort has been
put into attempting to clearly explain the details via comments to help
with review.
For now, this merely introduces the method without modifying any of the
code to make use of it. The intention is to do so in future commits.
The following benchmark shows a comparison between calculating the
square root via the new specialized method and via generic big ints.
As can be seen, the new method is roughly twice as fast and reduces
allocations by 100%.
benchmark old ns/op new ns/op delta
--------------------------------------------------
BenchmarkSqrt 24605 11087 -54.94%
benchmark old allocs new allocs delta
--------------------------------------------------
BenchmarkSqrt 91 0 -100.00%
benchmark old bytes new bytes delta
--------------------------------------------------
BenchmarkSqrt 1016 0 -100.00%
This modifies the function to serialize signatures to use the new
ModNScalar type instead of big ints and takes the opportunity to clean
up the relevant code so that it is more readable and avoid unnecessary
extra allocations.
It also improves the comments to reference the DER specification and
include the specific details of the format inline.
This is work towards eventually using the new more efficient mod n
scalar throughout.
While the primary focus of this change is not optimization since
signature serialization happens infrequently enough that it is not in a
performance hot path, as can be seen from the benchmark below, it does
have the effect of making the serialization faster and significantly
reducing the number of allocations.
The following benchmark shows a before and after comparison of
serializing a typical signature:
benchmark old ns/op new ns/op delta
---------------------------------------------------------
BenchmarkSigSerialize 373 316 -15.28%
benchmark old allocs new allocs delta
---------------------------------------------------------
BenchmarkSigSerialize 5 3 -40.00%
benchmark old bytes new bytes delta
---------------------------------------------------------
BenchmarkSigSerialize 256 144 -43.75%
This modifies the function that produces signatures to use the new
ModNScalar type instead of big ints and also fleshes out the comments to
include a reference to original signing algorithm in GECC, a paraphrased
version of it for those without access to the book, and the
modifications made so that it is deterministic per RFC6979 and conforms
to BIP0062.
This is work towards eventually using the new more efficient mod n
scalar throughout.
While the primary focus of this change is not optimization since signing
happens infrequently enough that it is not in a performance hot path, as
can be seen from the benchmark below, it does have the effect of making
signing a bit faster.
The following benchmark shows a before and after comparison of producing
a typical signature:
benchmark old ns/op new ns/op delta
-------------------------------------------------
BenchmarkSign 59788 54905 -8.17%
benchmark old allocs new allocs delta
-------------------------------------------------
BenchmarkSign 43 36 -16.28%
benchmark old bytes new bytes delta
-------------------------------------------------
BenchmarkSign 2338 1776 -24.04%
This modifies the Signature type to be opaque in order to facilitate
eventually using the new more efficient mod n scalar. For now, it makes
no other changes beyond making the type opaque.
This modifies the signature verification function use the new ModNScalar
type instead of big ints.
This is work towards eventually using the new more efficient mod n
scalar throughout.
The following benchmark shows a before and after comparison of typical
signature verification:
benchmark old ns/op new ns/op delta
------------------------------------------------------
BenchmarkSigVerify 219346 180815 -17.57%
benchmark old allocs new allocs delta
------------------------------------------------------
BenchmarkSigVerify 57 36 -36.84%
benchmark old bytes new bytes delta
------------------------------------------------------
BenchmarkSigVerify 2818 1697 -39.78%
This adds a function to find the modular multiplicative inverse in
non-constant time by making use big ints along with the associated tests
and benchmarks.
Ideally this will be replaced with an implementation that does not rely
on big integers, as well as ultimately have a constant-time
implementation. However, this version is being introduced to avoid
blocking the ongoing work towards using the new more efficient mod n
scalar throughout.
This modifies the internal scalarMultJacobian and scalarBaseMultJacobian
functions to accept the new ModNScalar type instead of a raw byte slices
and updates the code accordingly.
It also moves the moduloReduce function from curve.go to
ellipticadaptor.go since it is only required in the adaptor code now.
This is work towards eventually using the new more efficient mod n
scalar throughout.
This modifies the NonceRFC6979 function to return the new ModNScalar
type instead of a big.Int to move closer towards big integer removal in
the primary code paths.
While here, this also adds a few more explicit zeros of memory when
signing which is good practice when dealing with key material.
This modifies the NonceRFC6979 function to follow the RFC for data that
is larger than the expected size. In particular, section 2.3.2
specifically says that the sequence is first _truncated_ or expanded to
the expected length. The exact wording is:
if qlen < blen, then the qlen leftmost bits are kept, and subsequent
bits are discarded
The current code is dropping the leftmost bits (most significant in big
endian) instead of the rightmost.
While here, this also switches to a byte slice for the key param instead
of a big integer to avoid requiring big integers which will be going
away in the future.
This also adds a bunch of additional tests to ensure the expected
behavior in all scenarios with malformed data as well as including tests
for the extra data and version portion which were previously lacking.
In practice, nothing in the code base is calling the function with too
many bytes, so this has no realistic effect on anything, but the
function should actually follow the spec.
This modifies the PrivateKey type to be opaque and updates the internal
implementation of it to use the new ModNScalar type instead of a
big.Int.
It also modifies NewPrivateKey to accept the new ModNScalar type instead
of a big.Int and improves the PrivKeyFromBytes comment to explicitly
call out the semantics that realistically already existed when using the
value in calculations.
It is also worth noting that there are still a couple of internal
conversions to big.Int when dealing with signatures that are required
since that code still relies on them for now. The goal is to eventually
update them to use the more efficient specialized types and this change
help facilitate that process.
This introduces a specialized type for handling arithmetic modulo the
group order which is significantly faster than using big integers from
the standard library
The implementation is also constant time, whereas big integers from the
standard library are not, which helps protect against side channel
attacks in certain scenarios.
Comprehensive tests with 100% coverage and a full suite of benchmarks
are included.
Due to the fact the code relies on a lot of low-level bit manipulations
to achieve the speed and constant time characteristics, it is not the
easiest code to review for correctness, so significant effort has been
put into attempting to clearly explain the details via comments to help
with review.
For now, this merely introduces the type without modifying any of the
code to make use of it, and it also does not yet include the ability to
perform inversions which will be required in order to switch over to it.
That capability will be added in a future commit.
The following benchmarks show a comparison between the new specialized
type and generic big ints for various operations:
benchmark old ns/op new ns/op delta
------------------------------------------------------------
BenchmarkModN 267 18.9 -92.92%
BenchmarkZero 7.28 0.64 -91.25%
BenchmarkIsZero 0.32 0.32 +0.00%
BenchmarkEquals 9.14 5.58 -38.95%
BenchmarkAddModN 305 23.8 -92.20%
BenchmarkMulModN 457 237 -48.14%
BenchmarkSquareModN 459 238 -48.15%
BenchmarkNegateModN 295 9.82 -96.67%
BenchmarkIsOverHalfOrder 9.25 8.55 -7.57%
benchmark old allocs new allocs delta
------------------------------------------------------------
BenchmarkModN 2 0 -100.00%
BenchmarkZero 0 0 +0.00%
BenchmarkIsZero 0 0 +0.00%
BenchmarkEquals 0 0 +0.00%
BenchmarkAddModN 2 0 -100.00%
BenchmarkMulModN 2 0 -100.00%
BenchmarkSquareModN 2 0 -100.00%
BenchmarkNegateModN 2 0 -100.00%
BenchmarkIsOverHalfOrder 0 0 +0.00%
