This implements a subsidy cache with funcs to calculate subsidies for
proof-of-work, votes, and the treasury and includes a full suite of
tests.
It consists of ports of the related code in blockchain/standalone with
the following modifications:
- Removes the subsidy split parameters from SubsidyParams since the
chaincfg implementations are deprecated
- Uses hard-coded constants for the PoW, PoS, and treasury splits both
prior to and after the activation of DCP0010
- Removes the version 1 methods and renames the version 2 variants to
take their place:
- CalcWorkSubsidyV2 -> CalcWorkSubsidy
- CalcStakeVoteSubsidyV2 -> CalcStakeVoteSubsidy
- Consolidates the tests specific to the decentralized treasury agenda
into their counterparts
- Includes additional maintainer notes with the expected way to update
the code without needing a major module bump should it ever be
necessary
This deprecates the per-network subsidy split parameters for the
following reasons.
- They are the same on all networks so there really is no good reason to
parameterize them as such
- The subsidy calculation code can be simplified and made slightly more
efficient without the unnecessary parameterization
This reworks the tests that deal with signing random messages with
random private keys and recovering the associated public key to make
them more consistent with modern practices in the code as well as to
expand the testing methodology to include additional assurances.
Specifically, the relevant test now:
- Signs random representative hashes instead of a fixed non-hashed message
- Signs, recovers, and verifies both the compressed and uncompressed
public key variants for each iteration
- Ensures mutating a random bit in the good signature results in failure
to recover the original public key
- Ensures mutating a random bit in the message hash that was originally
signed results in failure to recover the original public key from the
good signature
This adds tests for compact signature generation and associated public
key recovery for both the positive and negative paths.
The following is a high-level overview of the changes:
- Add tests to ensure test data that was independently verified with the
Sage computer algebra system produces the expected results
- Add tests which are specifically designed to exercise all of the
recovery failure paths
This converts the error handling in compact signature recovery to be
consistent with the rest of the code base such that it returns concrete
errors that support errors.Is/As and thus are programmatically
detectable by callers.
Currently the code that rejects old block versions once the majority of
the network has upgraded is performed using a loop that iterates
backwards from latest enforced block version while checking for a
majority upgrade at each version. This is inefficient for blocks early
in the chain since it means multiple versions need to be checked when it
isn't really necessary.
This optimizes the relevant code by taking into account that the
relevant enforcement semantics reduce to the following:
- Block versions greater than or equal to the latest enforced block
version can never be rejected for being old
- Block versions must be rejected when the majority of the network has
upgraded to ANY version greater than that version
The following timing information obtained from profiling shows the
overall reduction achieved with the optimized code for mainnet:
existing: Total time to check old versions for 645603 headers: 10.61118s
optimized: Total time to check old versions for 645603 headers: 3.52555s
This adds a test to sign and verify random data in addition to the
existing tests the deal with known good data and edge conditions.
Specifically, the test:
- Generates random keys and messages with each run from a new random
seed and log that seed in the event of failure
- Signs the rand message with the random key and ensures the produced
signature verifies correctly
- Ensures mutating a random bit in each good signature results in that
mutated signature failing to verify the original message
- Ensures mutating a random bit in each message hash that was originally
signed results in the original good signature failing to verify the
new mutated message
Although the ECDSA code has been extensively tested through usage and
external fuzz testing, there are not currently any unit tests for the
non-compact signing and verification path.
This resolves that by adding signing and verification tests for both the
positive and negative paths. The following is a high-level overview of
the changes:
- Create new known good test data that is easily reproducible and has
been independently verified with the Sage computer algebra system
- The test data includes the original messages so the resulting hashes
are independently verifiable and have a known source
- The test data includes variations of signing the same data with
different keys and nonces, both deterministically generated via
RFC6979 and random, and signing different data with the same keys
- Add tests to ensure the aforementioned test data that was
independently verified produces the expected results
- Add tests which are specifically designed to exercise conditions that
lead to invalid signatures
- Add tests to help ensure the signature verification code fails to
verify edge conditions as expected by crafting signatures that are
specifically designed to hit them
This refactors the primary logic for the regular ECDSA signing code to a
separate function that accepts an arbitrary nonce. The primary
motivation is to facilitate upcoming unit tests which aim to fully test
both the positive and negative paths, however, it is also beneficial in
that it provides more flexibility for potentially using other nonce
generation mechanisms in the future should it be necessary.
Upcoming changes constitute breaking public API changes to the
rpcclient module, therefore, this follows the process for introducing
major API breaks which consists of:
- Bump the major version in the go.mod of the affected module if not
already done since the last release tag
- Add a replacement to the go.mod in the main module if not already done
since the last release tag
- Update all imports in the repo to use the new major version as
necessary
- Make necessary modifications to allow all other modules to use the new
version in the same commit
- Repeat the process for any other modules the require a new major as a
result of consuming the new major(s)
Upcoming changes constitute breaking public API changes to the
rpc/jsonrpc/types module, therefore, this follows the process for
introducing major API breaks which consists of:
- Bump the major version in the go.mod of the affected module if not
already done since the last release tag
- Add a replacement to the go.mod in the main module if not already done
since the last release tag
- Update all imports in the repo to use the new major version as
necessary
- Make necessary modifications to allow all other modules to use the new
version in the same commit
- Repeat the process for any other modules the require a new major as a
result of consuming the new major(s)
This adds a script to the contrib directory that initializes a Go
multi-module workspace and adds all of the modules provided by the dcrd
repository to it on an as needed basis. Note that workspaces require Go
1.18+.
This is useful when developing across multiple modules in the repository
and allows development environments that make use of the Go language
server (aka gopls), such as VSCode, to provide full support without also
needing to temporarily create replacements in the various go.mod files
or individually add every module.
Do note, however, that workspaces are local, so final submissions to the
repository will still require the appropriate changes to the relevant
go.mod files to ensure resolution outside of the workspace.
Finally, it also updates the README.md in the contrib directory
accordingly.
This fixes an issue with indexers wait for sync where the indexer gets
synced before a sync subcsriber is created. The sync subscriber is left
idling, waitingfor the next update which blocks the caller. The index
subscriber has been updated to periodically update sync subcribers
of all subscribed indexers to it as a result.
This bumps the go directive for all of the modules provided by the
repository to 1.17 which will allow the new module graph pruning and
lazy loading capabilities introduced in Go 1.17 to be used once the
updated modules are released.
This means that, as described by the documentation, the go.mod files for
each module now include a separate require block that includes all of
the indirect dependencies
* Updated the minimum recommended Disc Space from 12 GB to 16 GB,
* Added an estimate of ~2 GB/yr
* Followed the SI style guide for units: A space is always used to separate the unit from
the number.
With the release of Go 1.18, the supported versions listed in the
README and used in the go.yml workflow should be 1.17 and 1.18.
Also update the golangci-lint to v1.45.2
Upcoming changes constitute breaking public API changes to the
blockchain module, therefore, this follows the process for introducing
major API breaks which consists of:
- Bump the major version in the go.mod of the affected module if not
already done since the last release tag
- Add a replacement to the go.mod in the main module if not already done
since the last release tag
- Update all imports in the repo to use the new major version as
necessary
- Make necessary modifications to allow all other modules to use the new
version in the same commit
- Repeat the process for any other modules the require a new major as a
result of consuming the new major(s)
The directory created by `T.TempDir` is automatically removed when the
test and all its subtests complete.
This commit also refactor several cleanup logic by using `t.Cleanup`
instead of `defer`.
Reference: https://pkg.go.dev/testing#T.TempDir
Reference: https://pkg.go.dev/testing#T.Cleanup
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
Profiling shows that around 7.5% of the time in scalar base
multiplication is attributed to duffcopy. Upon further examination,
this is the result of a combination of the range statement making copies
of the bytes and the need to construct a Jacobian point from the
individual field values stored in the in-memory byte points table.
This optimizes the function to avoid that as follows:
- Perform the conversion to Jacobian once when the affine byte table is
decompressed from the stored values
- Make use of those Jacobian points directly
- Use an indexed for loop instead of a range over the bytes
- Perform the calculation using the result variable directly instead of
via a local variable that is copied to the result
The following benchmark results show the speedup is in line with the
expected gains per the profiling results:
name old time/op new time/op delta
------------------------------------------------------------------------------
ScalarBaseMultNonConst 24.1µs ±22% 22.5µs ± 2% -6.97% (p=0.000 n=98+96)
Profiling allocations during an initial chain sync shows that a large
portion of the allocations due to the CHECKMULTISIG opcode are the
result of generating the list of public keys and signatures.
This eliminates those allocations as follows:
- Use slices into fixed-size stack-based backing arrays for the typical
number of public keys and signatures
- Make the slice of signature info contiguous instead of pointers which
not only eliminates the associated allocations, but also provides
better cache locality
The CI pipeline was previously downloading golangci-lint using the tool
goreleaser/godownloader which is now deprecated. This PR changes the GitHub
actions config file to use the latest recommended installation method.
This updates the main module to use the latest base58 and
gorilla/websocket module versions.
- github.com/decred/base58@v1.0.4
- github.com/gorilla/websocket@v1.5.0
This optimizes the scalar decomposition code by rewriting it to make use
of the highly-efficient zero-allocation ModNScalar type along with
math/bits (which provides hardware acceleration on most platforms)
instead of big.Ints.
The net effect is that the decomposition is significantly faster,
allocation free, and constant time.
It also adds a bunch of detailed comments to better describe the
endomorphism and how it is used in scalar multiplication in addition to
fully describing the scalar decomposition process and derivation.
Finally, it adds logic to derive and print the new constants the
reworked code makes of to the precomputation code that runs with 'go
generate'.
The following is a high level overview of the changes:
- Rewrites splitK to use the ModNScalar type instead of big.Ints:
- Allocation free
- Constant time
- Provides hardware acceleration on most platforms
- Includes detailed comments describing the scalar decomposition
process and derivation
- Updates endomorphism parameter constant definitions to be ModNScalars
instead of big.Int
- Moves the convenience func hexToModNScalar to the main package
- Adds new endoZ1 and endoZ2 constants for the rounded multiplication
used by the new decomposition code
- Adds logic to derive and print the new endomorphism constants with
'go generate'
- Updates the scalar multiplication to account for the new semantics
- Adds detailed comments to scalar multiplication
- Tightens negation magnitudes in addZ1EqualsZ2 and remove no longer
needed normalization
- Ensures the calculation when recovering compact signatures uses
normalized points
- Updates associated tests
- Updates associated benchmark
- Removes splitKModN test helper since conversion is no longer needed
The following benchmarks show a before and after comparison of scalar
decomposition as well as how it that translates to scalar multiplication
and signature verification:
name old time/op new time/op delta
---------------------------------------------------------------------
SplitK 1.61µs ±32% 0.89µs ± 2% -44.69% (p=0.000 n=50+47)
ScalarMult 125µs ± 1% 115µs ± 1% -7.82% (p=0.000 n=43+46)
SigVerify 161µs ±25% 160µs ±19% -0.53% (p=0.001 n=50+50)
name old allocs/op new allocs/op delta
-----------------------------------------------------------------------
SplitK 10.0 ± 0% 0.0 -100.00% (p=0.000 n=50+50)
ScalarMult 11.0 ± 0% 0.0 -100.00% (p=0.000 n=50+50)
SigVerify 28.0 ± 0% 16.0 ± 0% -42.86% (p=0.000 n=50+50)
While it only saves about 1 µs per signature verification in the
benchmarking scenario, the primary win is the reduction in the number of
allocations per signature verification which has a much more significant
impact when verifying large numbers of signatures back to back, such as
when processing new blocks, and especially during the initial chain sync
process.
This modifies the code related to precomputing the linearly independent
vectors used by the endomorphism to compute all of the related params,
including lambda and beta, as well as the alternative params. It also
ensures the derived values satisfy the required equations.
This helps makes it clear how they are derived as opposed to being
magical constants and also makes it easier to include new derived values
which are planned for an upcoming optimization.
This reworks the k (scalar) splitting tests to make them more consistent
with modern practices in the code as well as to make the testing
methodology less reliant on the specific implementation details and to
test edge cases.
It also updates the random k splitting tests to ensure new random values
are tested each run as opposed to the existing tests which always test
the same values since they use the same random seed.
Specifically, the following properties are asserted:
- The decomposed scalars must result in the original scalar via the
equation k = k1 + k2*lambda (mod n)
- Both decomposed scalars must have small magnitude as determined by
having a maximum bit length of one more than half of the bit size of
the underlying field
Currently, when the server is started with a chain that is already
synced, a new block template will not be generated until either a new
template is forced to be generated via the regentemplate RPC or a new
block is connected.
In practice, the most observable consequence is that getwork won't
return results right away when restarting the server when the chain is
already synced until either of the aforementioned cases happened which
makes it appear as if it is hung.
This resolves that by modifying the logic of the background block
template generator to wait for the initial chain sync to complete prior
to attempting to generate the first template unless unsynced mining is
allowed.
This adds entries to ignore the new local workspace related files
being introduced with Go1.18 and also adds some entries for testing,
profiling, and benchmarking artifacts and removes a binary that no
longer exists while here.
This updates various modules to require go1.13 since they make use of
shifts on integers (technically signed shifts although they are only
ever used with positive values) and the upcoming version of go1.18 along
with the latest version of gopls complains about not requiring go1.13.
This reworks the tests that deal with scalar point multiplication with
points projected into Jacobian coordinates for randomly-generated
scalars and points to make them more consistent with modern practices in
the code as well as to expand the testing methodology to include
additional assurances.
It also updates the tests to ensure new random values are tested each
run as opposed to the existing tests which always test the same values
since they use the same random seed.
Specifically:
- The points are no longer converted to affine with each iteration which
ensures z values other than 1 are tested
- Each iteration now ensures calculating the negative version of the
point and adding it results in the point at infinity
- The check to ensure the same final point was calculated is now done
outside of the loop to speed up the test
This adds a new test helper for generating random mod n scalars without
also generating a companion big integer and updates the tests to make
use of it. It also marks the combined version as a test helper while
here.
The new helper will be useful for future tests.
This cleans up the tests for performing Jacobian point doubling to make
them more consistent with modern practices in the code.
The following is a high level overview of the changes:
- Adds names to each individual test for easier identification
- Makes the individual test definitions consistent with the rest of the
code
This cleans up the tests for performing Jacobian addition to make them
more consistent with modern practices in the code.
The following is a high level overview of the changes:
- Adds names to each individual test for easier identification
- Makes the individual test definitions consistent with the rest of the
code
