This modifies the dcr_tmux_simnet_setup.sh script to update the mining
and transfer addresses to account for the change to dcrwallet master
which always upgrades the cointype on simnet.
This modifies the dcr_tmux_simnet_setup.sh script to use a variable to
consolidate wallet creation answers instead of a file so there are no
artifacts left.
This modifies the dcr_tmux_simnet_setup.sh script to set the root of all
of the nodes to the environment variable DCR_SIMNET_ROOT when it is set
and fallback to the existing path when it is not.
It also updates contrib/README.md and docs/simnet_environment.mediawiki
to document the new capability.
The prompts when creating the wallet were changed when the wallet
passphrase was part of the application configuration, which caused the
script to hang. This fixes the hang by replying to the additional
prompt, and updates the script to pipe input to dcrwallet's stdin to
reply to the prompts, rather than using tmux to send keys.
In order to allow tests to use mocked mempool transaction data, replace
the RPC server's TxMemPool with an interface that satisfies what is
needed for the server.
This categorizes the various configuration options in the code to
improve readability.
It also updates the doc.go CLI options reference to reflect reality.
This adds the code to migrate the database to a new v3 block index.
It involves removing the ticket hashes associated with vote info and
revocations that are no longer needed. The process can be interrupted
at any point and future invocations will resume from the point it was
interrupted.
Originally, some information that is related to vote ticket hashes and
revocations was added to the block index because it was needed prior to
the full node index being available and also because there was no
dynamic loading of the relevant stake node information in various
circumstances which meant the information had to be available on the
nodes at all times.
However, since that time, things have changed quite a bit such that the
full block index is now available in memory, the stake nodes and related
information are now efficiently loaded on demand, and stake node pruning
has been optimized accordingly.
As a result, it no longer makes sense to store all of this information.
Profiling shows this reduces the total inuse memory by about 19.5%.
Although permissioned, the RPC server will allow for checking any number
of values for its various exists commands. It would be reasonable to
limit unreasonable requests that could exhaust dcrd's available memory
and cause node failure. Changes will come with a major RPC version bump.
This removes the ExistsAddrIndex method from the SyncManager interface
and replaces it with a new interface ExistsAddresser that supplies the
two methods used by the rpcserver, ExistsAddress and ExistsAddresses.
This adds documentation for the following ws notifications:
- notifywinningtickets
- notifyspentandmissedtickets
- notifynewtickets
- notifystakedifficulty
This adds a script for simplifying simnet usage by creating a fresh
version of a preconfigured environment each time it is started along
with documentation that provides information regarding simnet, how to
make use of the script and environment, including some basic tasks and
common issues, as well as some information about manual configuration.
This script is something that has evolved over the years from one I
originally wrote and was later modified by Matheus Degiovani to make use
of tmux. The version provided here also contains some new features and
convenience scripts that were written specifically to help newer users
and simplify common tasks such as performing a chain reorganization and
shutting down the environment.
Some of the documentation is based on a gist I created a long time ago,
which was then converted to the content currently on the documentation
site, but this is a much more polished version that includes the extra
information about the script and associated environment it provides and
is better served alongside the script in the repository.
This adds Filterer and FiltererV2 interfaces to rpcserver, which
decouples rpcserver from the implementation to retrieve committed
filters and allows mocked versions to be provided for testing.
This modifies the dial timeout test to make use of a channel for the
cancellation signal synchronization instead of a sleep to prevent false
positive test failures in slower environments such as CI servers.
Part of adding nodes to the block index involves updating the current
chain tip in order to be able to efficiently keep track of all known
chain tips.
Since there can be multiple tips at a given height, the current code
uses a slice for each height to track them. However, this is wasteful
since it is extremely rare for there to be more than one chain tip at
the same height.
It is particularly notable at startup since the block index has to load
up hundreds of thousands of nodes.
Given that, this modifies the chain tip tracking logic to avoid creating
a slice unless it is actually needed by making use of an entry with a
static field for the normal case and a dynamic slice for the case when
more than 1 tip for the height actually exists.
It also keeps a running sum of the total number of known chips for use
as a hint to further reduce allocations when serving up related data and
updates the chain tip tests to ensure 100% branch coverage is maintained.
The end result is that this reduces the total number of allocs at
startup by around 1%, which results in faster startup, and also reduces
the overall memory usage by a few MB for nodes that have a long running
history of side chains.
Similar to the recent changes made to txscript logging, this updates the
peer code to take advantage of the ability to determine the logging
level when constructing expensive log strings in order to avoid the
allocations caused by closures and removes the no longer necessary code
related to creating them.
After recent optimizations, the current next biggest offender of more
allocations than would be expected revealed by profiling is due to the
trace logging closures for the scripting engine execution.
Once upon a time, there was no way to check the current logging level
with the logging infrastructure at the time and thus a logging closure
was used to defer the fairly expensive construction of the trace logging
information until it was actually invoked (meaning tracing is enabled).
However, those closures come at the cost of allocations, and since
script execution is something that happens non-stop during normal
operation, those allocations really add up, as the profiling shows.
As some point, the logging infrastructure was changed out, and it is now
possible to determine the logging level, so this updates the code to
take advantage of that and avoid the closures while still only
performing the fairly expensive construction of trace logging
information when tracing is enabled. In other words, with this change
there is zero cost (other than the conditional check, of course) when
tracing is not enabled.
Finally, this also removes the no longer necessary code related to
creating the logging closures.
Profiling shows that approximately 0.67% of the total allocations are
coming from DropN even though it should realistically be zero.
The cause is that DropN returns an error if it is called with 0 and the
code that drops the alternate data stack in between scripts calls it
with the depth of the alternate data stack, which is almost always zero.
The call has no observable effect on the result of the script execution
in that case since the error is intentionally ignored, there is nothing
to be dropped, and is therefore correct semantically, however, it is
wasteful to generate and throw away the error.
There really is no good reason for the internal stack implementation to
return an error from DropN when called with 0 since it equivalent to a
NOP.
I have done a careful analysis of all call sites to ensure the change
will break any observable semantics.
Thus, this modifies DropN to remove the unnecessary error condition and
updates the associated internal stack implementation test accordingly.
This removes the internal blockchain/dbnamespace package in favor of
defining the bucket names and keys locally in the file that works with
them in the blockchain package.
An internal package for this gives the false impression that there is
some expectation that other packages in the module would be digging
around in the internals of what is intended to be an implementation
detail of the package.
It would also significantly complicates the ability to change the
database implementation.
The IDs are assigned in increasing order, so this has the net effect
of peer results returned in the order they were connected. It also
keeps the order the same across multiple calls.
This matches the behavior of the current pull request implementing
getpeerinfo in dcrwallet.
This modifies the signature cache Exists method to make use of the new
efficient equality testing methods provided by the secp256k1 types
versus indirectly checking their equality via their serializations.
The following benchmark shows a before and after comparison of a typical
signature cache exists check:
name old time/op new time/op delta
-------------------------------------------------------------------------
SigCacheExists 400ns ± 0% 62ns ± 0% -84.54% (p=0.000 n=5+5)
name old alloc/op new alloc/op delta
--------------------------------------------------------------------------
SigCacheExists 256B ± 0% 0B -100.00% (p=0.008 n=5+5)
name old allocs/op new allocs/op delta
--------------------------------------------------------------------------
SigCacheExists 4.00 ± 0% 0.00 -100.00% (p=0.008 n=5+5)
This modifies the websocket notification manager to make use of the
newer pattern that synchronizes the Run context with an internal quit
channel to ensure that any exported functions that communicate with the
goroutines can't possibly hang during shutdown.
This was previously done by storing the context in the struct (which is
generally discouraged) and protecting it by a mutex which means every
single notification had to take the mutex.
As can be seen by the diff of this commit, this approach is not only
more performant, it's also less code.
Profiling revealed that the number one cause of allocations in
blockchain is currently due to stake node pruning, which definitely
should not be the case.
Upon closer examination, the culprit is that every single time the stake
nodes are pruned the entire ancestry of nodes prior to the prune height,
including those that have already been pruned, are appended to the list
of nodes to prune. In practice, this amounts to hundreds of thousands
of nodes and thus the slice is resized multiple times (likely around
18 times per run given the current runtime semantics).
This remedies that by ensuring that only the nodes that actually need to
be pruned are added to the relevant slice and also preallocating space
per a hint that will handle the number of nodes expected to be pruned
with a 99% confidence level to further reduce allocations when more than
a single node needs to be pruned.
It also renames the slice to reflect what it actually does.
This modifies the pruning logic to set the pruning interval to the
target block time for the given chain parameters instead of hard coding
a constant with the mainnet value.
This modifies the main startup logic to make use of the fact that the
server now has its lifecycle tied to the same context that is cancelled
when a shutdown signal is received.
The end result is that the separate goroutine it was running in before
and associated synchronization logic is no longer necessary.
