This replaces the ErrDoubleSpend and ErrMissingTx error codes with a
single error code named ErrMissingTxOut and updates the relevant errors
and expected test results accordingly.
Once upon a time, the code relied on a transaction index, so it was able
to definitively differentiate between a transaction output that
legitimately did not exist and one that had already been spent.
However, since the code now uses a pruned utxoset, it is no longer
possible to reliably differentiate since once all outputs of a
transaction are spent, it is removed from the utxoset completely.
Consequently, a missing transaction could be either because the
transaction never existed or because it is fully spent.
Also, while here, consistently use the LookupEntry function on the
UtxoView instead of directly accessing the internal map as intended.
This renames the mempool.Config.RelayNonStd option to AcceptNonStd which
more accurately describes its behavior since the mempool was refactored
into a separate package.
The reasoning for this change is that the mempool is not responsible for
relaying transactions (nor should it be). Its job is to maintain a pool
of unmined transactions that are validated according to consensus and
policy configuration options which are then used to provide a source of
transactions that need to be mined.
Instead, it is the server that is responsible for relaying transactions.
While it is true that the current server code currently only relays txns
that were accepted to the mempool, this does not necessarily have to
be the case. It would be entirely possible (and perhaps even a good
idea as something do in the future), to separate the relay policy from
the mempool acceptance policy (and thus indirectly the mining policy).
This modifies the SSGenBlockVotedOn function to directly copy the hash
the vote commits to into a statically sized array versus calling the
hash function and thus can remove the potential runtime error.
This is preferred because the code will fail to compile if the size
of chainhash.HashSize is every changed which is desirable because it
would mean the assumptions the code makes would be invalidated.
It also updates all callers in the repository accordingly.
This merge commit adds the following code from the
github.com/decred/dcrutil package into a new
github.com/decred/dcrd/dcrutil package:
* Address handling
* Amount type
* AppDataDir func
* bitflags functions
* Block wrapper type
* Hash160 func
* Tx wrapper type
* WIF type
as well as all tests for this code.
The old github.com/decred/dcrutil/hdkeychain package has also been
merged and moved to github.com/decred/dcrd/dcrutil/hdkeychain.
dcrd packages have been updated to use the new packages and the dep
files have been updated for this change.
This updates the default policy for standard transactions to enforce the
new CSV opcode. In other words, with this change, the new opcode will
be enforced when considering candidate transactions for acceptance to
the mempool, relaying, and inclusion into block templates.
This adds enforcement of the newly introduced relative time locks via
transaction sequence numbers when considering candidate transactions for
acceptance to the mempool, relaying, and inclusion into block templates.
It also raises the maximum standard transaction version to 2
accordingly. This is acceptable because it is a soft forking change and
the aforementioned areas only enforce policy.
The following is an overview of the changes:
- Modify mempool to consider version 2 transactions standard
- Introduce a new callback to the mempool config for obtaining the
sequence lock for block after the current best chain tip so it can
remain decoupled from the chain
- Update mempool to enforce relative locks on all candidate transactions
- Update the mock chain in the mempool test harness accordingly
This reverts the changes related to the CheckSequenceVerify opcode that
were merged from upstream since additional changes are needed and it's
much cleaner to implement all of code related to the sequence locks in
the same PR which will be referenced by the DCP as opposed to being
split up in multiple.
This modifies the mempool transaction finality checks to use the past
median time instead of the adjusted network time. This ensures the
standardness rules match the upcoming consensus rules which reject
transactions that are not after the median time of the last several
blocks. Without this change, it is possible for transactions which can
never make it into a block to be admitted to the mempool thereby wasting
memory.
The following is an overview of the changes:
- Introduce a new callback to the mempool config named PastMedianTime
for obtaining the median time for the current best chain so it can
remain decoupled from the chain
- Update the tx finality standardness check to use the callback
- Remove the now unused TimeSource mempool config parameter
- Update the mempool test harness and mock chain to use a mock median time
- Update server to provide a closure for the mempool config that uses
the cached median time for the current best chain to negate any
additional performance impact the new calculations would otherwise
cause from recalculating the median time for every new candidate tx
validation
This modifies the way standard verification flags are handled so that it
is possible to selectively enable them based on the result of agenda
votes.
First, it moves the StandardVerifyFlags constant from the txscript
package to the mempool/policy code and rename it to
BaseStandardVerifyFlags. As the TODO in the comment of the moved
constant indicated, these flags are policy related and thus really
belong in policy. Ideally there would be a completely separate policy
package, but since the policy code currently lives in mempool/policy.go,
the constant has been moved there.
Next, it introduces a new function named standardScriptVerifyFlags,
which accepts the chain as an argument and, for now, just returns the
BaseStandardVerifyFlags along with a nil error. This will allow
additional flags to be selectively enabled depending on the result of an
agenda vote.
Finally, it updates the mempool policy struct to require a closure for
obtaining the flags so it can remain decoupled from the chain which in
turn allows easier and more robust unit testing of mempool functionality
since it allows a mocks to be used.
The old code ran dust checks on regular, vote, and revocation
transactions, while ignoring any dust checking on ticket purchases.
This change also removes the dust checking for votes and revocations
as these are required for the network to run and should not be
rejected just to minimize dust. The output amounts for these votes
and revocations are determined by consensus rules so standard checks
must be relaxed on them.
If there were to be any dust checks performed on stake transactions,
it should only be done on the commitment amounts in ticket purchases.
By calculating the output amounts that a revocation would require
creating, the mempool could determine if the revocation outputs would
be considered dust with the current policy and reject the ticket
purchase.
Decred's serialized format for transactions split the 32-bit version
field into two 16-bit components such that the upper bits are used to
encode a serialization type and the lower 16 bits are the actual
transaction version.
Unfortunately, when this was done, the in-memory transaction struct was
not also updated to hide this complexity, which means that callers
currently have to understand and take special care when dealing with the
version field of the transaction.
Since the main purpose of the wire package is precisely to hide these
details, this remedies the situation by introducing a new field on the
in-memory transaction struct named SerType which houses the
serialization type and changes the Version field back to having the
desired semantics of actually being the real transaction version. Also,
since the maximum version can only be a 16-bit value, the Version field
has been changed to a uint16 to properly reflect this.
The serialization and deserialization functions now deal with properly
converting to and from these fields to the actual serialized format as
intended.
Finally, these changes also include a fairly significant amount of
related code cleanup and optimization along with some bug fixes in order
to allow the transaction version to be bumped as intended.
The following is an overview of all changes:
- Introduce new SerType field to MsgTx to specify the serialization type
- Change MsgTx.Version to a uint16 to properly reflect its maximum
allowed value
- Change the semantics of MsgTx.Version to be the actual transaction
version as intended
- Update all callers that had special code to deal with the previous
Version field semantics to use the new semantics
- Switch all of the code that deals with encoding and decoding the
serialized version field to use more efficient masks and shifts
instead of binary writes into buffers which cause allocations
- Correct several issues that would prevent producing expected
serializations for transactions with actual transaction versions that
are not 1
- Simplify the various serialize functions to use a single func which
accepts the serialization type to reduce code duplication
- Make serialization type switch usage more consistent with the rest of
the code base
- Update the utxoview and related code to use uint16s for the
transaction version as well since it should not care about the
serialization type due to using its own
- Make code more consistent in how it uses bytes.Buffer
- Clean up several of the comments regarding hashes and add some new
comments to better describe the serialization types
This removes the function IsSupportedMsgTxVersion function from wire
since that is a policy decision and does not belong in wire.
It also updates the mempool standard transaction checks to be more
inline with the upstream code such that the maximum supported
transaction version is specified via a field in the mempool policy
configuration struct.
Finally, it adds a new test to the standard transaction tests to ensure
transactions that are not serialized with the full seriaization type are
considered nonstandard.
This updates the comments in the isDust function to match the current
default relay fee of 1e5, correct tx size values, and make them
more consistent with the field order. It's nice to have the most recent
values as a reference in the comments.
Also, while here, add a couple of tests just below and above the dust
point for the current default relay fee.
This contains the following upstream commits:
- dc5486a579
- 815ded348e
- c6d50b7abf
The merge commit also includes the contains necessary Decred-specific
alterations.
Upstream commit 15bace88dc.
In addition to the necessary Decred-specific alterations to the existing
upstream code, the merge commit also introduces several new functions on
the fake chain instance to handle the additional capabilities needed by
the Decred mempool such as the NextStakeDifficulty, BlockByHash, and
BestHash functions along with a SubsidyCache instance.
Upstream commit a109bea3f1.
The merge commit also contains necessary Decred-specific alterations for
other uses of the mutex and reduces a bit of defer usage while here
since future upstream commits do the same for other instances.
Upstream commit 641182b2ad.
In addition, the merge commit also adds two more functions to the
config, named BlockByHash and BestHash, along with an additional field
for the SubsidyCache in order to maintain the spirit of the upstream
commit breaking the dependency on a specific chain instance.
The minimum ticket fee is now handled the same as for all other
transactions and it as well as the relative maximum high fee threshold
can be changed using the application config.
When the minimum relay fee was modified from 0.01 to 0.001 DCR/kB this
value was not changed, causing the new high fee threshold to be 0.1
DCR/kB. After discussion, we want to keep the same default high fee
threshold as before, especially as this value is used to limit
unintended high fees for ticket purchases.
This lowers the default minimum relay fee to 0.001 DCR/Kb from its
previous value of 0.01 DCR/Kb.
It should be noted that this is only a default node policy change and as
such does not affect the consensus rules in any way.
Upstream commit 7fac099bee.
Also, the merge commit contains the necessary decred-specific
alterations, adds a new policy parameter for AllowOldVotes to keep with
the spirit of the merged commit, modifies the RawMempoolVerbose to
accept an optional filter type to retain that functionality, and various
other cleanup intended to bring the code bases more in line with one
another.
This renames the mempool.Config.RelayNonStd option to AcceptNonStd which
more accurately describes its behavior since the mempool was refactored
into a separate package.
The reasoning for this change is that the mempool is not responsible for
relaying transactions (nor should it be). Its job is to maintain a pool
of unmined transactions that are validated according to consensus and
policy configuration options which are then used to provide a source of
transactions that need to be mined.
Instead, it is the server that is responsible for relaying transactions.
While it is true that the current server code currently only relays txns
that were accepted to the mempool, this does not necessarily have to
be the case. It would be entirely possible (and perhaps even a good
idea as something do in the future), to separate the relay policy from
the mempool acceptance policy (and thus indirectly the mining policy).
This coincides with the mempool only, policy change which enforces
transaction finality according to the median-time-past rather than
blockheader timestamps. The behavior is pre-cursor to full blown BIP
113 consensus deployment, and subsequent activation.
As a result, the TimeSource field in the mempoolConfig is no longer
needed so it has been removed. Additionally, checkTransactionStandard has been
modified to instead take a time.Time as the mempool is no longer explicitly
dependant on a Chain instance.
This commit adds an additional closure function to the mempool’s config
which computes the median time past from the point of view of the best
node in the chain. The mempool test harness has also been updated to allow
setting a mock median time past for testing purposes.
In addition to increasing the testability of the mempool, this commit
should also speed up transaction and block validation for BIP 113 as
the MTP no longer needs to be re-calculated each time from scratch.
This changes the transaction input standardness checks as follows:
- Allow any script in a pay-to-script-hash transaction to be relayed and
mined so long as it has no more than 15 signature operations
- Remove the obsolete checks which naively calculated the number of
expected inputs in favor of the more accurate ScriptVerifyCleanStack
and ScriptVerifySigPushOnly functionality of the script engine that was
added after these checks.
This adds a basic test harness infrastructure for the mempool package
which aims to make writing tests for it much easier.
The harness provides functionality for creating and signing transactions
as well as a fake chain that provides utxos for use in generating valid
transactions and allows an arbitrary chain height to be set. In order
to simplify transaction creation, a single signing key and payment
address is used throughout and a convenience type for spendable outputs
is provided.
The harness is initialized with a spendable coinbase output by default
and the fake chain height set to the maturity height needed to ensure
the provided output is in fact spendable as well as a policy that is
suitable for testing.
Since tests are in the same package and each harness provides a unique
pool and fake chain instance, the tests can safely reach into the pool
policy, or any other state, and change it for a given harness without
affecting the others.
In order to be able to make use of the existing blockchain.Viewpoint
type, a Clone method has been to the UtxoEntry type which allows the
fake chain instance to keep a single view with the actual available
unspent utxos while the mempool ends up fetching a subset of the view
with the specifically requested entries cloned.
To demo the harness, this also contains a couple of tests which make use
of it:
- TestSimpleOrphanChain -- Ensures an entire chain of orphans is
properly accepted and connects up when the missing parent transaction
is added
- TestOrphanRejects -- Ensure orphans are actually rejected when the
flag on ProcessTransactions is set to reject them
This modifies the config for the new mempool package such that it takes
a callback function to obtain the best chain height instead of requiring
a fully initialized blockchain.BlockChain instance.
This will make it much easier to test the mempool since the tests will
be able to provide their own height function to test various
functionality without having create and manipulate full blocks and chain
instances.
This does the minimum work necessary to refactor the mempool code into
its own package. The idea is that separating this code into its own
package will greatly improve its testability, allow independent
benchmarking and profiling, and open up some interesting opportunities
for future development related to the memory pool.
There are likely some areas related to policy that could be further
refactored, however it is better to do that in future commits in order
to keep the changeset as small as possible during this refactor.
Overview of the major changes:
- Create the new package
- Move several files into the new package:
- mempool.go -> mempool/mempool.go
- mempoolerror.go -> mempool/error.go
- policy.go -> mempool/policy.go
- policy_test.go -> mempool/policy_test.go
- Update mempool logging to use the new mempool package logger
- Rename mempoolPolicy to Policy (so it's now mempool.Policy)
- Rename mempoolConfig to Config (so it's now mempool.Config)
- Rename mempoolTxDesc to TxDesc (so it's now mempool.TxDesc)
- Rename txMemPool to TxPool (so it's now mempool.TxPool)
- Move defaultBlockPrioritySize to the new package and export it
- Export DefaultMinRelayTxFee from the mempool package
- Export the CalcPriority function from the mempool package
- Introduce a new RawMempoolVerbose function on the TxPool and update
the RPC server to use it
- Update all references to the mempool to use the package.
- Add a skeleton README.md
