c1a3d640ef
UtxoCache is an unspent transaction output cache that sits on top of the
utxo set database and provides significant runtime performance benefits
at the cost of some additional memory usage. It drastically reduces the
amount of reading and writing to disk, especially during initial block
download when a very large number of blocks are being processed in quick
succession.
The UtxoCache is a read-through cache. All utxo reads go through the
cache. When there is a cache miss, the cache loads the missing data
from the database, caches it, and returns it to the caller.
The UtxoCache is a write-back cache. Writes to the cache are
acknowledged by the cache immediately but are only periodically flushed
to the database. This allows intermediate steps to effectively be
skipped. For example, a utxo that is created and then spent in between
flushes never needs to be written to the utxo set in the database.
Due to the write-back nature of the cache, at any given time the
database may not be in sync with the cache, and therefore all utxo reads
and writes MUST go through the cache, and never read or write to the
database directly.
An overview of the changes is as follows:
- Add UtxoCache and UtxoCacheConfig struct types and NewUtxoCache method
- Update server to create the utxo cache with the configured max size
and pass to the block chain instance that is created
- Update all test block chains to create a utxo cache
- Add FetchEntry to UtxoCache
- FetchEntry returns the specified transaction output from the utxo
set
- If the output exists in the cache, it is returned immediately.
Otherwise, it uses an existing database transaction to fetch the
output from the database, caches it, and returns it to the caller.
- Add AddEntry to UtxoCache
- AddEntry adds the specified output to the cache
- Add SpendEntry to UtxoCache
- SpendEntry marks the specified output as spent
- Remove entries that are marked as fresh and then subsequently spent.
This is an optimization to skip writing to the database for outputs
that are added and spent in between flushes to the database.
- Update UtxoViewpoint to hold the UtxoCache
- Update fetching entries from the database to fetch entries from the
cache instead
- Add Commit to UtxoCache
- Commit updates all entries in the cache based on the state of each
entry in the provided view
- All entries in the provided view that are marked as modified and
spent are removed from the view
- Additionally, all entries that are added to the cache are removed
from the provided view
- Add MaybeFlush to UtxoCache
- MaybeFlush conditionally flushes the cache to the database
- If the maximum size of the cache has been reached, or if the
periodic flush duration has been reached, then a flush is required
- A flush can be forced by setting the force flush parameter
- Flushing commits all modified entries to the database and
conditionally evicts entries
- Entries that are nil or spent are always evicted since they are
unlikely to be accessed again. Additionally, if the cache has
reached its maximum size, entries are evicted based on the height of
the block that they are contained in.
- Update connect block and disconnect block to commit to the cache and
conditionally flush to the database
- Rather than writing to the utxo set in the database every time that
a block is connected or disconnected, commit the updated view to the
cache and call MaybeFlush on the cache to conditionally flush it to
the database
- Add InitUtxoCache to UtxoCache
- InitUtxoCache initializes the utxo cache by ensuring that the utxo
set is caught up to the tip of the best chain
- Since the cache is only flushed to the database periodically, the
utxo set may not be caught up to the tip of the best chain
- InitUtxoCache catches the utxo set up by replaying all blocks from
the block after the block that was last flushed to the tip block
through the cache
- Add ShutdownUtxoCache to BlockChain
- ShutdownUtxoCache flushes the utxo cache to the database on
shutdown. Since the cache is flushed periodically during initial
block download and flushed after every block is connected after
initial block download is complete, this flush that occurs during
shutdown should finish relatively quickly
- Note that if an unclean shutdown occurs, the cache will still be
initialized properly when restarted as during initialization it will
replay blocks to catch up to the tip block if it was not fully
flushed before shutting down. However, it is still preferred to
flush when shutting down versus always recovering on startup since
it is faster
- Track the hit ratio of UtxoCache
- Track the number of hits and misses when accessing the cache in
order to calculate the overall hit ratio of the cache to gauge its
performance
|
||
|---|---|---|
| .. | ||
| doc.go | ||
| error_test.go | ||
| error.go | ||
| log.go | ||
| mempool_test.go | ||
| mempool.go | ||
| policy_test.go | ||
| policy.go | ||
| README.md | ||
mempool
Package mempool provides a policy-enforced pool of unmined Decred transactions.
A key responsibility of the Decred network is mining transactions – regular transactions and stake transactions – into blocks. In order to facilitate this, the mining process relies on having a readily-available source of transactions to include in a block that is being solved.
At a high level, this package satisfies that requirement by providing an in-memory pool of fully validated transactions that can also optionally be further filtered based upon a configurable policy.
The Policy configuration options has flags that control whether or not "standard" transactions and old votes are accepted into the mempool. In essence, a "standard" transaction is one that satisfies a fairly strict set of requirements that are largely intended to help provide fair use of the system to all users. It is important to note that what is considered to be a "standard" transaction changes over time as policy and consensus rules evolve. For some insight, at the time of this writing, an example of some of the criteria that are required for a transaction to be considered standard are that it is of the most-recently supported version, finalized, does not exceed a specific size, and only consists of specific script forms.
Since this package does not deal with other Decred specifics such as network communication and transaction relay, it returns a list of transactions that were accepted which gives the caller a high level of flexibility in how they want to proceed. Typically, this will involve things such as relaying the transactions to other peers on the network and notifying the mining process that new transactions are available.
Feature Overview
The following is a quick overview of the major features. It is not intended to be an exhaustive list.
- Maintain a pool of fully validated transactions
- Reject non-fully-spent duplicate transactions
- Reject coinbase transactions
- Reject double spends (both from the chain and other transactions in pool)
- Reject invalid transactions according to the network consensus rules
- Full script execution and validation with signature cache support
- Individual transaction query support
- Stake transaction support (ticket purchases, votes and revocations)
- Option to accept or reject old votes
- Orphan transaction support (transactions that spend from unknown outputs)
- Configurable limits (see transaction acceptance policy)
- Automatic addition of orphan transactions that are no longer orphans as new transactions are added to the pool
- Individual orphan transaction query support
- Configurable transaction acceptance policy
- Option to accept or reject standard transactions
- Option to accept or reject transactions based on priority calculations
- Rate limiting of low-fee and free transactions
- Non-zero fee threshold
- Max signature operations per transaction
- Max orphan transaction size
- Max number of orphan transactions allowed
- Additional metadata tracking for each transaction
- Timestamp when the transaction was added to the pool
- Most recent block height when the transaction was added to the pool
- The fee the transaction pays
- The starting priority for the transaction
- Manual control of transaction removal
- Recursive removal of all dependent transactions
License
Package mempool is licensed under the copyfree ISC License.