diff --git a/contrib/README.md b/contrib/README.md index 44394a86..92b0b039 100644 --- a/contrib/README.md +++ b/contrib/README.md @@ -23,3 +23,14 @@ and related software. - [systemd](services/systemd/dcrd.service) Provides an example service file for configuring dcrd as a background service on operating systems that use systemd for service management. + +### Simulation Network (--simnet) Preconfigured Environment Setup Script + +The [dcr_tmux_simnet_setup.sh](./dcr_tmux_simnet_setup.sh) script provides a +preconfigured `simnet` environment which facilitates testing with a private test +network where the developer has full control since the difficulty levels are low +enough to generate blocks on demand and the developer owns all of the tickets +and votes on the private network. + +See the full [Simulation Network Reference](../docs/simnet_environment.mediawiki) +for more details. diff --git a/contrib/dcr_tmux_simnet_setup.sh b/contrib/dcr_tmux_simnet_setup.sh new file mode 100755 index 00000000..da20ff82 --- /dev/null +++ b/contrib/dcr_tmux_simnet_setup.sh @@ -0,0 +1,284 @@ +#!/usr/bin/env bash +# +# Copyright (c) 2020 The Decred developers +# Use of this source code is governed by an ISC +# license that can be found in the LICENSE file. +# +# Tmux script to create 2 dcrd nodes (named dcrd1 and dcrd2) connected in series +# along with 2 wallets (named wallet1 and wallet2) configured such that wallet1 +# is connected via JSON-RPC to dcrd1 and, likewise, wallet2 to dcrd2. +# +# Both wallet1 and wallet2 use the same seed, however, wallet1 is configured to +# automatically buy tickets and vote, while wallet2 is only configured to vote. +# +# The primary dcrd node (dcrd1) is configured as the primary mining node. +# +# Network layout: +# dcrd1 (p2p: localhost:19555) <-> dcrd2 (p2p: localhost:19565) +# +# RPC layout: +# dcrd1 (JSON-RPC: localhost:19556) +# ^--- wallet1 (JSON-RPC: locahost:19557, gRPC: localhost:19558) +# dcrd2 (JSON-RPC: localhost:19566) +# ^--- wallet2 (JSON-RPC: locahost:19567, gRPC: None) + +set -e + +SESSION="dcrd-simnet-nodes" +NODES_ROOT=~/dcrdsimnetnodes +RPCUSER="USER" +RPCPASS="PASS" +WALLET_SEED="b280922d2cffda44648346412c5ec97f429938105003730414f10b01e1402eac" +WALLET_MINING_ADDR="SspUvSyDGSzvPz2NfdZ5LW15uq6rmuGZyhL" # NOTE: This must be changed if the seed is changed. +WALLET_XFER_ADDR="SsonWQK6xkLSYm7VttCddHWkWWhETFMdR4Y" # same as above + +if [ -d "${NODES_ROOT}" ] ; then + rm -R "${NODES_ROOT}" +fi + +PRIMARY_DCRD_NAME=dcrd1 +SECONDARY_DCRD_NAME=dcrd2 +PRIMARY_WALLET_NAME=wallet1 +SECONDARY_WALLET_NAME=wallet2 +mkdir -p "${NODES_ROOT}/${PRIMARY_DCRD_NAME}" +mkdir -p "${NODES_ROOT}/${SECONDARY_DCRD_NAME}" +mkdir -p "${NODES_ROOT}/${PRIMARY_WALLET_NAME}" +mkdir -p "${NODES_ROOT}/${SECONDARY_WALLET_NAME}" + +cat > "${NODES_ROOT}/dcrd.conf" < "${NODES_ROOT}/dcrctl.conf" < "${NODES_ROOT}/wallet.conf" < "${NODES_ROOT}/${PRIMARY_DCRD_NAME}/ctl" < "${NODES_ROOT}/${PRIMARY_DCRD_NAME}/mine" < "${NODES_ROOT}/${PRIMARY_WALLET_NAME}/ctl" < "${NODES_ROOT}/${PRIMARY_WALLET_NAME}/tickets" < "${NODES_ROOT}/${PRIMARY_WALLET_NAME}/xfer" < "${NODES_ROOT}/${SECONDARY_DCRD_NAME}/ctl" < "${NODES_ROOT}/${SECONDARY_DCRD_NAME}/reorg" < "${NODES_ROOT}/${SECONDARY_WALLET_NAME}/ctl" < "${NODES_ROOT}/${PRIMARY_DCRD_NAME}/stopall" </dev/null +cd "${NODES_ROOT}/${PRIMARY_WALLET_NAME}" +./ctl stop 2>/dev/null +cd "${NODES_ROOT}/${SECONDARY_DCRD_NAME}" +./ctl stop 2>/dev/null +cd "${NODES_ROOT}/${SECONDARY_WALLET_NAME}" +./ctl stop 2>/dev/null + +DELAY=3 +countdown \$DELAY "until shutdown" +tmux kill-session -t $SESSION +EOF +cp "${NODES_ROOT}/${PRIMARY_DCRD_NAME}/stopall" "${NODES_ROOT}/${SECONDARY_DCRD_NAME}/" +cp "${NODES_ROOT}/${PRIMARY_DCRD_NAME}/stopall" "${NODES_ROOT}/${PRIMARY_WALLET_NAME}/" +cp "${NODES_ROOT}/${PRIMARY_DCRD_NAME}/stopall" "${NODES_ROOT}/${SECONDARY_WALLET_NAME}/" +chmod +x "${NODES_ROOT}/${PRIMARY_DCRD_NAME}/stopall" +chmod +x "${NODES_ROOT}/${SECONDARY_DCRD_NAME}/stopall" +chmod +x "${NODES_ROOT}/${PRIMARY_WALLET_NAME}/stopall" +chmod +x "${NODES_ROOT}/${SECONDARY_WALLET_NAME}/stopall" + +################################################################################ +# Attach +################################################################################ + +tmux select-window -t $SESSION:0 +tmux attach-session -t $SESSION diff --git a/docs/README.md b/docs/README.md index eec04d17..50932480 100644 --- a/docs/README.md +++ b/docs/README.md @@ -15,6 +15,7 @@ 2. [JSON-RPC Reference](#JSONRPCReference) 3. [Go Modules](#GoModules) 4. [Module Hierarchy](#ModuleHierarchy) +6. [Simulation Network (--simnet) Reference](#SimnetReference) @@ -231,3 +232,22 @@ The following diagram shows an overview of the hierarchy for the modules provided by the dcrd repository. ![Module Hierarchy](./assets/module_hierarchy.svg) + + + +**6. Simulation Network (--simnet)** + +When developing Decred applications or testing potential changes, it is often +extremely useful to have a test network where transactions are actually mined +into blocks, difficulty levels are low enough to generate blocks on demand, it +is possible to easily cause chain reorganizations for testing purposes, and +otherwise have full control over the network. + +In order to facilitate these scenarios, `dcrd` provides a simulation network +(`--simnet`), where the difficulty starts extremely low to enable fast CPU +mining of blocks. Simnet also has some modified functionality that helps +developers avoid common issues early in development. + +See the full reference for more details: + +* [Simulation Network Reference](https://github.com/decred/dcrd/tree/master/docs/simnet_environment.mediawiki) diff --git a/docs/simnet_environment.mediawiki b/docs/simnet_environment.mediawiki new file mode 100644 index 00000000..eefc2b70 --- /dev/null +++ b/docs/simnet_environment.mediawiki @@ -0,0 +1,327 @@ +==1. Overview== + +When developing Decred applications or testing potential changes, it is often +extremely useful to have a private test network where transactions are actually +mined into blocks, difficulty levels are low enough to generate blocks on +demand, it is possible to easily cause chain reorganizations for testing +purposes, and otherwise have full control over the network. + +In order to facilitate these scenarios, dcrd provides a simulation +network (--simnet), where the difficulty starts extremely low to +enable fast CPU mining of blocks. Simnet also has some modified functionality +that helps developers avoid common issues early in development. + +Other components in the Decred ecosystem, such as dcrwallet and +dcrctl, also support the simulation network to facilitate testing. + +As simnet is a temporary network for private use, there are no public block +explorers or faucets available. + +===1.1 Distinguishing Properties=== + +The following is an overview of the most important properties that distinguish +it from the main network: + +* The difficulty starts extremely low to enable fast CPU mining of blocks +* Networking changes: +** All code related to peer discovery and IP address dissemination is disabled to help ensure the network remains private +** The peer and RPC network ports are different +** A unique network byte sequence is used in the peer-to-peer message protocol so the blocks can't accidentally be crossed with the main network +* All chain and payment address parameters are unique to prevent confusion with the main network: +** Different genesis block +** Payment addresses start with different prefixes: +*** Standard pay-to-pubkeyhash (P2PKH) starts with uppercase S +*** Standard pay-to-scripthash (P2SH) starts with lowercase s +** Exported hierarchical deterministic extended keys (BIP32) start with different prefixes: +*** Public extended keys start with spub +*** Private extended keys start with sprv +** The BIP44 coin type used in HD key paths is lowercase s + +==2. Preconfigured Simnet Environment== + +The easiest way to get started using the simulation network is to make use of +the preconfigured simnet environment setup shell script that is +[[../contrib/dcr_tmux_simnet_setup.sh|provided in the repository at contrib/dcr_tmux_simnet_setup.sh]]. + +This script makes use of [https://github.com/tmux/tmux tmux], which is +typically readily available on Unix and Linux platforms, to setup a +self-contained environment with several terminal-based windows and panes with +dcrd and dcrwallet already configured. + +===2.1 Preconfigured Environment Overview=== + +The following provides an overview of how the environment that is automatically +setup by the aforementioned script is configured: + +* Multiple windows, each with two panes +** The upper pane is either a dcrd instance or a dcrwallet instance +** The lower pane provides a shell already in the directory associated with the process running in the upper pane +* Two dcrd instances running simnet (named dcrd1 and dcrd2) +** The second connects to the first via a persistent/permanent connection +* Two dcrwallet instances also running simnet (named wallet1 and wallet2) +** Both wallets are configured with the same seed +** The wallet1 instance: +*** Connects to the dcrd1 instance via JSON-RPC +*** Automatically purchases tickets via the integrated ticket buyer +*** Automatically votes +** The wallet2 instance: +*** Connects to the dcrd2 instance via JSON-RPC +*** Automatically votes +* Helper scripts accessible from the lower pane in each window that communicate with the associated process in the upper pane +* Scripts available in the directories of all instances: +** ctl - Invokes dcrctl communicating to the associated instance process +** stopall - Stops all of the processes and closes the tmux session +* Additional scripts for the dcrd1 and dcrd2 instances: +** mine - Mines a specified number of blocks with a delay between +* Additional scripts for the dcrd2 instance: +** reorg - Forces a chain reorganization in the instance +* Additional scripts for the wallet1 and wallet2 instances: +** tickets - Manually purchases the specified number of tickets with a default of 1 +** xfer - Creates a transaction that transfers (sends) funds to an address owned by the instance + +===2.2. Using the Preconfigured Environment=== + +In order to get started, first ensure that the dcrd, +dcrwallet, and dcrctl binaries you want to test with +are available in your system path and that tmux is installed: + + $ dcrd -V + dcrd version x.y.z (Go version ) + $ dcrwallet -V + dcrwallet version x.y.z (Go version ) + $ dcrctl -V + dcrctl version x.y.z (Go version ) + $ tmux -V + tmux x.y.z + +Once that above has been verified, make the script executable and run it: + + $ chmod +x dcr_tmux_simnet_setup.sh + $ ./dcr_tmux_simnet_setup.sh + +It will take a few seconds before the environment appears since it mines a bunch +of initial blocks so that the environment has spendable coins immediately +available. + +Once the environment launches, you will be in the bottom pane of the +dcrd1 window. As the overview section describes, this lower pane +provides convenience scripts for interacting with the dcrd instance +running in the upper pane. + +At this point, everything is setup and ready to go. Since it is running in a +[https://github.com/tmux/tmux tmux] session, the various windows and panes can +be accessed with normal tmux command sequences. By default, +Ctrl+B followed by a window number will switch to that number. For +example Ctrl+B 1 will switch the window with the +dcrwallet1 instance and Ctrl+B 0 will switch back to +the window with the dcrd1 instance. + +See the next sections for a quick tour of performing some basic tasks. + +====2.2.1 Basic Tasks - Interfacing with the dcrd1 instance==== + +Ensure the window that houses the dcrd1 instance is active and the +lower pane has focus. This '''might''' require switching to the window with the +tmux command sequence Ctrl+B 0 and Ctrl+B +to switch to the pane. + +* Query the current best block of the private network: + $ ./ctl getbestblock + { + "hash": "this will be different each time the environment is launched", + "height": 32 + } + +* To mine a couple of blocks: + $ ./mine 2 + [ + "block hash here" + ] + [ + "different block hash here" + ] + +====2.2.2 Basic Tasks - Interfacing with the wallet1 instance==== + +Ensure the window that houses the dcrwallet1 instance is active and +the lower pane has focus. This '''might''' require switching to the window with +the tmux command sequence Ctrl+B 1 and +Ctrl+B to switch to the pane. + +* Querying the wallet balance: + $ ./ctl getbalance + { + ... + "totalimmaturecoinbaserewards": 4800, + "totallockedbytickets": 0.0006894, + "totalspendable": 4800, + "cumulativetotal": 9899.9999675, + "totalunconfirmed": 299.9992781, + "totalvotingauthority": 0.0006 + } + +* Manually buying tickets: + $ ./tickets 3 + [ + "first ticket hash here", + "second ticket hash here", + "third ticket hash here" + ] + +====2.2.3 Basic Tasks - Forcing a reorg via the dcrd2 instance==== + +Ensure the window that houses the dcrd2 instance is active and the +lower pane has focus. This '''might''' require switching to the window with the +tmux command sequence Ctrl+B 2 and Ctrl+B +to switch to the pane. + +Force the reorg using the reorg convenience script: + + $ ./reorg + [ + "block hash mined in dcrd1 instance" + ] + [ + "second block hash mined in dcrd1 instance" + ] + [ + "block hash mined in dcrd2 instance" + ] + +The log file of dcrd in the upper pane will show something similar +to: + + [INF] CHAN: REORGANIZE: Chain forks at 012419307b328916facfce7120fdb526537b5f3faccd5039e1650deda87832fe (height 34) + [INF] CHAN: REORGANIZE: Old best chain tip was 0613b391b86d07a35c0c1ad09b2d14d0aaecc09e138b2646de4475996b9b24b7 (height 35) + [INF] CHAN: REORGANIZE: New best chain tip is 003af7ab2a8ec1708694ba96ad49038c3c20d5ac01da354125b67469d120f351 (height 36) + +====2.2.4 Basic Tasks - Shutting down the environment==== + +From any instance command pane: + + $ ./stopall + dcrd stopping. + dcrwallet stopping + dcrd stopping. + dcrwallet stopping + Seconds until shutdown: 3 + ... + + + +===2.3. Common Issues=== + +====2.3.1. Code changes don't seem to have any effect==== + +The script runs the binaries from the system path. Make sure you +go install the changed binaries so the updated binaries are the +ones that are being run. + +====2.3.2. Tickets aren't being purchased==== + +This happens when too many blocks are mined too quickly and the wallet can't +keep up. The solution is to use the ./mine script which adds a +delay between each block to give the wallet a chance to purchase and send the +tickets. + +====2.3.3. Rejected ticket purchases with errors similar to rejected transaction X: transaction X expired at height Y==== + +This is caused by the same thing as the previous issue and has the same +solution. + +====2.3.4. Errors similar to list size too small: 0 < 5==== + +This happens if you stop buying tickets and mine enough blocks to the point the +chain is unrecoverable due to the ticket pool becoming depleted. The solution +is to start over and ensure you are purchasing enough tickets. A new chain is +created every time the simnet environment setup script is run. + +Note that this error is temporary and will be replaced with a +chain continuity error in the future. + +====2.3.5. Errors similar to chain continuity not guaranteed after height X==== + +This happens if you stop buying tickets and mining the next block would cause +the chain to become unrecoverable if no tickets are included due to depletion of +the live ticket pool. The solution is to purchase enough tickets so the block +can be mined. + +==3. Manual Configuration== + +'''

+It is _HIGHLY_ recommended to use the previously described preconfigured simnet +environment instead of manually configuring things. Not only does it greatly +simplify things, it also provides a much more sophisticated setup than the +basic one described here. +

''' + +Running a single dcrd node on simnet is simply starting +dcrd with the --simnet flag. However, in order to be +really useful, you'll typically want to be able to send coins amongst addresses +which implies that blocks will need to be mined and interfacing with a wallet +will be needed. + +In addition, since there are effectively no coins yet on the new private +network, an initial series of blocks will need to be mined which pay to an +address you own so there are usable coins to spend. + +As previously mentioned, simnet uses unique addresses to prevent confusion with +the main network. Thus, it means that a wallet which supports the address +format must be used. For this, dcrwallet with the +--simnet flag can be used. + +The following is a command reference to get going: + +'''

+NOTE: All of these commands can be simplified by creating config files and +making use of them, however the commands here use all switches on the +command line to show exactly what is needed for each. +

''' + +* Create a new simnet wallet (this will be called the first window): + $ dcrwallet --simnet --create + Enter the private passphrase for your new wallet: + Confirm passphrase: + Do you want to add an additional layer of encryption for public data? (n/no/y/yes) [no]: + Do you have an existing wallet seed you want to use? (n/no/y/yes) [no]: + Your wallet generation seed is: + + Hex: + ... + Once you have stored the seed in a safe and secure location, enter "OK" to continue: OK + Creating the wallet... + Mining address: S <---- Note this address + The wallet has been created successfully. + +* In a second window, start dcrd on simnet paying to the mining address from above: + $ dcrd --simnet --rpcuser=someuser --rpcpass=somepass --miningaddr=S + +* In a third window, instruct dcrd to generate enough initial blocks for the first coinbase to mature and to have enough funds for purchasing tickets: + $ dcrctl --simnet --rpcuser=someuser --rpcpass=somepass generate 32 + +* Back in the first window, start dcrwallet on simnet: +'''

+NOTE: It is important to perform this step AFTER the blocks have been mined +above or the wallet will believe it's on an old coin type and will not see the +balance properly. +

''' + + $ dcrwallet --simnet --username=someuser --password=somepass + ... + *** ATTENTION *** + Since this is your first time running we need to sync accounts. Please enter + the private wallet passphrase. This will complete syncing of the wallet + accounts and then leave your wallet unlocked. You may relock wallet after by + calling 'walletlock' through the RPC. + ***************** + Enter private passphrase: + ... + [INF] SYNC: Blockchain sync completed, wallet ready for general usage. + +* Back in the third window, check the wallet balance to ensure spendable coins are available: + $ dcrctl --simnet --wallet --rpcuser=someuser --rpcpass=somepass getbalance + +At this point, there is a fully functional private simnet with coins available +to send to other simnet addresses. Any time one or more transactions are sent, +a generate 1 RPC must be issued to mine a new block with the +transactions included.